SLAT-Phys: Fast Material Property Field Prediction
from Structured 3D Latents

Rocktim Jyoti Das1, Dinesh Manocha1
1University of Maryland, College Park
Paper

SLAT-Phys predicts spatially varying material fields from a single RGB image and runs physics-based simulation — 120× faster than prior methods.

Flower Vase

Rubber Duck

Snow Man

Abstract

Estimating material property fields of 3D assets is critical for physics-based simulation, robotics, and digital twin generation. Existing vision-based approaches are either too expensive and slow or rely on 3D information. We present SLAT-Phys, an end-to-end method that predicts spatially varying material property fields of 3D assets directly from a single RGB image without explicit 3D reconstruction.

Our approach leverages spatially organized latent features from a pretrained 3D asset generation model (TRELLIS) that encode rich geometry and semantic priors, and trains a lightweight neural decoder to estimate Young’s modulus (E), density (ρ), and Poisson’s ratio (ν). The coarse volumetric layout and semantic cues of the latent representation enable accurate material estimation. SLAT-Phys requires only ∼9.9 seconds per object on an NVIDIA RTX A5000 GPU and avoids reconstruction and voxelization preprocessing, resulting in a ∼120× speedup compared to prior methods.

Method

SLAT-Phys pipeline overview
Figure 1. Overview of SLAT-Phys. Given a single RGB image, the TRELLIS encoder produces a Structured LATent (SLAT) representation. Two parallel decoders predict the 3DGS geometry and spatially varying material properties (E, ρ, ν) at each voxel. The resulting geometry and material fields are passed to an MPM physics solver for simulation.

How it works

1

SLAT Feature Extraction

A single RGB image is encoded by the frozen TRELLIS transformer into Structured LATent (SLAT) features — sparse voxel coordinates with 8-dimensional latent vectors on a 643 grid.

2

Physics Decoder

A lightweight sparse Swin-style transformer decoder maps SLAT features directly to per-voxel material properties: Young’s modulus, density, and Poisson’s ratio, plus a discrete material class label.

3

Geometry Decoding

In parallel, the same SLAT representation is decoded into a 3D Gaussian Splatting (3DGS) representation, providing simulation particles whose geometry and appearance faithfully match the input image.

4

MPM Physics Simulation

Material parameters are transferred to Gaussian particles via nearest-neighbor interpolation and passed to an MPM solver to simulate physically plausible deformation under external forces.

Simulation Results

Physics-based simulations of diverse objects using material properties predicted by SLAT-Phys.

Flower

Flower Vase

Rubber Duck

Rubber Teddy

Snow Man

The flower vase example (top right) illustrates spatially varying material fields: the deformable leaves receive low stiffness while the rigid vase receives high Young’s modulus.

Comparison with Prior Methods

Quantitative comparison with prior methods
Table 1. (a) Runtime comparison: SLAT-Phys achieves ∼9.9 s per object versus ∼20 min for NeRF2Physics and Pixie, a 120× speedup. (b) Material property prediction accuracy on the PixieVerse benchmark.

9.9s

per object
(NVIDIA RTX A5000)

120×

faster than
prior methods

No

per-object reconstruction
preprocessing

Key Contributions

  • Single-image material field regression. The first approach to directly regress continuous, spatially varying material property fields from a single RGB image without explicit 3D reconstruction or multi-view aggregation.
  • Physics-informative structured latents. We demonstrate that SLAT features learned by large-scale 3D generation models encode physically meaningful information beyond geometry, enabling accurate estimation of Young’s modulus, density, and Poisson’s ratio.
  • Simulation-ready digital twins at scale. SLAT-Phys enables faster simulation-ready digital twin generation from a single image, achieving a ∼120× speedup compared to prior reconstruction-based pipelines while maintaining competitive prediction accuracy.

BibTeX

@article{das2025slatphys,
  title     = {{SLAT-Phys}: Fast Material Property Field Prediction
               from Structured {3D} Latents},
  author    = {Das, Rocktim Jyoti and Manocha, Dinesh},
  year      = {2026},
  institution = {University of Maryland, College Park},
}