RigidFormer: Learning Rigid Dynamics using Transformers

Why these properties matter in practice.

Mesh-Free.

• Real-world inputs are typically captured as point clouds (LiDAR, depth cameras, 3D scanners), so a mesh-free formulation removes the brittle "point cloud → mesh reconstruction" preprocessing stage required by mesh-based simulators.
• Operating directly on points is robust to partial occlusion, sparse sampling, and irregular density; mesh-based methods are sensitive to broken or inconsistent topology under such conditions.
• A point-only interface aligns naturally with on-line robotic perception and sim-to-real pipelines, allowing the dynamics loop to consume sensor streams without intermediate geometric reconstruction.
• Point inputs further admit a rich family of training-time augmentations — downsampling, jitter, occlusion masking, and density variation — that can be tailored by the user to improve robustness against imperfect geometric observations at deployment.

Var. Δt (variable integration step).

• The step size becomes a deployment-time knob: large Δt for fast interactive rollout, small Δt for high-fidelity simulation, all within a single trained model.
• Larger Δt reduces the number of autoregressive steps required to cover a fixed horizon, lowering both cumulative error and inference cost in long-horizon rollouts.
• For world-model use cases, planning rarely requires fine-grained dynamics; coarser steps yield more future information per unit of compute, which directly accelerates downstream planning and decision-making.

Preproc.-Free (no offline geometry computation).

• Methods such as SDF-Sim require several hours of per-dataset SDF pre-learning, while HopNet's simplicial-complex construction takes on the order of fifteen days; these costs must be paid again for every new scene or dataset, hindering practical scalability.
• Eliminating this stage enables immediate deployment to previously unseen geometries and removes a major bottleneck in scaling training data.

#Warmup Frames ↓ (fewer is better).

• Warmup frames are the historical observations the model must consume before producing its first prediction.
• A smaller warmup horizon translates directly into lower cold-start latency: prediction can begin after only one or two observations.
• For real-time control loops such as robot MPC and interactive simulation, this is essential — waiting three to five frames before issuing the first action is rarely acceptable.
• RigidFormer requires only two warmup frames, compared to three for FIGNet, SDF-Sim, and HopNet, enabling faster startup in streaming scenarios.

Runtime (FPS) ↑ (higher is better).

• Higher throughput is universally desirable: it expands the set of feasible deployment scenarios — interactive simulators, real-time game physics, high-throughput RL rollouts, and the inner loops of robot MPC — and allows more episodes, longer horizons, or more parallel scenes on identical hardware.