SimForcing

Distilling Simulation Motion Priors
into Real-Domain Robot World Models

Xiaodong Wang · Tianle Li · Chuanxin Song · Junliang Xie · Zhanmi Zhong · Suiying Wu · Peixi Peng

Peking University

Paper ↗GitHub ↗

Comparison of training paradigms for action-conditioned robot world models: direct training on real videos, training with explicit visual priors, and SimForcing motion-prior distillation from a simulation world model. — **From simulation motion to real-world prediction.** SimForcing distills motion priors from a simulation world model into a real-domain robot world model through latent-space motion supervision.

ONE STUDENT. TWO DOMAINS.

From motion priors to visual prediction.

Explore the galleries ↗

01 / THE METHOD

Distill motion.
Guide generation.

Simulation provides structured robot motion, but its appearance differs from real video. SimForcing transfers temporal changes in latent space and uses predicted simulation latents as an adjustable reference for real-domain prediction.

Learn in simulation

Replay and perturb robot action trajectories to train a simulation world model. Video and action transformers interact through Mixture-of-Transformers attention.

Distill motion priors

A frozen simulation teacher supervises changes between adjacent video latents. The student learns motion knowledge while training on both simulation and real videos.

Control the guidance

Simulation latents enter multiple transformer blocks with condition dropout. At inference, classifier-free guidance balances the student's internal motion priors and predicted simulation conditions.

SimForcing training: a simulation world model initializes and supervises a student through latent motion distillation, multi-block simulation conditioning, and flow matching in both domains. — **Training overview.** The simulation teacher is used during training. The final student generates its own simulation conditions and real-domain predictions at inference.

AT INFERENCE

A shared model, two passes.

Initial observation
+ action trajectory

Simulation prediction
student · condition off

Real-world prediction
same student · guided

02 / QUALITATIVE RESULTS

See SimForcing in action.

Loading galleries…