DeforM: Reasoning-Guided Physics-Aware Video Generation via Spatial-Temporal Masking

Yunyi Li¹, Yu Qiao¹, Yaohui Wang¹, Xinyuan Chen^1*

¹ Shanghai Artificial Intelligence Laboratory

* Corresponding author

TL;DR

We propose DeforM, a reasoning-guided image-to-video framework that generates realistic deformation dynamics by localizing physics-critical regions through VLM-guided reasoning. Our method offers both a training-free strategy for mechanism analysis and a powerful training-based generator, significantly enhancing physical consistency and visual quality in deformation scenarios.

🎥 DeforM-Injection Results

A colorful cylindrical sponge being gradually flattened by a hydraulic press.

A chocolate vanilla ice cream is melting.

A stick of butter melting rapidly in a black pan over heat, with a pool of oil forming around its edges.

A CD disc is heated in a black dry pan, gradually melting into a flowing liquid.

A circular slicer segments a block of cheese into eight uniform triangular pieces.

A transparent gel embedded with colored beads is being continuously stretched.

A car wheel moving forward over a brick, crushing a plastic pumpkin toy until it shatters into pieces.

A carrot-shaped sponge is being sliced into strips with a knife.

🎬 DeforM-Free Results

A single scoop of blue ice cream on a clean white plate melts gradually, with vibrant blue liquid pooling around its base.

A blue and green sponge is crushed by a hydraulic press, gradually flattening under the immense pressure.

A row of multi-colored crayons is heated and melts gradually, with vibrant liquid wax dripping and flowing down the edges.

A white plastic sponge is continuously crushed and squeezed by a heavy hydraulic press.

Abstract

Video generation models achieve high visual quality but often struggle to generate physics-aware videos. Unlike rigid-body motion, which can be described by explicit trajectories or formulas, complex deformation dynamics remain challenging to synthesize. We observe that a lack of physical reasoning for localizing dynamic areas allows irrelevant regions to dilute the model's attention and leads to generation failure. In this paper, we propose DeforM, a reasoning-guided image-to-video generation framework that directs the model's focus toward physics-critical regions. To reason and localize these critical regions, we introduce a VLM-guided physical reasoning module, DeforM-Reason, to identify target objects and generate spatial-temporal masks. For physical guidance, we develop two alternative strategies: DeforM-Free for training-free mechanism analysis, and DeforM-Injection as a powerful training-based generator. Experimental results demonstrate that DeforM improves the realism of generated deformation scenarios, outperforming baseline models in both visual quality and physical consistency.

More Results