CausalMotion: Structured Physical Reasoning as Keyframe and Trajectory Guidance for Training-Free Video Generation

Sihan Zhuang1,2*, Xinyuan Chen2†, Tianfan Xue3,2†, Yaohui Wang2†
1University of Science and Technology of China 2Shanghai Artificial Intelligence Laboratory 3The Chinese University of Hong Kong
*Work done during internship at Shanghai Artificial Intelligence Laboratory †Corresponding author

A training-free framework that injects explicit physical reasoning into video generation through structured keyframes and object-centric trajectories.

arXiv Code Overview Method Results Videos

Overview

Overview of CausalMotion

CausalMotion first performs explicit physical reasoning to decompose a text prompt into causally grounded keyframes and object-centric trajectories. These structured representations are then used as guidance to steer a pretrained video diffusion model during inference, improving physical plausibility and temporal consistency without training.

Abstract

Recent video diffusion models can generate visually compelling videos, yet they often struggle with physically grounded motion and long-horizon interactions. We present CausalMotion, a training-free framework that introduces explicit physical reasoning into video generation. Given a text prompt, CausalMotion uses a multimodal reasoning model to infer causally consistent keyframes and object-centric motion trajectories, providing structured guidance for a pretrained video diffusion model. Rather than replacing the latent representation with trajectory-conditioned features, we perform localized latent updates at selected denoising steps, allowing the model to preserve its generative prior while following the planned motion. Experiments show that CausalMotion improves physical plausibility and temporal consistency across diverse physically grounded video generation scenarios.

Method

CausalMotion separates physical reasoning from video synthesis. Given a text prompt, it first plans causally grounded keyframes and object-centric motion, and then uses these structured representations to guide a pretrained video diffusion model during inference.

01

Physical Reasoning

A multimodal reasoning model decomposes the prompt into causally meaningful key states and predicts the states of relevant objects.

↓
02

Trajectory Construction

Sparse object states are connected into dense, physically grounded trajectories while preserving the planned interactions.

↓
03

Latent Guidance

Keyframes and trajectories are injected through localized latent updates at selected denoising steps, guiding the pretrained diffusion model without additional training.

Architecture of CausalMotion

Overview of the CausalMotion pipeline. Physical reasoning produces structured keyframes and object-centric trajectories, which are subsequently used to guide video generation at inference time.

Results

CausalMotion improves physical consistency while preserving the generative capabilities of pretrained video diffusion models. We evaluate it on both physics-focused benchmarks and general video quality.

Physics-Grounded Video Generation

On PhyGenBench, CausalMotion achieves strong performance across mechanics, optics, thermal, and material phenomena.

Quantitative results on PhyGenBench

General Video Quality

Evaluation with VBench-VLM shows that the proposed guidance maintains general video quality while introducing explicit physical constraints.

VBench-VLM evaluation

Intermediate Results

CausalMotion explicitly plans the physical evolution of objects before video synthesis. The following example illustrates how a text prompt is converted into structured keyframes and object-centric trajectories, which are subsequently aligned with the generated video.

Prompt

A basketball is thrown upward, reaches its highest point, and then falls back down.

01

Keyframe Reasoning

The reasoning model identifies causally meaningful states that describe the evolution of the physical event.

Keyframe 0 Initial state
Keyframe 1 Ascending
Keyframe 2 Bouncing
Keyframe 3 Highest point
02

Trajectory Planning

Object states are transformed into a continuous trajectory that provides explicit spatial and temporal motion guidance.

Show trajectory JSON 
"Frames": {
 "1":  [{"id": 0, "name": "basketball", "box": [255, 93, 204, 194]}],
 "2":  [{"id": 0, "name": "basketball", "box": [255, 167, 204, 194]}],
 "7":  [{"id": 0, "name": "basketball", "box": [255, 380, 204, 150]}], // Collision
 "13": [{"id": 0, "name": "basketball", "box": [255, 93, 204, 194]}]  // Rebound
}
03

Keyframe-Trajectory Alignment

The planned motion is aligned with the denoising process to guide the generated video toward the intended physical evolution.

Qualitative Comparisons

We compare CausalMotion with representative video generation methods across challenging physical interactions. Each example uses the same text prompt for all methods.

01

Liquid Interaction

Milk is poured into a cup of black coffee, producing a visible mixing process.

Wan2.2
LTX-Video
Helios
CausalMotion
02

Object–Fluid Interaction

A pencil is dipped into water, showing the interaction between the object and the surrounding fluid.

Wan2.2
VLIPP
PhyT2V
CausalMotion
03

Long-Horizon Interaction

A burning ball of paper is thrown into a pile of dry paper.

Wan2.2
VLIPP
PhyT2V
CausalMotion

More Examples Gallery

A diverse collection of physical scenarios simulated by CausalMotion.

Air being gradually extracted from a thin, sealed balloon.
A group of oil droplets is floating on the surface of water in the sunlight.
A tennis ball placed gently on the water surface.
A frail porcelain plate flung at a wooden table and shattering.
A vibrant, elastic beach ball is thrown forcefully towards the ground, capturing its dynamic interaction with the surface upon impact.
A glistening dewdrop sliding across a waxed apple.
A timelapse of a water-filled soft cloth being forcefully squeezed by hand, with the pressure intensifying rapidly over time.
A chameleon extending its tongue to capture a flying insect.
Pouring grapefruit juice into a water-filled glass.
Two pillows on a table and two grabber tools hanging above them from which a brown tennis ball and an orange block are suspended. The grabber tools let go of the ball and block.
A timelapse captures the transformation as mist in a rainforest comes into contact with a cold metal surface.
A timelapse captures the gradual transformation of a piece of ice as the temperature rises significantly.

BibTeX

@misc{zhuang2026causalmotionstructuredphysicalreasoning,
  title={CausalMotion: Structured Physical Reasoning as Keyframe and Trajectory Guidance for Training-Free Video Generation}, 
  author={Sihan Zhuang and Xinyuan Chen and Tianfan Xue and Yaohui Wang},
  year={2026},
  eprint={2606.14317},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2606.14317}, 
}