CausalAlign: Causal Alignment of Video Generation Models for Consistent Ego-Motion

Kai Zhu^1,* Yintianrun Zhang^1,* Shuwei Jiang¹ Ruolin Zuo¹ Yixin Chen² Duomin Wang³ Miao Liu⁴ Yin Li⁵ Daquan Zhou¹ Yiwu Zhong^1,†

¹Peking University ²Beijing Institute for General Artificial Intelligence ³NVIDIA ⁴Tsinghua University ⁵University of Wisconsin–Madison

^*Equal contribution ^†Corresponding author

Paper Hugging Face Code Demo

Demo

Abstract

Diffusion-based video generation models (VGMs) can synthesize visually compelling content, but maintaining geometrically consistent camera ego-motion across frames remains an open problem. Camera ego-motion describes these cross-frame viewpoint changes; geometrically inconsistent future predictions can induce systematic errors in downstream action prediction. To address this problem, we propose CausalAlign, a framework combining causal diagnosis and delta-token alignment.

First, we diagnose how ego-motion is represented, used, and retained within VGMs using linear probes and causal interventions. In the base model, both measures rise and then fall across denoising layers and timesteps: ego-motion is used at intermediate stages but is not retained later, a pattern we call geometric state forgetting. Second, we construct a delta-token space from a 3D foundation model and align VGM representations where ego-motion begins to fade. Delta tokens encode changes between consecutive frames, focusing supervision on motion rather than redundant static scene content.

With Wan2.2 fine-tuned on DL3DV and RealEstate10K, CausalAlign preserves visual quality while improving motion accuracy, geometric consistency, and temporal smoothness. For example, it reduces geometric consistency errors by more than 20% relative to the base model. Moreover, using the fine-tuned VGM, CausalAlign improves downstream world action models (WAMs) on the RoboCasa365 benchmark. Overall, CausalAlign improves ego-motion consistency in diffusion-based VGMs and yields gains in downstream embodied models on motion-intensive tasks.

Method

CausalAlign identifies where a video generation model begins to forget ego-motion, then aligns those internal states with motion-centric delta tokens from a 3D foundation model.

Three-stage CausalAlign pipeline: build teacher delta tokens, train a projector, and fine-tune causally selected VGM layers. — The dashed Align paths connect the motion-centric teacher space, the projector, and the selected VGM layers.

Build the teacher delta-token space

Encode changes between frames with a 3D foundation model, separating ego-motion from static scene content.

Train the VGM projector

Map VGM feature transitions into the teacher delta-token space, then freeze the projector.

Fine-tune selected VGM layers

Apply delta-token supervision through LoRA adapters at the layers identified by causal diagnosis.

Quantitative Results

The diagnosis tracks internal ego-motion across denoising. Camera-motion benchmarks then measure geometry and temporal consistency, while visual-quality scores check that the gains do not come from degraded appearance.

Geometric state retention

Wan2.2 probe decodability and causal effect rise and fall, while CausalAlign retains higher values in later layers and timesteps. — Probe decodability and causal intervention effects across transformer layers and denoising timesteps. CausalAlign shifts the pattern from rise-and-fall toward rise-and-maintain.

Camera ego-motion consistency

Higher is better for proximity and smoothness; lower is better for geometric errors and jerk.

Method	Motion magnitude		Geometric consistency			Temporal dynamics
Method	Trans. prox. ↑	Angular prox. ↑	Reproj. ↓	Depth–flow ↓	Epipolar ↓	Smoothness ↑	Jerk ↓
Base model
Wan2.2	0.68	0.70	8.4	6.9	2.9	0.71	0.182
External 3D cache
GEN3C	0.63	0.75	7.8	6.5	2.8	0.71	0.175
Voyager	0.69	0.73	7.1	6.8	2.8	0.73	0.179
Preference alignment
VideoGPA	0.66	0.78	8.4	6.7	2.5	0.73	0.176
Epipolar-DPO	0.68	0.72	7.8	5.9	2.2	0.78	0.162
Direct feature alignment
JOG3R	0.70	0.75	7.2	5.9	2.5	0.79	0.158
Geometry Forcing	0.69	0.78	6.9	5.7	2.4	0.80	0.164
FantasyWorld	0.72	0.77	7.0	5.4	2.3	0.82	0.159
CausalAlign (Ours)	0.79	0.89	6.5	5.5	2.1	0.88	0.151

Visual quality preservation

VBench-I2V
Method	Quality ↑	I2V score ↑	Smooth. ↑
Wan2.2	0.805	0.958	0.983
CausalAlign	0.806	0.958	0.985

Other visual metrics
Method	FVD ↓	CLIP ↑	SSIM ↑	LPIPS ↓
Wan2.2	182.4	0.311	0.742	0.183
CausalAlign	182.7	0.314	0.744	0.183

Visual quality remains comparable to the Wan2.2 base model under matched prompts and sampling settings.

Qualitative Results

More consistent camera trajectories

Two scenes compare generated frames and camera trajectories. CausalAlign shows less trajectory jitter in the straight scene and less loop-closure drift in the looping scene. — The straight-motion example shows less trajectory jitter. In the loop example, CausalAlign ends closer to the origin, indicating less loop-closure drift.

Successful mobile manipulation

Two rollout sequences are stacked so each frame remains legible.