WorldAlign: Decoupled 4D Reward for World-Consistent Video Generation

Jing He1,3 Kaixin Ding3,4 Xingye Tian3 Guibao Shen1 Wenhang Ge1 Xin Tao3 Pengfei Wan3 Ying-Cong Chen1,2,✉

1The Hong Kong University of Science and Technology (Guangzhou) 2The Hong Kong University of Science and Technology

3KlingAI 4The University of Hong Kong

✉ Corresponding author

arXiv

A world should stay coherent as it moves.
WorldAlign improves static scene structure and dynamic subject consistency through decoupled world-prior alignment.

Swipe horizontally to inspect the complete teaser.

Two annotated comparisons: the base model loses cabinetry and deforms the oven, and changes train-car appearance and direction. WorldAlign preserves scene structure and plausible motion.

Comparison between WorldAlign and the base model (Wan2.1-I2V-14B-480P). The first case highlights improvements in static consistency: the base model loses the cabinetry (orange arrows), while the oven handle disappears and its door deforms before vanishing (magenta arrows). The second case highlights improvements in dynamic consistency: the train cars abruptly change color and shape and move opposite to the locomotive's heading (see the cars marked by same-colored arrows). WorldAlign instead preserves static scene structure and plausible dynamic evolution.

What is WorldAlign?

Abstract

Faithful visual world simulation requires generated videos to maintain 4D world consistency, encompassing both static and dynamic consistency. Static consistency requires coherent 3D structure in static environments across viewpoints, while dynamic consistency requires plausible subject motion and consistent appearance over time. Geometry-aware post-training offers a promising way to improve world consistency. However, existing methods often rely on a static-scene assumption. Even those that accommodate dynamic scenes struggle to provide reliable static-consistency feedback, while dynamic consistency is often overlooked or inadequately assessed. To address these limitations, we introduce WorldAlign, a decoupled 4D reward framework that semantically separates static regions and dynamic subjects and provides feedback by aligning each with a world prior suited to its assumptions. For static regions, WorldAlign aligns static geometry with a geometric world prior through semantically guided masked reprojection, enabling more reliable static-consistency evaluation; an auxiliary camera-motion reward discourages nearly static solutions. For dynamic subjects, WorldAlign uses a strong vision-language model (VLM) as a dynamic world prior and constructs a VLM-as-a-judge reward based on sample-specific checklists that assess dynamicity, physical plausibility, shape, and texture consistency. This decoupled design enables more effective online post-training without requiring human preference annotations. Across two pretrained image-to-video generators, Wan2.1 and Wan2.2, WorldAlign jointly improves static and dynamic consistency over existing methods without suppressing overall or subject motion. These results support decoupled world-prior alignment for more faithful visual world simulation.

How it works

Different regions. The right world prior.

Separate static environments from dynamic subjects, then give each the feedback it needs.

Swipe the overview horizontally, or tap to open the full image.

WorldAlign pipeline: sample videos from a pretrained generator, semantically separate regions, score static regions with masked reprojection and camera motion, score dynamic subjects with a VLM checklist, and aggregate normalized rewards for a policy update.
WorldAlign uses semantic segmentation and tracking to separate the two branches. Group-normalized rewards are combined for online policy optimization, without human preference annotations or changes to the generator architecture.

01Preserve static geometry

A geometric world prior reconstructs and reprojects the scene. Semantic masks exclude moving subjects while retaining inconsistent static regions, so legitimate subject motion is not penalized.

02Evaluate dynamic subjects

A strong vision-language model judges sample-specific checklists for dynamicity, physical plausibility, shape consistency, and texture consistency.

03Keep the world moving

An auxiliary camera-motion reward discourages nearly static solutions. The combined feedback improves consistency while preserving camera and subject motion.

How well does it work?

Quantitative results

Evaluated on two pretrained image-to-video generators, with both reward-related and independent consistency metrics.

Consistency improves while motion is preserved. On Wan2.1, WorldAlign raises the checklist “Yes” rate from 0.760 to 0.856 and Dynamic Degree from 0.62 to 0.74, while reducing masked epipolar error from 0.469 to 0.264.

Wan2.2-TI2V-5B

Static consistencyDynamic consistencyVBench
MethodM.Epi. ↓M.Reproj. ↓Subject DINO ↑Checklist “Yes” ↑Dynamic Degree ↑Motion Smooth. ↑Aesthetic Quality ↑
Base0.4630.0400.9220.697 (0.63)0.630.9570.479
Epipolar-DPO0.2680.0220.9610.646 (0.43)0.310.9750.481
VideoGPA0.3080.0240.9480.652 (0.56)0.540.9710.476
World-R10.4320.0380.9450.657 (0.61)0.560.9650.459
GeoFlow0.3670.0310.9340.736 (0.74)0.670.9710.478
WorldAlign (Ours)0.3210.0260.9380.781 (0.85)0.720.9790.488

Wan2.1-I2V-14B-480P

Static consistencyDynamic consistencyVBench
MethodM.Epi. ↓M.Reproj. ↓Subject DINO ↑Checklist “Yes” ↑Dynamic Degree ↑Motion Smooth. ↑Aesthetic Quality ↑
Base0.4690.0370.9180.760 (0.81)0.620.9630.481
Epipolar-DPO0.2150.0190.9670.594 (0.53)0.330.9760.494
VideoGPA0.2430.0220.9630.608 (0.61)0.420.9700.489
World-R10.4280.0340.9510.636 (0.72)0.530.9620.440
GeoFlow0.3360.0280.9360.807 (0.83)0.650.9750.485
WorldAlign (Ours)0.2640.0240.9390.856 (0.92)0.740.9780.491

Reading the table. ↓ Lower is better; ↑ higher is better. Bold and underlined entries mark best and second-best values as in the paper. Gray values indicate results that should be interpreted alongside reduced motion. Parentheses in Checklist “Yes” report the subject Dynamicity “Yes” rate. M.Reproj. and Checklist are reward-related; M.Epi. and Subject DINO are reward-independent. Consistency scores should always be read together with Dynamic Degree.

Swipe tables horizontally to see all metrics.

See the difference

Qualitative comparisons

Explore the selected video results. Look for stable surroundings, plausible subject motion, and consistent appearance over time.

Swipe each comparison horizontally to inspect all six methods.

01 /

Camera pans right to follow the horse. A girl on a white horse rides through a sandy jumping arena.

Base ModelWorldAlignEpipolar-DPOVideoGPAWorld-R1GeoFlow

832 × 480 per method · 81 frames · 16 fps · Six methods, synchronized in one video

02 /

Camera tracks forward behind the pilot. A paraglider pilot runs forward to launch over a vast valley.

Base ModelWorldAlignEpipolar-DPOVideoGPAWorld-R1GeoFlow

832 × 480 per method · 81 frames · 16 fps · Six methods, synchronized in one video

03 /

Camera tracks left alongside the railway. A vintage train moves forward through a rugged mountain landscape.

Base ModelWorldAlignEpipolar-DPOVideoGPAWorld-R1GeoFlow

832 × 480 per method · 81 frames · 16 fps · Six methods, synchronized in one video

04 /

Camera pans left across the woodland. A zebra slowly walks through the dry grass.

Base ModelWorldAlignEpipolar-DPOVideoGPAWorld-R1GeoFlow

832 × 480 per method · 81 frames · 16 fps · Six methods, synchronized in one video