Recursive Context Allocation · Preprint 2026

ReCA: Multi-Shot Long Video Extrapolation via Recursive Context Allocation.

ReCA is an inference-time framework that expands an observed visual anchor into a coherent, cinematically structured multi-shot continuation — preserving identity, scene, object, and event-causality state by allocating context hierarchically across Plan, Allocate, and Refresh operators.

Akide Liu^1,2,3‡ Jinbo Xing^2† Chaojie Mao² Ye Li³ Zeyu Zhang³ Yefei He³ Weijie Wang³ Zihan Wang⁴ Yu Liu² Gholamreza Haffari¹ Bohan Zhuang^3†

¹Monash University · ²Tongyi Lab, Alibaba Group · ³Zhejiang University · ⁴University of Queensland
^†Corresponding Authors · ^‡Intern at Tongyi Lab, Alibaba Group

Paper Code Interactive Demo BibTeX

Code release in progress of internal review — repository will be populated at github.com/ali-vilab/ReCA

Abstract

Long-video failure is not a context-length problem — it is a context-allocation problem.

Minute-scale cinematic video generation is a central challenge for generative video models. Existing paradigms address only fragments of this challenge: single-shot extrapolation preserves an anchor but lacks cinematic structure, while multi-shot storytelling imposes structure yet remains free to invent its visual states rather than continue an observed one. We define Multi-Shot Video Extrapolation (MSVE), a task that extends an observed frame or clip into a sequence of cinematically structured shots while preserving anchor state and advancing narrative intent, under the finite per-call generation budget of short-video models.

We identify three coupled bottlenecks: (1) global planners over-specify unsupported details from full screenplays; (2) shot-level prompts dilute task-relevant state when carrying the complete story; (3) temporal chaining turns generated frames into a lossy memory in which identity, scene, object, and action state decay. MSVE reveals that long-video failure is not merely a limitation of context length, but a failure of context allocation.

We propose Recursive Context Allocation (ReCA), an inference-time framework that allocates context hierarchically across planning and generation. ReCA recursively decomposes MSVE into context-bounded subproblems, invokes frozen generators at leaf nodes, and propagates structured state updates across time. To evaluate this setting, we further propose MSVE-Bench and NB-Q, a source-grounded protocol with prompts purpose-built for 3–5 minute long-video generation. Compared to previous methods, ReCA improves average normalized score by 8–16% over the strongest competing controller and improves multi-shot consistency metrics by 28–43%.

Context allocation failures in MSVE: three diagnostic panels showing global planning overload, shot-level prompt dilution, and stale temporal chaining. — **Context allocation failures in MSVE.** The usable context of a frozen video generator is narrower than its nominal window. The three panels diagnose the three bottlenecks: *(a) global planning overload* — over-detailed plans introduce unsupported constraints; *(b) shot-level dilution* — longer per-shot prompts reduce the salience of state needed by the current call; and *(c) temporal chaining* — generated frames become a lossy, aging memory of identity, scene, object, and action state. Additional context helps only while it raises the density of *usable* state.

Task Definition

Multi-Shot Video Extrapolation (MSVE).

MSVE extends an observed frame or clip into a sequence of cinematically structured shots while preserving anchor state and advancing narrative intent — under the finite per-call generation budget of short-video models.

MSVE task overview: an observed visual anchor is extended into a multi-shot continuation by a short-clip generator under bounded per-call duration and prompt budgets. — **MSVE task overview.** Existing paradigms address only fragments of the minute-scale generation challenge: single-shot extrapolation preserves an anchor but lacks cinematic structure, while multi-shot storytelling imposes structure yet remains free to invent its visual states rather than continue an observed one. MSVE bridges these two regimes — it carries the observed anchor's identity, scene, and object state forward across a cinematically structured continuation that fits inside the per-call duration and prompt budgets of short-video generators.

Capability Chapters

ReCA decomposes multi-shot video extrapolation into context-bounded shot jobs.

The method keeps task state outside the frozen video generator, plans the long continuation as a tree of shots, allocates a per-shot context window to each leaf, and writes the tail frame back into state so the next leaf can pick up where the last one left off.

ReCA framework: recursive planning of the long video into context-bounded subproblems, leaf execution by the frozen generator, parallel/sequential leaf-execution logic, and adaptive state update feeding back to the planner. — **Recursive Context Allocation (ReCA).** Four phases the recursion induces: (1) *recursive planning* of the long-video task into context-bounded subproblems; (2) *leaf execution* where each leaf calls the frozen generator to produce one segment; (3) *leaf-execution logic* that runs independent leaves in parallel and chains boundary-conditioned ones sequentially; and (4) *adaptive state update* that extracts, verifies, and refreshes external state, then feeds back to the planner.

Terminology

Shot, segment, and parallel execution.

ReCA separates the movie timeline from the short-video calls that render it, then decides which units need a boundary handoff and which can run at the same time.

Shot

A cinematic unit in the final long video. The system chooses its goal, duration, boundary condition, and preserved state.

Segment

An executable leaf call to the frozen short-video generator. A long shot can be divided into several budgeted segments.

Parallel Execution

Parallelism is at the shot level: shots whose anchor frames are independent render together through the frozen generator. Segments inside a single shot stay serial — each segment's first frame is the previous segment's tail.

01 / Plan

Root-to-shot recursive planning

Starting from multiple visual anchors and narrative intent, ReCA builds a shot schedule with semantic goals, durations, boundary conditions, and dependency links. The schedule keeps each generator call inside prompt and duration budgets instead of sending the whole story at once.

02 / Allocate

Local-global context allocation

Each shot receives a compact context slice — character / location identity references, the anchor frame, and the most recent state-memory entries — packed into a per-shot system message. The planner never sees the whole movie at once, so per-call token budgets stay bounded.

03 / Render

Shot-level parallel rendering

Once contexts are assigned, each shot is an independent leaf job: render the anchor frame first, then walk its segments serially — each segment's first frame is the previous segment's tail.

Shot Preview / Timeline

Shot Preview Plan for the 18-shot Heavenly Titans sequence.

The sequence expands a single duel into cloud combat, titan-scale transformations, and a final standoff while keeping the same characters, weapons, and action state readable.

Featured ReCA Cases

Multi-shot continuations across landmarks, action, food, and dialogue.

Each case starts from a compact visual premise and asks the model to carry long-range state through several camera changes, scene transitions, and narrative beats.

Model Performance / Compare

Framework comparison on multi-anchor video extrapolation.

ReCA is compared with VGoT, Mora, and MovieAgent on identity consistency, scene handoff, object-state continuity, and narrative causality across long generated sequences.

Quantitative Results

ReCA leads every backbone on MSVE-Bench and overall normalised score.

Mean ± SE across 20 MSVE prompts. Within each backbone block, ReCA and all baselines share the same frozen generator, per-call duration budget, and long-video prompt package — ReCA differs only in how it allocates planning, shot-local, and temporal context. The Wan 2.7 block is shown by default; the open-source Wan 2.2 block and the HappyHorse 1.0 generality check are available below. Numbers reproduced from the paper's Table 1.

Quantitative evaluation on **Wan 2.7** (proprietary backbone, 20 MSVE prompts).
Method	VBench ↑	StoryMem ↑	ViStory-Self ↑	ViStory-Cross ↑	MovieBench ↑	MSVE-Bench ↑	Avg. ↑
I2V Extension	0.951	0.952	0.587	0.142	19.881	0.123	0.492
VGoT	0.978	0.965	0.798	0.274	21.959	0.182	0.569
Mora	0.972	0.988	0.855	0.263	26.062	0.276	0.602
MovieAgent	0.974	0.978	0.913	0.286	25.467	0.738	0.691
ReCA (Ours)	0.984	0.992	0.936	0.324	31.454	0.942	0.749

+8–16%

average normalised score over the strongest baseline on every backbone block.

+28–43%

MSVE-Bench gain over the strongest baseline, on the metric purpose-built for multi-shot extrapolation.

4.74×

real-time factor — the lowest among all methods, ~18–22% below every baseline.

Wan 2.2 backbone (open-source generalisation, 20 MSVE prompts)

Quantitative evaluation on **Wan 2.2** (open-source backbone, our primary reproducibility target).
Method	VBench ↑	StoryMem ↑	ViStory-Self ↑	ViStory-Cross ↑	MovieBench ↑	MSVE-Bench ↑	Avg. ↑
I2V Extension	0.853	0.943	0.512	0.127	19.327	0.094	0.454
VGoT	0.871	0.988	0.631	0.263	25.144	0.113	0.520
Mora	0.885	0.990	0.666	0.303	23.726	0.187	0.545
MovieAgent	0.878	0.945	0.753	0.232	26.513	0.572	0.608
ReCA (Ours)	0.891	0.992	0.755	0.378	26.062	0.819	0.683

HappyHorse 1.0 backbone (proprietary generalisation check, 20 MSVE prompts)

Quantitative evaluation on **HappyHorse 1.0** (proprietary backbone, generality check across hosted vendors).
Method	VBench ↑	StoryMem ↑	ViStory-Self ↑	ViStory-Cross ↑	MovieBench ↑	MSVE-Bench ↑	Avg. ↑
I2V Extension	0.948	0.961	0.539	0.158	20.214	0.102	0.485
VGoT	0.978	0.978	0.710	0.370	24.470	0.153	0.572
Mora	0.974	0.975	0.798	0.331	25.165	0.273	0.600
MovieAgent	0.976	0.980	0.610	0.284	27.962	0.687	0.636
ReCA (Ours)	0.985	0.990	0.866	0.383	28.517	0.913	0.737

User Study

Human raters rank ReCA first on every Likert criterion.

Four methods (VGoT, Mora, MovieAgent, ReCA) over the 20 MSVE prompts on Wan 2.7 — 80 long videos at 3–5 min each, six 0–5 Likert criteria, raters scoring random subsets with method identity hidden. The largest gaps sit on the cross-shot dimensions that saturated short-clip metrics cannot distinguish.

Grouped-bar chart comparing VGoT, Mora, MovieAgent, and ReCA across six human-rating criteria: visual appeal, script faithfulness, character consistency, background consistency, physical law, and narrative coherence. ReCA leads every criterion. — **User-study ratings for automated movie generation.** Grouped bars compare VGoT, Mora, MovieAgent, and ReCA on six human-rating criteria — visual appeal, script faithfulness, character consistency, background consistency, physical law, and narrative coherence. ReCA wins every criterion (**4.27, 3.99, 4.25, 3.62, 3.74, 3.76** in that order), with the widest gaps on character and background consistency, where the next-best baseline stays below 2.1. Whiskers report per-cell standard error of the mean across raters; higher is better.

Citation

If you find ReCA useful, please cite our work.

@article{liu2026reca,
  title        = {ReCA: Multi-Shot Long Video Extrapolation via Recursive Context Allocation},
  author       = {Liu, Akide and Xing, Jinbo and Mao, Chaojie and Li, Ye and
                  Zhang, Zeyu and He, Yefei and Wang, Weijie and Wang, Zihan and
                  Liu, Yu and Haffari, Gholamreza and Zhuang, Bohan},
  journal      = {arXiv preprint arXiv:2605.26525},
  year         = {2026},
  eprint       = {2605.26525},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  url          = {https://arxiv.org/abs/2605.26525}
}