1 Introduction↩︎

Long-form video generation requires more than extending clip duration. A coherent multi-shot sequence must preserve recurring characters, spatial layout, visual style, object affordances, and causal event progression while allowing each shot to introduce new motion, viewpoint, and narrative information. This balance is especially difficult in text-conditioned generation: conditioning signals are often local to the next clip, whereas the errors that matter most accumulate across shots. As a result, long video systems frequently drift in identity, duplicate or forget objects, flatten event structure, or overfit to early visual context.

Recent work has made progress through explicit story memories, shot planning, and recursive context allocation [1]–[4]. ReCA is particularly relevant: it frames multi-shot extrapolation as a recursive allocation problem, deciding which previous context should condition each future shot [4]. However, many existing approaches either fine-tune large portions of the generator or rely on external memory modules whose capacity and retrieval behavior can become brittle over long horizons. This raises a natural question: can a frozen video diffusion backbone be steered with enough precision using only lightweight, parameter-efficient modules?

We propose PACR-Video, a Prompt-Adapter Context Routing framework for parameter-efficient multi-shot long video extrapolation. PACR-Video inserts low-rank temporal adapters into a frozen text-to-video diffusion transformer and conditions them with learned shot-role prompt tokens. Instead of storing dense video memories, the model maintains a recursive prompt bank containing compact entity, location, action, and style prompts extracted from previous shots. At generation time, these prompts are routed through adapter gates according to predicted narrative dependency strength, allowing the model to reuse relevant context while suppressing stale or distracting history.

Our contributions are fourfold. First, we introduce Prompt-Adapter Context Routing, a parameter-efficient alternative to full generator fine-tuning for multi-shot extrapolation. Second, we propose a recursive prompt bank that stores structured shot-level prompts and routes them through gated temporal adapters. Third, we design a Shot-Local/Story-Global tuning objective combining next-shot reconstruction, cross-shot identity contrast, and prompt sparsity regularization to reduce long-horizon drift. Fourth, we introduce an adapter composition schedule that reuses early-shot adapters for visual consistency while activating later adapters for event progression and viewpoint change.

We evaluate PACR-Video on diverse story and instructional video benchmarks, including FlintstonesSV, Pororo-SV, ActivityNet Captions, YouCook2, Shot2Story, and MovieNet [5]. Across comparisons with text-to-video, tuning-based, streaming, memory-augmented, and recursive-context baselines such as VideoCrafter2, ModelScopeT2V, AnimateDiff, Tune-A-Video, Text2Video-Zero, SEINE, StoryMem, ShotStream, and ReCA [1], [2], [4], PACR-Video yields substantial gains in distributional quality, semantic alignment, identity preservation, temporal consistency, transition coherence, and human preference while keeping the backbone frozen.

2 Related Work↩︎

Long-form video generation has been approached through shot planning, story memory, and autoregressive continuation. Planning-based systems decompose narratives into shot scripts or chain-of-shot prompts before generation [3], [6]–[8], while memory-based methods maintain explicit representations of entities, scenes, or previous frames to reduce drift [1], [9], [10]. Streaming and agentic formulations further address interactive or horizon-extending generation by updating context online [2], [11], [12]. PACR-Video shares the goal of long-horizon coherence, but differs by replacing large external memories and full-generator adaptation with compact prompt banks routed through lightweight temporal adapters.

Recursive context allocation is especially relevant to multi-shot extrapolation. ReCA formulates long video continuation as deciding which historical shots should condition each future shot, improving over fixed-window conditioning and naive autoregression [4], [13], [14]. Related long-context and selective-computation methods study how to preserve useful temporal information while avoiding unnecessary conditioning cost [15], [16]. PACR-Video builds directly on this allocation view, but moves the controllable capacity into parameter-efficient adapter gates and learned shot-role prompt tokens. This makes context routing differentiable, sparse, and compatible with a frozen diffusion transformer backbone.

A complementary line of work studies long-video understanding, segmentation, and semantic grounding, which provides useful structure for generation. Shot boundary and event-localization methods motivate the use of shot-level units rather than uniform temporal windows [17]–[19]. Benchmarks and models for multi-shot comprehension emphasize entity continuity, temporal reasoning, and cross-shot narrative dependencies [5], [20]–[22]. PACR-Video uses these insights operationally: its recursive prompt bank stores entity, location, action, and style prompts, and its Shot-Local/Story-Global objective explicitly aligns next-shot reconstruction with cross-shot identity preservation.

Finally, PACR-Video is related to parameter-efficient adaptation and multimodal alignment. Prior work on transferability, evidence accumulation, and deep model adaptation suggests that small trainable modules can steer large frozen networks when supervision is structured appropriately [23]–[25]. Video generation and evaluation work has also highlighted the importance of perceptual quality, temporal stability, and audio-visual or caption-level semantic alignment [26], [27]. In contrast to methods that tune the full generator or attach high-capacity memory modules, PACR-Video concentrates adaptation in low-rank temporal adapters, regularizes prompt sparsity, and composes early- and late-shot adapters to balance visual consistency with event progression.

3 Method↩︎

3.0.0.1 Overview.

PACR-Video generates a long video as an ordered sequence of shots \(\mathcal{V}=\{v_1,\ldots,v_T\}\) conditioned on shot-level text prompts \(\mathcal{Y}=\{y_1,\ldots,y_T\}\). Given observed or previously generated shots \(v_{<t}\), the goal is to synthesize the next shot \(v_t\) while preserving entities, locations, style, and causal progression across the story. As shown in Figure 2, PACR-Video keeps a pretrained text-to-video diffusion transformer frozen and adds only three trainable components: low-rank temporal adapters, learned shot-role prompt tokens, and a recursive prompt router. This design follows the recursive context allocation view of ReCA [4], but represents historical context as compact prompts rather than dense frame memories.

3.0.0.2 Frozen video backbone with temporal adapters.

Let \(G_\theta\) denote the frozen diffusion transformer backbone with parameters \(\theta\). For a noised latent video \(z_t^\tau\) at diffusion step \(\tau\), the backbone predicts noise \(\epsilon_\theta(z_t^\tau,\tau,c_t)\) under conditioning \(c_t\). PACR-Video inserts a trainable adapter into each selected temporal attention block. For hidden states \(h_{\ell,t}\in\mathbb{R}^{N\times d}\) at layer \(\ell\), the adapted update is \[\tilde{h}_{\ell,t} = h_{\ell,t} + \alpha_{\ell,t} B_{\ell} \sigma\!\left(A_{\ell}\operatorname{LN}(h_{\ell,t})\right), \qquad A_{\ell}\in\mathbb{R}^{d\times r},\; B_{\ell}\in\mathbb{R}^{r\times d},\; r\ll d .\] Here \(A_\ell\) and \(B_\ell\) are low-rank temporal adapter matrices, \(\sigma\) is a GELU nonlinearity, and \(\alpha_{\ell,t}\) is a routed adapter gate. The adapters operate on temporally grouped tokens, so they can modulate motion, viewpoint, and cross-frame continuity without changing the spatial generator or text encoder.

3.0.0.3 Shot-role prompt tokens.

Each shot is assigned a learned role embedding indicating its narrative function, such as establishing, continuation, reaction, transition, or resolution. The role token is concatenated with the text prompt encoding and the routed prompt-bank entries. The resulting conditioning sequence is \(c_t=[e(y_t);q_t;p_t^{\mathrm{role}}]\), where \(e(y_t)\) is the frozen text encoder output, \(q_t\) is the routed historical prompt context, and \(p_t^{\mathrm{role}}\) is the learned shot-role token. These role tokens provide controllable capacity for different shot types while sharing the same frozen video backbone.

3.0.0.4 Recursive prompt bank.

After each generated or training shot, PACR-Video writes compact prompts into a recursive prompt bank \(\mathcal{B}_t=\{b_i^{e},b_i^{l},b_i^{a},b_i^{s}\}_{i\leq t}\), corresponding to entity, location, action, and style summaries. Each entry is represented as a short learned prompt vector rather than a dense video feature. For shot \(t\), a dependency predictor estimates how strongly the next shot depends on each prior bank entry: \[\rho_{t,i,k} = \operatorname{softmax}_{i,k} \left( w_k^\top \phi\!\left[ p_t^{\mathrm{role}}, e(y_t), b_i^k, \Delta(t,i) \right] \right), \qquad q_t = \sum_{i<t}\sum_{k\in\{e,l,a,s\}} \rho_{t,i,k}\, b_i^k .\] The term \(\Delta(t,i)\) encodes relative shot distance and causal order. The weights \(\rho_{t,i,k}\) form a sparse routing distribution over bank entries and prompt types. Entity and style prompts tend to persist across long horizons, whereas action prompts are more local and location prompts are reused when the story returns to a scene.

3.0.0.5 Adapter-gated context routing.

The routed prompt context \(q_t\) controls adapter gates at each layer. A small MLP maps \([q_t;p_t^{\mathrm{role}};e(y_t)]\) to layer-specific gates \(\alpha_{\ell,t}\in[0,1]\). Early temporal layers receive stronger entity and style routing, while later layers receive stronger action and viewpoint routing. This separates visual consistency from event progression: early-shot adapters stabilize character appearance and scene geometry, while late-shot adapters support new motion, camera changes, and causal transitions.

3.0.0.6 Adapter composition schedule.

For long sequences, PACR-Video composes adapters according to shot position. The effective adapter at shot \(t\) is a convex mixture of an early consistency adapter and a late progression adapter. The mixture weight increases with narrative time, but is modulated by the dependency predictor so that recurring scenes can still reuse early-shot adapters. This schedule prevents the model from either copying early visual context too rigidly or drifting into unrelated later-shot dynamics.

3.0.0.7 Training objective.

PACR-Video is trained with a Shot-Local/Story-Global objective. The local term is the standard diffusion noise-prediction loss for next-shot reconstruction. The global terms encourage cross-shot identity consistency and sparse prompt routing: \[\mathcal{L} = \mathbb{E}_{t,\tau,\epsilon} \left[ \left\| \epsilon - \epsilon_{\theta,\psi} (z_t^\tau,\tau,e(y_t),q_t,p_t^{\mathrm{role}}) \right\|_2^2 \right] + \lambda_{\mathrm{id}}\mathcal{L}_{\mathrm{id}} + \lambda_{\mathrm{sp}}\sum_{t}\|\rho_t\|_1 .\] Only adapter, router, and prompt parameters \(\psi\) are updated; the diffusion backbone remains frozen. The identity term \(\mathcal{L}_{\mathrm{id}}\) contrasts DINO features of recurring entities across shots against features of different entities in the same minibatch. The sparsity term discourages indiscriminate use of the prompt bank, reducing context drift and stale memory retrieval.

3.0.0.8 Inference.

At inference time, PACR-Video proceeds autoregressively. For each new shot, the router reads the current prompt bank, predicts narrative dependencies, activates the corresponding temporal adapters, and generates the next shot with the frozen diffusion backbone. The generated shot is then summarized into entity, location, action, and style prompts and appended to the bank. Because the bank stores compact prompts and the backbone is frozen, inference cost grows slowly with the number of shots and does not require full-model fine-tuning for each story.

4 Experiments↩︎

4.0.0.1 Experimental setup.

We evaluate PACR-Video on six multi-shot and long-video benchmarks: FlintstonesSV, Pororo-SV, ActivityNet Captions, YouCook2, Shot2Story [5], and MovieNet. FlintstonesSV and Pororo-SV emphasize recurring animated characters and scene layouts, ActivityNet Captions and YouCook2 emphasize temporally grounded real-world actions, and Shot2Story and MovieNet emphasize multi-shot narrative structure. Following prior multi-shot extrapolation protocols [1], [2], [4], each model is given the first two shots and corresponding shot-level text prompts, then recursively generates the next eight shots. Unless otherwise stated, each generated shot contains 16 frames at \(256{\times}256\) resolution.

We compare against representative text-to-video, tuning-based, zero-shot, streaming, memory-augmented, and recursive-context baselines: VideoCrafter2, ModelScopeT2V, AnimateDiff, Tune-A-Video, Text2Video-Zero, SEINE, StoryMem [1], ShotStream [2], and ReCA [4]. For all baselines, we use the strongest publicly described long-video setting when available; otherwise we apply recursive shot-by-shot generation with the same shot prompts and observed-shot prefix. PACR-Video freezes the video backbone and tunes only temporal adapters, shot-role prompt tokens, and the prompt router.

We report distributional quality using FVD, semantic alignment using CLIPScore and BLIP-2 caption alignment, cross-shot identity preservation using DINO identity consistency, local temporal smoothness using LPIPS temporal consistency, motion stability using RAFT-based warping error, story continuity using shot transition coherence, and human preference rate from pairwise comparisons. Lower is better for FVD, LPIPS, and RAFT error; higher is better for all other metrics. Human preference is computed against ReCA over 600 randomized pairwise comparisons.

4.0.0.2 Main results.

Table 1 reports average performance across the six benchmarks. PACR-Video achieves the best result on every metric. Compared with the strongest prior baseline, ReCA, PACR-Video reduces FVD from 268.4 to 231.7, improves CLIPScore from 31.2 to 32.8, increases DINO identity consistency from 0.724 to 0.771, and improves shot transition coherence from 0.681 to 0.734. The gains are largest on long-horizon consistency metrics, suggesting that routing compact prompt-bank entries through temporal adapters provides more stable cross-shot control than conditioning on a fixed visual context window. PACR-Video is also preferred over ReCA in 63.8% of human comparisons, with annotators most frequently citing stronger character persistence and more plausible event progression. Qualitative examples in Figure 1 show that PACR-Video preserves recurring entities while still allowing viewpoint changes and action advancement.

4.0.0.3 Dataset-level analysis.

PACR-Video improves consistently across domains. On FlintstonesSV and Pororo-SV, the largest gains appear in DINO identity consistency, reflecting better preservation of recurring animated characters. On ActivityNet Captions and YouCook2, the strongest improvements occur in BLIP-2 caption alignment and RAFT-based warping error, indicating that the routed action prompts help maintain procedural order without destabilizing short-range motion. On Shot2Story and MovieNet, PACR-Video improves shot transition coherence most strongly, supporting the role of learned shot-role tokens and recursive prompt routing for narrative dependencies.

4.0.0.4 Efficiency.

PACR-Video tunes 3.8% of the backbone parameter count and does not update the text encoder or diffusion transformer weights. In contrast, full tuning variants of Tune-A-Video and long-context baselines require substantially larger trainable parameter budgets. At inference time, the recursive prompt bank stores only compact entity, location, action, and style prompt vectors, so memory grows with the number of shots rather than the number of generated frames. This makes PACR-Video practical for multi-shot extrapolation while retaining the stability of the frozen generator.

5 Ablation Study↩︎

5.0.0.1 Component ablations.

Table 2 ablates the main components of PACR-Video using the same six-benchmark average as Table 1. Removing any component degrades performance, with the largest drops coming from disabling recursive prompt-bank routing and replacing gated adapters with ungated adapters. This indicates that the gains over recursive context allocation [4] come not only from adding trainable capacity, but from selectively assigning that capacity to the historical prompts most relevant to the next shot.

The recursive prompt bank is the most important single component. Without it, PACR-Video must condition primarily on the current shot prompt and local history, increasing FVD by 45.2 and reducing transition coherence from 0.734 to 0.672. This variant often preserves short-term motion but forgets earlier entities or scene constraints after several generated shots.

Shot-role prompt tokens mainly improve semantic and narrative organization. Removing them lowers CLIPScore and BLIP-2 alignment, suggesting that explicit role conditioning helps distinguish establishing, continuation, transition, and resolution shots even when the visual context is similar. Adapter gates are also critical: ungated adapters provide extra capacity, but they cannot suppress irrelevant prompt-bank entries, leading to worse identity consistency and higher temporal error.

Temporal adapters account for most of the parameter-efficient adaptation benefit. When they are removed and only prompt tuning remains, performance falls close to memory-based baselines, especially on DINO identity consistency and RAFT warping error. The objective terms contribute complementary effects. The cross-shot identity contrast loss improves recurring character preservation, while prompt sparsity prevents the router from over-conditioning on stale history. Finally, removing the adapter composition schedule hurts transition coherence and motion stability, confirming that reusing early-shot adapters for appearance while activating later adapters for event progression is useful for long-horizon extrapolation.

6 Conclusion↩︎

PACR-Video presents a parameter-efficient framework for multi-shot long video extrapolation that keeps the text-to-video diffusion backbone frozen while routing compact historical prompts through lightweight temporal adapters. By combining learned shot-role prompt tokens, a recursive prompt bank, gated adapter routing, and a Shot-Local/Story-Global tuning objective, PACR-Video allocates cross-shot context without relying on full generator fine-tuning or large external memory modules. This design extends the recursive allocation perspective of ReCA [4] while making the controllable memory sparse, differentiable, and adapter-conditioned.

Across FlintstonesSV, Pororo-SV, ActivityNet Captions, YouCook2, Shot2Story, and MovieNet, PACR-Video consistently improves over text-to-video, tuning-based, streaming, memory-augmented, and recursive-context baselines, including StoryMem, ShotStream, and ReCA [1], [2], [5]. The method achieves lower FVD, LPIPS temporal inconsistency, and RAFT warping error, while improving CLIPScore, DINO identity consistency, BLIP-2 caption alignment, shot transition coherence, and human preference. The ablation study further shows that recursive prompt-bank routing, adapter gates, temporal adapters, identity contrast, prompt sparsity, and adapter composition each contribute to reducing long-horizon drift. These results suggest that carefully routed prompt-adapter capacity is sufficient to preserve identity, scene structure, style, and event progression over extended multi-shot generation while leaving the underlying video generator stable.

7 Future Work↩︎

PACR-Video leaves several directions open. First, the current prompt bank stores compact entity, location, action, and style prompts, but it does not explicitly model uncertainty in these summaries. Future work could make prompt entries probabilistic, allowing the router to distinguish persistent facts from ambiguous or transient evidence when extrapolating over very long horizons.

Second, PACR-Video assumes shot-level prompts are available. A natural extension is to couple prompt-adapter routing with automatic shot planning and revision, so that generation, dependency prediction, and narrative decomposition are optimized jointly. This could connect parameter-efficient extrapolation with planning-based long-video systems [3], [6], [7] while avoiding full-generator fine-tuning.

Third, the adapter composition schedule is currently learned for fixed-length extrapolation. Future work could study interactive and streaming settings in which users edit entities, locations, or events midway through a story. Combining PACR-Video with online routing mechanisms from streaming and recursive-context methods [2], [4] may enable controllable long-video generation that remains stable under user intervention.

References↩︎