1 Introduction↩︎

A video world model takes an observation history (typically a text description, an image, or a short video clip) and predicts a plausible video continuation. Modern video diffusion systems such as Sora [1], Stable Video Diffusion [2], Movie Gen [3], and NVIDIA Cosmos3-Nano and Cosmos-Predict 2.5 [4] now produce coherent multi-second video from text prompts and have been framed as world simulators for embodied agents [5], [6]. The promise is that a faithful world model could let an embodied agent plan, evaluate counterfactuals, generate training data, and verify policies before acting in the physical world. We discuss the broader landscape of video diffusion, world models, and embodied deployment in 2.

That promise depends on domain alignment. Foundation video models are pretrained on internet-scale generic video (entertainment content, vlogs, dashcam footage, and robot demonstrations) which may underrepresent the structured visual vocabulary of specific deployment domains. Retail environments are a clear example: dense product shelving, narrow parallel aisles, repetitive geometry at multiple scales, distinctive signage and end-caps, and multi-person dynamics around carts and checkout counters. A retail world model must render scenes built from this vocabulary and continue shopping behaviors consistent with the physical and behavioral regularities of supermarket environments. Without explicit adaptation, generated retail scenes drift toward generic interiors, and predicted human motion drifts away from realistic shopping within a few seconds of rollout.

This paper studies parameter-efficient adaptation of a pretrained foundation video world model to retail scenes via Low-Rank Adaptation (LoRA) [7], and introduces the dataset that makes such a study possible. We use Cosmos3-Nano [4] as the base model, a 16-billion-parameter text-and-image-conditioned video diffusion transformer that is, to our knowledge, one of the largest openly released foundation video models with a documented post-training recipe. As training data we introduce RetailSMV (Retail Synchronized Multi-View), a retail video corpus of \(32{,}105\) captioned clips collected across five real-world supermarkets. RetailSMV provides synchronized ego/exo capture of store-staff operational work—stocking, arranging, weighing and labelling produce, carrying crates, pushing supply carts, scanning items at the checkout, and assisting customers—rather than the customer-side behaviour that prior retail video corpora capture. Every work episode is recorded simultaneously by a head-mounted egocentric camera worn by the staff member and a fixed exocentric scene camera that observes the same activity from a third-person perspective. To our knowledge no prior dataset combines a real-world retail deployment domain with synchronized ego/exo capture of the same activity at the scale required to fine-tune a foundation video diffusion model; this combination is what enables the controlled view-stratified study reported here.

We organize the study around a single experimental question (see 2 for the full pipeline):

To answer this we train three LoRA configurations of Cosmos3-Nano under matched hyperparameters and identical optimization budget: (i) egocentric-only (\(n{=}16{,}120\) clips), (ii) exocentric-only (\(n{=}15{,}985\) clips), and (iii) combined ego\(+\)exo (\(n{=}32{,}105\) clips). To control for seed variance we additionally train a second seed of the combined configuration. We evaluate every configuration on the same held-out test set of \(200\) stratified retail clips balanced across both views (\(n{=}100\) egocentric and \(n{=}100\) exocentric), generating one video per (configuration, clip) pair under identical inference settings. The balanced stratification reflects both natural deployment viewpoints (the head-mounted egocentric camera and the fixed exocentric scene camera) at equal weight, and the egocentric subset enables a cross-view transfer check (8.2). The test set is disjoint from both the training data and the adapter-selection validation pool, eliminating selection leakage (4).

Findings.  Our analysis surfaces three robust empirical findings:

1. Adaptation succeeds uniformly. All three LoRA configurations reduce the validation diffusion loss by a factor of approximately \(2.8\) (from \(1.006\) to a tight range of \(0.355{-}0.363\)). Every one of \(200\) paired evaluation samples improves over the pretrained baseline under each LoRA configuration, with both parametric and non-parametric paired tests at \(p\ll 0.001\) (4; exact values in 8.1). Seed-induced variation on the combined configuration is smaller than the configuration-to-configuration spread, supporting the interpretation that configuration rankings reflect properties of the training data rather than optimization noise.

2. Exocentric-only adaptation matches or exceeds combined adaptation on six of seven point estimates and is significantly better on LPIPS, PSNR, and DreamSim. Despite training on only the \(15{,}985\) exocentric clips (versus the \(32{,}105\) clips available to the combined configuration), the exocentric-only adapter achieves the lowest R3D-Fréchet (\(35.14\) vs.base \(42.69\), a \(17.7\%\) relative reduction; combined \(36.20\), a \(15.2\%\) reduction), the lowest JEDi (\(0.829\) vs.base \(1.246\), a \(33.5\%\) relative reduction; combined \(0.892\), a \(28.5\%\) reduction), the largest LPIPS reduction of \(0.058\) at \(100\%\) paired win-rate (\(p{<}10^{-34}\)), the largest PSNR improvement of \(+0.575\) dB at \(85\%\) win-rate, and the largest DreamSim reduction of \(0.062\) at \(92.5\%\) win-rate (\(p{<}10^{-31}\)). The combined configuration matches exo on SSIM (\(+0.022\)) and CLIPScore (\(+0.017\) in the Hessel formulation) but is significantly worse than exo on the perceptual and pixel-fidelity metrics in direct paired tests (LPIPS \(p{<}10^{-7}\), PSNR \(p{=}0.003\), DreamSim \(p{<}10^{-9}\)). The two distributional metrics agree on the ranking exo \(<\) combined \(<\) ego \(<\) base, so the conclusion does not rely on either single feature backbone. This result suggests that combined ego\(+\)exo training does not improve over exo-only training on this corpus, and that the in-distribution exocentric subset is what carries the adaptation signal.

3. The adaptation gap is concentrated in the near-horizon prediction window. The absolute LPIPS gap to the pretrained base is largest at the shortest rollout time we sample (\(t{=}0.5\) s, base mean LPIPS \(0.542\) vs.exocentric-only \(0.460\), a gap of \(0.082\)) and narrows steadily as \(t\) grows. Restricting the LPIPS analysis to \(t{=}1.0\) second of rollout widens the paired gap from the full-clip averages (\(+0.041\), \(+0.058\), \(+0.049\) for ego, exo, combined) to \(+0.061\), \(+0.075\), and \(+0.068\) respectively, at \(82{-}89\%\) paired win-rate (\(p{<}10^{-20}\)). The near-horizon window is precisely the regime that world-model literature targets for embodied use [8].

Contributions.  We make four contributions to the study of pretrained video world model adaptation:

1. We introduce RetailSMV (Retail Synchronized Multi-View), a retail video corpus of \(32{,}105\) captioned clips collected across five real-world supermarkets, with synchronized egocentric and exocentric capture of store-staff operational work (stocking, arranging, weighing, managing supply carts, scanning at checkout) rather than the customer-side behaviour captured by prior retail video corpora, dense paragraph-level captions, and pre-defined train, validation, and test splits. To our knowledge no prior dataset combines a real-world retail deployment domain with synchronized staff-perspective ego/exo capture of the same activity at this scale.

2. Using RetailSMV, we present, to our knowledge, the first controlled comparison of egocentric-only, exocentric-only, and combined LoRA configurations of a foundation video world model, isolating training-data viewpoint as the variable of interest while holding all other factors constant.

3. We document a reproducible headline finding: exocentric-only adaptation matches or exceeds combined adaptation on six of seven point estimates and is significantly better on LPIPS, PSNR, and DreamSim, despite using only the \(15{,}985\) exocentric training clips while combined uses the full \(32{,}105\). The combined configuration edges exo on SSIM by a small margin, and the two are statistically indistinguishable on CLIPScore. We additionally characterize the temporal structure of the adaptation gap and show that the absolute LPIPS gap to the pretrained base is largest at the shortest rollout time and narrows steadily as \(t\) grows.

4. We provide a fully paired statistical protocol for video world model evaluation and report all metrics under both parametric (\(t\)-test) and non-parametric (Wilcoxon signed-rank) tests, addressing the documented absence of paired statistical reporting in current video-generation literature.

The remainder of the paper is organized as follows. 2 reviews related work on foundation video world models, video generation evaluation, and parameter-efficient adaptation. 3 describes the multi-view retail data, the three view-stratified LoRA configurations, and the training and inference protocols. 4 defines the test set, the metric suite, and the paired statistical tests. 5 presents the main results, the temporal analysis, and metric ablations. 6 concludes with discussion and limitations.

2 Related Work↩︎

Foundation video generation models.  Text-to-video and image-to-video diffusion models have advanced rapidly. Make-A-Video [9] and AnimateDiff [10] demonstrated that motion priors can be learned without paired text-video data; DynamiCrafter [11] and VideoCrafter2 [12] extended these to open-domain image-to-video conditioning; CogVideoX [13], HunyuanVideo [14], Lumiere [15], Movie Gen [3], Open-Sora [16], Stable Video Diffusion [2], Sora [1], and VideoPoet [17] pushed scale, fidelity, and duration. The NVIDIA Cosmos family of world foundation models [4] targets embodied AI specifically and is the basis of our adaptation experiments. The underlying training objectives include denoising diffusion [18], [19], flow matching [20], and rectified flow [21], with UniPC [22] now a common deterministic sampler at inference. Almost all of these models are pretrained on internet-scale generic video and report no domain-specific evaluation in retail environments.

World models for embodied simulation.  The framing of pretrained video models as world simulators for embodied agents traces to the world-models lineage of Ha and Schmidhuber [5] and is now exemplified by DreamerV3 [23], Genie [24], and the learning-an-interactive-simulator framework of UniSim [25], as well as more recent surveys [6]. Domain-specialized driving world models add explicit action conditioning: GAIA-1 [26], GAIA-2 [8], Vista [27], and DriveDreamer [28]. Embodied robotics has produced action-conditioned generative policies that consume or emit video: RT-2 [29], PaLM-E [30], GR00T [31], \(\pi_0\) [32], AgiBot [33], and the Gemini Robotics family [34], supported by retail-adjacent benchmarks such as RoboVQA [35]. Outside driving, however, systematic studies of adapting a foundation video model to a specific deployment environment are scarce.

Parameter-efficient adaptation of diffusion models.  Adapting a large pretrained diffusion model to a new domain is typically done through parameter-efficient fine-tuning. The dominant family of methods comprises Low-Rank Adaptation [7] and its variants [36], [37]; subject-driven personalization with DreamBooth [38]; word-level personalization via textual inversion [39]; and multi-concept composition [40]. For video specifically, motion-customization methods have started to appear: MotionDirector [41], MotionCtrl [42], and AnimateDiff [10] all adopt LoRA-style adapters to inject motion or domain priors into a frozen image- or video-diffusion backbone. NVIDIA’s Cosmos-Predict 2.5 cookbook [4] provides documented LoRA post-training recipes. Most prior video LoRA work treats viewpoint as a nuisance variable or focuses on a single viewpoint; to our knowledge there is no prior systematic comparison of egocentric, exocentric, and combined LoRA configurations on a synchronized multi-view video corpus.

Video generation evaluation.  The standard suite is built around the Fréchet video distance (FVD) [43] on I3D Kinetics features [44], per-frame LPIPS [45] with AlexNet features [46], PSNR and SSIM [47], and CLIPScore [48] computed from CLIP image-text embeddings [46]. VBench [49] and VBench-2 [50] provide curated capability dimensions for generic video generation. These benchmarks were designed assuming large evaluation sets; FVD is documented to require large samples for stable covariance estimation [51], a regime rarely available in domain-specific applications. Modern alternatives respond to this gap. DreamSim [52] combines CLIP, OpenCLIP, and DINO [53] features with human-similarity fine-tuning, producing a perceptual metric that outperforms LPIPS at semantic structure and object identity. JEDi [51] replaces I3D-FVD with an MMD on V-JEPA features [54], reaching a stable estimate at far smaller sample counts. We adopt DreamSim and JEDi alongside the classical suite, use the Hessel formulation of CLIPScore [48], and use an R3D-Fréchet variant of the Kinetics-based Fréchet distance with the ResNet-3D-18 backbone [55] described in 4.2.

Egocentric and exocentric video understanding.  Egocentric video research is dominated by Ego4D [56], Epic-Kitchens [57], HOI4D [58], and Aria [59], all of which skew toward kitchens, outdoor activities, and daily living scenes. EgoSchema [60] provides a long-form egocentric video understanding benchmark. Charades-Ego [61] and Ego-Exo4D [62] introduce paired ego/exo recordings and demonstrate that joint training improves cross-view action recognition and retrieval. The pretext-task literature has used arrow-of-time prediction [63] and spatio-temporal jigsaw solving [64] to learn temporal structure from unlabelled video. These studies focus on recognition and representation learning; we focus on generation, where the question is qualitatively different: does adding ego data to exo data improve generation of new retail scenes, or does it dilute the learning signal? Our experiments show that on retail video the empirical answer is that combined ego\(+\)exo training does not improve distributional or semantic metrics over exo-only training: exocentric-only adaptation matches or beats combined adaptation on R3D-Fréchet and DreamSim. 1 summarizes the gap in publicly available foundation video models with respect to retail coverage and synchronized multi-view data.

Retail-specific video datasets and benchmarks.  Retail and shopping environments have been studied as a deployment domain across recognition, simulation, and language-grounded tasks. MERL Shopping [65] provides surveillance-style exocentric clips of retail customer interactions; the retail-action benchmark [66] and retail-vision [67] survey extend this to fine-grained action recognition; SariBench [68] provides a simulated retail environment for embodied agents; PRISM [69] is a recent multi-view multi-capability retail video dataset built for embodied vision-language models, with chain-of-thought supervision and an SFT-ready format. These works focus on recognition, question answering, or simulation, not on video generation; in particular none of them has been used to fine-tune or evaluate a foundation video diffusion model. They also focus on the customer side of the retail scene (browsing, picking, paying). RetailSMV (this paper) is, to our knowledge, the first retail video corpus with synchronized ego/exo capture of the store-staff operational viewpoint—stocking, arranging, weighing, managing supply carts, and scanning at the checkout—at a scale that supports finetuning of a foundation video world model (2). RetailSMV is distinct from PRISM: PRISM targets embodied vision-language supervision, whereas RetailSMV is organized for video-world-model adaptation with paired ego/exo clips, generation captions, adaptation splits, and generated-video evaluation protocols.

Statistical reporting in video generation.  A practical observation in recent video-generation literature [8], [49], [52] is that paired statistical tests, win-rates, and confidence intervals are rarely reported, even though metric standard deviations are often comparable in magnitude to the differences between methods. The general statistical-comparison framework of Demšar [70] predates the diffusion-model era but its core advice (report paired tests with both parametric and non-parametric variants) remains directly applicable. We adopt this protocol throughout: every per-clip metric is reported with mean improvement, paired win-rate, and both parametric (\(t\)-test) and non-parametric (Wilcoxon signed-rank) \(p\)-values.

3 Method↩︎

We describe the RetailSMV corpus introduced with this paper (3.1), the parameter-efficient adaptation recipe applied to the pretrained foundation model (3.2), the three view-stratified configurations whose comparison forms the core of our study (3.3), and the inference protocol used to generate evaluation videos (3.4).

3.1 The RetailSMV Corpus↩︎

We introduce RetailSMV (Retail Synchronized Multi-View), a retail video corpus of 32,105 captioned clips with synchronized egocentric and exocentric capture of the same store-staff activities across five real-world supermarkets. To our knowledge no prior dataset combines a real-world retail deployment domain with synchronized ego/exo capture of the same activity at the scale required to fine-tune a foundation video world model. 2 positions RetailSMV against related embodied video datasets.

Capture protocol.  The corpus is collected in five operating supermarkets that span distinct store layouts, lighting conditions, aisle configurations, and product category distributions. Unlike prior retail video datasets which focus on customer-side behaviour, RetailSMV is captured from the perspective of store staff performing their day-to-day operational work: stocking shelves, arranging produce, weighing and labelling items, carrying crates, pushing supply carts, measuring shelf space, scanning items at the checkout, and assisting customers. Each work episode is recorded simultaneously by two complementary sensors. The egocentric sensor is a head-mounted camera worn by the staff member while they carry out these tasks. The exocentric sensor is a fixed scene-level camera that observes the same activity from a stable third-person perspective. Synchronized capture means that for every ego clip there exists an exo clip of the identical activity at the same wall-clock time, with the same person, products, and scene context, captured from the complementary viewpoint. This pairing is what enables the controlled view ablations reported in this paper.

Clip-level selection and captioning.  From the raw recordings we extract 32,105 short clips (\(1\)–\(8\) s each) that satisfy three conditions: (i) the clip captures a complete atomic store-staff action, (ii) the synchronized view from the paired sensor is available for the same time interval, and (iii) a dense paragraph-level caption is available. Each caption is a single paragraph describing the visible store and section, the person’s high-level task and low-level action, hand states and body pose, visible products and signage, and a frame-by-frame motion summary. The same caption is used as the text-conditioning input for every configuration of every training run, eliminating prompt variance as a source of cross-configuration difference. The full clip pool decomposes into \(16{,}120\) egocentric and \(15{,}985\) exocentric clips, an essentially balanced split that lets us match clip count exactly across single-view configurations.

Privacy.  All faces are de-identified through Gaussian blurring at the source. This is applied uniformly to both ego and exo views and to the train, validation, and test splits; we do not introduce or remove any face obfuscation in this work.

Train, validation, and test splits.  We partition RetailSMV into three pairwise disjoint splits by per-clip unique identifier. The training set consists of 32,105 clips and is used for LoRA adaptation. The validation set consists of 1,388 clips and is used for adapter selection: the rectified-flow validation loss is computed every 100 training steps on \(n{=}32\) paired samples drawn from it. The test set consists of 200 clips drawn via stratified sampling from the held-out evaluation pool, balanced across both egocentric (\(n{=}100\)) and exocentric (\(n{=}100\)) viewpoints so that both deployment scenarios are represented at equal weight in the per-clip paired statistics. The egocentric-only model is evaluated on its in-distribution view as well as on the exocentric view, exposing cross-view transfer behavior. 3 summarizes the splits.

3.2 LoRA Adaptation of the Foundation Model↩︎

We adapt the open-source Cosmos3-Nano video diffusion model [4], a 16-billion-parameter Mixture-of-Transformers Diffusion Transformer with a Wan2.2 video VAE and the Cosmos-Reason [71], [72] text encoder, using Low-Rank Adaptation [7].

LoRA configuration.  We attach LoRA adapters [7] of rank \(r{=}32\) and scaling \(\alpha{=}64\) to the cross-attention projections and the Mixture-of-Experts MLPs in every generative-modality transformer block, training \(\sim 0.55\%\) of the base model’s parameters. This follows the standard adapter-pluggable convention now widely used for diffusion-model personalization [36], [38]–[40] and motion customization [10], [41], [42]. Standard zero-initialization of \(B\) ensures the model at step zero reproduces the pretrained base. Full layer-wise target list, exact parameter count, and AdamW [73] settings are in 9.

Training objective.  We train under the rectified-flow [21] velocity-matching objective used by Cosmos3-Nano during its own pretraining, a stochastic-interpolant family closely related to denoising diffusion [18], [19] and flow matching [20]. For each training step we encode the input clip to its latent representation \(z_0\), draw a noise scale \(\sigma \sim \mathrm{Sigmoid}(\mathcal{N}(0,1))\), sample a Gaussian noise tensor \(\epsilon\sim\mathcal{N}(0,\mathbf{I})\), form the perturbed latent \(z_\sigma = \sigma\epsilon + (1{-}\sigma)z_0\), and target the velocity \(v_\sigma = \epsilon - z_0\). The training loss is the mean squared error \(\mathcal{L} = \|\hat{v}_\sigma - v_\sigma\|_2^2\) averaged over the spatio-temporal latent grid.

Training protocol.  We train at \(480{\times}832\) spatial resolution, \(81\) frames per clip, for \(3{,}000\) optimization steps under a cosine learning-rate schedule with a peak rate of \(3{\times}10^{-4}\). Validation diffusion loss is computed every \(100\) steps on \(n{=}32\) paired samples drawn from the \(1{,}388\)-clip validation set, and we select the iteration-\(1500\) adapter (lowest validation loss) for all subsequent generation experiments. The full training environment (precision, video decoder, hardware, effective batch size, and other infrastructure details) is in [sec:appendix_hp] 9.

3.3 View-Stratified Configurations↩︎

To isolate the effect of training-data viewpoint while holding all other variables constant, we train three matched configurations under identical hyperparameters and optimization budget:

Combined (ego \(+\) exo).  The reference configuration, trained on the full 32,105-clip mixture. This corresponds to the natural default of using all available data.

Egocentric-only.  Trained on the 16,120 egocentric clips alone. This configuration can specialize to first-person motion patterns and hand-object interaction, at the cost of losing exposure to wide-field scene structure.

Exocentric-only.  Trained on the 15,985 exocentric clips alone. This configuration can specialize to scene layout, multi-actor dynamics, and the wide field of view of the fixed scene camera, at the cost of losing exposure to fine-grained hand interaction.

Combined, second seed.  An additional combined configuration trained with a different random seed (137 instead of 42), used to bound the seed-induced variance on the validation metrics and to confirm that configuration-to-configuration differences exceed this variance.

3.4 Inference Protocol↩︎

For evaluation we generate one video per (configuration, test clip) pair under image-to-video conditioning: the first frame of the ground-truth test clip serves as the conditioning image, and the dense caption for that clip serves as the text prompt. We generate \(189\) frames at \(480{\times}832\), the native maximum length of Cosmos3-Nano (\(\sim 7.9\) s at \(24\) FPS), so that wall-clock time-aligned metric sampling is well defined for the full duration of every clip. The deterministic UniPC multistep sampler [22] is used at inference. Scheduler, inference steps, guidance scale, negative prompt, and seed are listed in 7.3. The trained adapter is merged into the base weights at inference; we report a numerical merge verification in 9.3.

4 Evaluation Protocol↩︎

This section defines the test set (4.1), the metric suite (4.2), the paired statistical protocol (4.3), and the near-horizon analysis used to characterize the temporal structure of the adaptation gap (4.4).

4.1 Test Set↩︎

Due to the high inference cost of video generation (on the order of minutes per clip on contemporary hardware), we evaluate on a small held-out test set, the standard mode of evaluation in current video-generation literature. We use the same test set across all four configurations and the pretrained baseline, generated under identical inference settings (3.4).

The test set consists of \(200\) retail clips drawn from a held-out evaluation pool via stratified sampling, balanced across both egocentric (\(n{=}100\)) and exocentric (\(n{=}100\)) viewpoints. The draw is deterministic and uses rejection sampling against the validation set used for adapter selection. We verify by per-clip unique-identifier intersection that the test set is disjoint from both the training data and the adapter-selection validation set.

Asset pipeline.  For each test clip we extract the full-length ground-truth video, the first frame as a JPEG image, and the dense caption. We then generate exactly one video per (configuration, test clip) pair, for a total of \(4{\times}200=800\) generated videos. All generations use the deterministic settings of 3.4.

4.2 Metric Suite↩︎

We report a deliberately wide panel of seven metrics to avoid over-interpreting any single number. All metrics are computed on the same generated videos. The panel comprises per-frame perceptual metrics (LPIPS, PSNR, SSIM), two distributional metrics (a classical Kinetics-feature Fréchet distance and a recent V-JEPA-based MMD), a semantic alignment metric (Hessel CLIPScore), and a modern human-aligned perceptual metric (DreamSim). We report JEDi as a recent distributional video-quality metric complementary to R3D-Fréchet; we do not rely on JEDi alone, and the same ranking is supported by R3D-Fréchet and DreamSim.

LPIPS.  [45] measures perceptual distance between matched generated/ground-truth frame pairs using AlexNet features. We sample 16 evenly-spaced wall-clock timestamps in \([0,\min(\text{gen\_dur},\text{gt\_dur})]\), dropping \(t{=}0\) to remove the trivial first-frame match introduced by image-to-video conditioning. Lower is better. The rationale for wall-clock sampling (avoiding a frame-index pitfall between \(24\) FPS generations and lower-FPS ground truths) is detailed in 9.2.

PSNR and SSIM.  [47] are computed per frame on the same time-aligned samples used for LPIPS and averaged over the clip. Higher is better.

Hessel CLIPScore.  [48] measures caption-video alignment. For each of 8 evenly-spaced frames of the generated video, we compute \(2.5\cdot\max(0,\cos(\text{img\_emb},\text{txt\_emb}))\) with CLIP ViT-B/32 (Hessel et al.’s convention), and average across frames. Higher is better; typical values fall in the \(0.6\)–\(0.85\) range.

R3D-Fréchet video distance.  We compute a Fréchet distance on features from the ResNet-3D-18 video classifier (torchvision.models.video.r3d_18 with KINETICS400_V1 weights) on 16 frames per clip, used in place of the canonical I3D [43], [44]. We call the resulting metric R3D-Fréchet to make the backbone substitution explicit; absolute values are not directly comparable to I3D-FVD values in the broader literature. Lower is better. See 9 for the rationale and a recipe for recomputing with the canonical I3D backbone.

DreamSim.  [52] is a perceptual distance built from an ensemble of CLIP, OpenCLIP, and DINO features fine-tuned on human similarity triplets. It is documented to outperform LPIPS, raw CLIP, and DINO on human similarity benchmarks. We compute it on the same 16 time-aligned frames used for LPIPS, averaged over the clip. Lower is better.

JEDi.  [51] is a recent distributional video-quality metric that replaces the I3D-FVD covariance estimate with a maximum mean discrepancy on V-JEPA [54] ViT-H/16 features. It is documented to reach a stable estimate at far smaller sample counts than I3D-FVD. We compute JEDi on \(16\) uniformly-sampled frames per clip at \(224{\times}224\), using the official videojedi package with the V-JEPA ViT-H/16 backbone, with the gold (ground-truth) loader and the generated loader containing the same set of clips. Lower is better. The reported JEDi value is a biased empirical polynomial-kernel MMD estimator and its absolute scale is sample-set dependent (we verified this directly by recomputing JEDi on disjoint sub-pools of our test set); the relative reduction vs.the pretrained base, and the configuration ranking, are the reportable quantities. We report JEDi as a complementary distributional signal to R3D-Fréchet; we do not rely on it alone, and the configuration ranking is consistent with R3D-Fréchet and DreamSim.

Validation diffusion loss.  For each configuration we additionally compute the rectified-flow training loss on a paired-noise evaluation set of \(n{=}200\) random samples from the validation set used for adapter selection (not from the held-out test set). To ensure exact pairing across configurations, the noise tensor and noise scale for sample \(i\) are derived from a fixed seed \(10000{+}i\) shared by all configurations and the base. This produces per-sample loss values that are directly comparable across configurations and forms the basis of 4.

4.3 Paired Statistical Protocol↩︎

For every per-clip metric and every LoRA configuration \(c\) we compute, for each test clip \(i\), the signed paired difference \[\delta_i(c) \;=\;\begin{cases}m_i(\text{base})-m_i(c),& m\text{ lower-is-better}\\ m_i(c)-m_i(\text{base}),& m\text{ higher-is-better}\end{cases}\] so that \(\delta_i(c){>}0\) means \(c\) beats the base on clip \(i\). We then report (i) the mean paired difference \(\bar\delta(c)\), (ii) the paired win-rate \(|\{i:\delta_i(c){>}0\}|/n\), (iii) a parametric two-sided paired \(t\)-test \(p\)-value, and (iv) a non-parametric Wilcoxon signed-rank \(p\)-value. We treat \(p{<}10^{-4}\) as highly significant. For the distributional R3D-Fréchet metric we report the per-configuration scalar score and the relative improvement, since it operates on the set of videos as a whole rather than on paired samples.

4.4 Near-Horizon LPIPS Analysis↩︎

World models are predominantly used to predict the immediate future of a scene, typically the next one to three seconds [8]. The mean LPIPS curves (13) rise monotonically with \(t\) for every configuration: the absolute gap to the pretrained base is widest at the shortest rollout time we sample (\(t{=}0.5\) s) and narrows steadily as rollout time grows, since all configurations drift away from the single ground-truth trajectory in the same way. To characterize the temporal structure of the adaptation gap, we additionally report LPIPS at fixed time stamps \(t\in\{1.0,2.0\}\) seconds on the same test clips. This is a reporting convention that we recommend for future video world model evaluations, since it isolates the regime in which the model’s prediction quality matters most for downstream embodied use.

5 Results and Analysis↩︎

We present the results in five subsections. 5.1 establishes that all three view-stratified configurations improve uniformly over the pretrained baseline on the unified validation diffusion loss. 5.2 reports the main paired metrics on the held-out test set, with direct paired comparisons between the adapted configurations (combined vs.exocentric-only and combined vs.egocentric-only) and the asymmetric data-contribution result. 5.3 walks through ten representative video examples to ground the metric-level differences in concrete failure modes. 5.4 examines the temporal structure of the adaptation gap and identifies the near-horizon prediction window as the regime in which the LoRA-adapted models differ most from the pretrained base. 5.5 reports time-aligned-vs-index-based sampling, adapter-merge verification, and modern-metric sensitivity. All \(p\)-values are computed under the paired protocol of 4.3.

5.1 Adaptation Improves Validation Loss Uniformly↩︎

4 reports the rectified-flow validation loss on \(200\) paired samples drawn from the validation set used for adapter selection. By construction, the noise tensor and noise scale for sample \(i\) are identical across all configurations, so the per-sample loss values are directly comparable.

Three observations follow. First, adaptation reduces validation loss by a factor of approximately \(2.8\), and every one of the \(200\) paired evaluation samples improves over the pretrained base under each configuration (perfect paired ordering under the Wilcoxon signed-rank test; exact \(p\)-values are reported in 8.1). Second, the seed-induced variation between the two combined seeds is \(0.002\), smaller than the configuration-to-configuration spread of \(0.008\) across the four LoRA rows; configuration-to-configuration ordering therefore reflects properties of the training data rather than optimization noise. Third, the three view-stratified configurations are within \(0.008\) of each other on this metric, which means validation loss alone does not discriminate between view-stratified data choices. Differences across configurations emerge when generation quality is evaluated directly.

5.2 Main Paired Metrics↩︎

5 reports the seven primary metrics on the held-out 200-clip test set. The exocentric-only configuration achieves the lowest R3D-Fréchet (\(35.14\) vs.base \(42.69\), a \(17.7\%\) relative reduction), the lowest JEDi (\(0.829\) vs.base \(1.246\), a \(33.5\%\) relative reduction), the largest LPIPS reduction of \(0.058\) at \(100\%\) paired win-rate (\(p{<}10^{-34}\)), the largest PSNR improvement of \(+0.575\) dB at \(85\%\) paired win-rate, and the largest DreamSim reduction of \(0.062\) at \(92.5\%\) paired win-rate (\(p{<}10^{-31}\)). The combined configuration is tied with exo on CLIPScore and edges exo on SSIM by \(0.0012\). Hessel CLIPScore improves under all three configurations with very strong paired significance (\(p{<}10^{-10}\)). We report JEDi as a recent distributional video-quality metric complementary to R3D-Fréchet: we do not rely on JEDi alone, and the same ranking is supported by R3D-Fréchet and DreamSim.

Headline finding: exocentric-only adaptation matches or exceeds combined adaptation on six of seven point estimates and is significantly better on LPIPS, PSNR, and DreamSim.  The exocentric-only adapter, trained on \(15{,}985\) exocentric clips, achieves an R3D-Fréchet of \(35.14\), a \(17.7\%\) relative reduction from the base, and a JEDi of \(0.829\), a \(33.5\%\) relative reduction (the largest distributional gap of any configuration); the combined adapter, with access to all \(32{,}105\) clips, achieves \(36.20\) R3D-Fr.(\(-15.2\%\)) and \(0.892\) JEDi (\(-28.5\%\)). On LPIPS the exocentric-only adapter produces the largest paired improvement of \(+0.058\) at \(100\%\) paired win-rate (every one of \(200\) test clips); combined is at \(+0.049\) at \(94.5\%\). On PSNR exo improves by \(+0.58\) dB at \(85.0\%\) win-rate versus combined’s \(+0.50\) dB at \(78.5\%\). On DreamSim exo achieves a reduction of \(0.062\) at \(92.5\%\) win-rate (\(p{<}10^{-31}\)) versus combined’s reduction of \(0.046\) at \(82.0\%\). The exocentric-only point estimate matches or exceeds the combined point estimate on six of the seven metrics we report (LPIPS, PSNR, CLIPScore, DreamSim, R3D-Fréchet, and JEDi); combined edges exo on SSIM only, by \(0.0012\). Of the five per-clip metrics for which a paired test is defined, exo is significantly better than combined on LPIPS, PSNR, and DreamSim (6), and the two configurations are statistically indistinguishable on SSIM and CLIPScore. The two distributional metrics agree on the ranking (exo \(<\) combined \(<\) ego \(<\) base), supporting the conclusion under either feature backbone.

Direct paired comparison: combined vs.exocentric-only.  The paired Wilcoxon tests in 5 are against the pretrained base. To support the headline claim that combined ego\(+\)exo training does not improve over exo-only, we additionally compute paired tests directly between the two adapted configurations on the same \(200\)-clip test set. 6 reports the mean paired difference (combined \(-\) exocentric-only), \(95\%\) bootstrap confidence interval over clips, combined win-rate, and Wilcoxon \(p\)-value for every per-clip metric. Exo is significantly better than combined on LPIPS, PSNR, and DreamSim. Combined and exo are statistically indistinguishable on SSIM and CLIPScore (the \(95\%\) CI for SSIM straddles zero, and the CLIPScore CI is symmetric around zero). For the set-level R3D-Fréchet metric we cannot run a paired test (it operates on whole feature sets), but the point estimate is also in favor of exocentric-only (\(35.14\) vs.combined \(36.20\)).

We find no statistically significant metric on which combined adaptation outperforms exocentric-only adaptation.  The picture from 6 is consistent across metrics: on three of the five per-clip metrics (LPIPS, PSNR, DreamSim) combined is statistically worse than exo at \(p\leq 0.003\), with combined-wins fractions of \(32\)–\(40\%\) (below chance), and the DreamSim gap is significant at \(p{<}10^{-9}\). On SSIM and CLIPScore the two are statistically indistinguishable (the \(95\%\) CI for SSIM straddles zero and the CLIPScore CI is symmetric around zero). The data-rich combined configuration does not deliver any statistically significant per-clip-metric advantage over the exo-only adapter; the in-distribution exocentric subset carries the adaptation signal.

Direct paired comparison: combined vs.egocentric-only.  For a symmetric view of the asymmetry, we additionally compute paired tests directly between the combined and egocentric-only adapters on the same \(200\)-clip test set. 7 reports the mean paired difference (combined \(-\) egocentric-only), \(95\%\) bootstrap confidence interval over clips, combined win-rate, and Wilcoxon \(p\)-value for every per-clip metric. Combined is significantly better than egocentric-only on LPIPS (\(p{=}0.002\)), PSNR (\(p{<}10^{-6}\)), and SSIM (\(p{<}10^{-14}\)); the two configurations are statistically indistinguishable on CLIPScore and DreamSim. For the set-level distributional metrics, combined is also in front of egocentric-only (R3D-Fr.\(36.20\) vs.\(37.65\); JEDi \(0.892\) vs.\(0.960\)).

Asymmetric data contribution: adding exocentric data to ego helps; adding egocentric data to exo hurts.  Read alongside 6, 7 exposes a clear asymmetry. Starting from the egocentric-only adapter, adding the exocentric subset (i.e., training combined) produces statistically significant improvements on LPIPS, PSNR, and SSIM at \(p\leq 0.0018\) and improves the set-level distributional scores (R3D-Fréchet and JEDi). Starting from the exocentric-only adapter, however, adding the egocentric subset (again, training combined) produces statistically significant regressions on LPIPS, PSNR, and DreamSim at \(p\leq 0.003\). The two adapted configurations are statistically tied on CLIPScore in both directions. Combined therefore improves over egocentric-only on several metrics, but we find no statistically significant metric on which combined outperforms exocentric-only on this corpus, indicating that the adaptation signal is asymmetrically carried by the exocentric subset.

Egocentric-only adaptation transfers but ranks third.  The egocentric-only configuration improves over the base on every metric at \(p{<}10^{-8}\), but it ranks behind both exo-only and combined on every column of 5. The egocentric adapter does transfer non-trivially to the mixed test viewpoint (LPIPS paired win-rate is \(90.5\%\) at \(p{<}10^{-27}\)) but the transfer is consistently weaker than the in-distribution exocentric-only adapter, indicating that paired ego\(+\)exo capture does not eliminate the need for view-matched training data.

5.3 Qualitative Analysis↩︎

The metric-level differences above translate cleanly into failure-mode patterns that are visible in the generated videos themselves. We walk through ten representative examples drawn from the held-out test set. Every figure in this subsection uses the same five-row layout: the top row is the ground-truth video (GT), and the four rows beneath show how the base, egocentric-only, exocentric-only, and combined configurations extend the shared conditioning frame over the next seven seconds. The column at \(t{=}0\) is the shared conditioning frame seen by all four generated configurations, which is the same frame as the first GT frame. The first column (\(t{=}0\)) is therefore identical across all five rows by construction. The cell highlighted in red marks, for each example, the base configuration’s frame at which the failure is most visible (typically \(t{=}1\), \(t{=}3\), \(t{=}5\), or \(t{=}7\) s).

Hand-off watermelon.  3 shows the conditioning frame of two people at a produce display, where the person is set up to receive a watermelon and hold it with both hands. The base configuration drifts away from the produce area within a few seconds and ends, at \(t{=}7\) s, with a single person holding the watermelon awkwardly in one hand in what looks like an unrelated chocolate aisle. The egocentric-only, exocentric-only, and combined configurations all preserve the multi-person produce context and a two-handed grip on the watermelon consistent with the prompt. The contrast is consistent with the metric-level finding that base lacks the in-distribution retail priors that the adapted configurations acquire.

Weighing the tomato crate.  4 shows a weighing-station scene where the person is supposed to lift a blue crate of tomatoes and place it onto a floor scale. Base never produces the bend-and-place motion required by the prompt and ends with the person positioned on (or directly above) the scale rather than the crate. The egocentric-only and exocentric-only configurations both execute a recognizable squat-to-scale motion, and the combined configuration produces the most explicit final frame in which the crate is in contact with the scale. The base failure here is one of action grounding rather than scene quality, and is exactly the type of error that the per-pixel reconstruction metrics fail to penalize but that the semantic and distributional metrics in 6 5 pick up.

Opening the fridge.  5 shows a refrigerator-side scene where the prompt asks the person to open the door and pick a vegetable from inside. Base produces an explicit geometry violation at \(t{=}5\) s in which the fridge door appears to pass through the person’s body. The adapted configurations preserve the open-door geometry and a plausible reach into the fridge. This is a low-level structural error in the base model that the adapted models avoid; it is the kind of mistake that disqualifies a world model for downstream embodied use, regardless of pixel-level similarity.

Open the freezer, take ice cream.  6 shows a freezer-aisle scene where the person opens a freezer cabinet to take out an ice-cream tub. Base distorts the freezer structure as the rollout progresses, producing a final frame in which the freezer has lost its rectilinear geometry. The adapted configurations preserve the freezer cabinet structure throughout the rollout, and exo-only produces the most natural-looking reach into the freezer. This example also illustrates the headline finding numerically reported in 6: exo-only and combined produce similar-quality outputs, but the small SSIM advantage of combined does not capture the additional semantic plausibility of exo-only’s reach geometry.

Carry the crate.  7 shows the person walking down an aisle carrying a yellow crate. Base produces a hand–object interpenetration at \(t{=}5\) s in which the person’s hands appear to pass through the crate while walking. All three adapted configurations preserve a plausible two-hand carry posture in which the hands remain on the crate handles. As in the fridge example above, this is a hand–object physics failure that the per-pixel reconstruction metrics fail to penalize but that is unmistakable to a human observer.

Pick up the metal scoop.  8 shows the person standing at the bulk-grain bins, ready to take a metal scoop and use it to fill a plastic bag with grain. Base produces a hard agent–object grasp failure at \(t{=}1\) s in which the scoop floats upward on its own with no hand around it. The egocentric-only, exocentric-only, and combined configurations all correctly close the person’s hand around the scoop and lift it as a single rigid body. This is a hand–object physics failure in the earliest stage of the rollout that disqualifies the base output for downstream embodied use regardless of pixel-level similarity, and is exactly the type of error that the per-pixel reconstruction metrics under-penalize.

Push the cart down the aisle.  9 shows a cart-navigation scene in which the person pushes a shopping cart loaded with small bottles down a backend aisle and then reaches into the cart with the right hand. The base configuration inserts a spurious opening in the back wall of the aisle at \(t{=}3\) s, exposing an unrelated section of the store that was not visible in the conditioning frame; this is a scene-structure insertion failure rather than a pure pixel-level error. The three adapted configurations all preserve the original aisle geometry and the product layout on the side shelves throughout the rollout. The exocentric-only configuration produces the most stable scene up to \(t{=}7\) s, consistent with the headline finding that exocentric-only adaptation specialises to wide-field scene structure.

Reach for the back-row bottle.  10 shows a beverage-cooler scene where the person reaches for a black bottle at the back of the shelf, picks it up, and places it in the front row. Base produces a hand-pose failure at \(t{=}3\) s in which the hand wraps the bottle in an unnatural configuration with no clear contact between fingers and bottle body. The egocentric-only, exocentric-only, and combined configurations all produce a recognisable back-row reach-and-grasp consistent with the prompt. This is a fine-grained hand-pose failure that is invisible to scene-level distributional metrics but immediately apparent to a human observer.

Carry the basket to the dairy section.  11 shows the person walking through the store aisles carrying a blue shopping basket with both hands, and approaching the dairy-section refrigerators. Base produces a hand–object interpenetration failure at \(t{=}3\) s and again at \(t{=}7\) s: the person’s fingers appear to enter the inside volume of the basket rather than wrapping around the handles. The egocentric-only, exocentric-only, and combined configurations all preserve a plausible two-hand basket carry posture throughout the rollout. This is the same class of physics failure as the crate example above, replicated on a different agent–object pair, and it is consistently absent from all three adapted configurations.

Inspect the cleaning-supplies bottle.  12 shows an inspection scene in which the person pushes a cart down the aisle, stops, and reaches with the right hand to inspect a red bottle of cleaning supplies on the shelf. Base produces a contact-physics artifact at \(t{=}7\) s in which the reaching hand penetrates the shelf guard rail rather than respecting it as a solid surface. The egocentric-only, exocentric-only, and combined configurations all preserve correct hand–shelf contact and produce a recognisable inspection pose. Together with the crate, basket, and fridge examples, this completes a family of four hand–surface and hand–object contact-physics failures in the base model, all of which the adapted configurations consistently avoid.

5.4 Temporal Structure of the Adaptation Gap↩︎

13 plots LPIPS as a function of rollout time on the test set. The absolute gap between the LoRA-adapted models and the pretrained base is widest at the shortest rollout time we sample (\(t{=}0.5\) s, base mean LPIPS \(0.542\) vs.exocentric-only \(0.460\), a gap of \(0.082\)) and narrows steadily as \(t\) grows, since all configurations drift away from the single ground-truth trajectory in the same way. 8 reports the paired LPIPS gap at fixed time stamps within this near-horizon regime.

The near-horizon LPIPS gap to the pretrained base is largest at the shortest rollout time we sample (\(t{=}0.5\) s, see 13 in the appendix) and remains substantial throughout the \(t{\in}[1.0,2.0]\) s window. At \(t{=}1.0\) s the exocentric-only configuration reaches a paired LPIPS gap of \(+0.075\) at \(89.0\%\) paired win-rate (\(p{<}10^{-28}\)) and the combined configuration \(+0.068\) at \(83.5\%\) (\(p{<}10^{-24}\)). At \(t{=}2.0\) s the gap remains substantial (\(+0.064\) to \(+0.070\) at \(87.5{-}96.0\%\) paired win-rate). At longer rollout horizons all configurations drift away from the single ground-truth trajectory in the same way, and the absolute LPIPS gap narrows; we omit those time stamps from the table for clarity and report the full curve in the appendix.

The near-horizon window has a clear interpretive meaning: it is the regime in which the world model’s prediction quality matters most for embodied use, since downstream planning and policy verification typically rely on rollouts of one to three seconds [8]. Our results indicate that LoRA adaptation is most beneficial precisely in this regime.

5.5 Ablations and Sanity Checks↩︎

Time-aligned vs.frame-index-based sampling.  A naive LPIPS implementation would compare generated and ground-truth frames by index, mis-aligning the generated 24 FPS output against ground-truth at 4 FPS. Switching to the wall-clock time-aligned protocol of 4.2 increases the paired LPIPS gap on our test set by approximately \(3.9\times\) over the index-based comparison. We recommend the time-aligned protocol for any video evaluation in which the generated and reference frame rates differ.

Adapter merge verification.  Across all 36 transformer blocks the post-merge weight differs from the pretrained weight in \(98.83\%\) of its entries, with per-tensor maximum delta magnitude of \(17.8\%\) of the base tensor magnitude. This excludes a class of failure modes in which the adapter loads silently as a zero increment due to a Diffusers PEFT name mismatch.

Seed variance.  The seed-induced spread on the combined configuration (4) is at least an order of magnitude smaller than the gap between any LoRA configuration and the base on every paired metric, supporting the interpretation that observed configuration-to-configuration ordering is not driven by optimization noise.

Modern metric sensitivity.  On our test set, the DreamSim paired improvement for the exocentric-only configuration corresponds to a relative reduction of \(23.6\%\) from the base mean, approximately \(2.7{\times}\) the LPIPS relative reduction of \(8.7\%\). On the distributional side, JEDi shows a \(33.5\%\) relative reduction for exocentric-only versus \(17.7\%\) for R3D-Fréchet, a \(1.9\times\) amplification. In both pairings the modern metric agrees with the classical metric on the configuration ranking but provides a more sensitive measurement of the adaptation effect, consistent with the design of DreamSim [52] and JEDi [51] to capture aspects of similarity that the classical metrics miss.

6 Conclusion↩︎

We introduced RetailSMV (Retail Synchronized Multi-View), a retail video corpus of \(32{,}105\) captioned clips with synchronized egocentric and exocentric capture of the same store-staff activities across five real-world supermarkets, and used it to conduct a controlled study of view-stratified Low-Rank Adaptation for foundation video world models. By training three matched LoRA configurations of Cosmos3-Nano on RetailSMV under identical hyperparameters and optimization budget, and by evaluating them on a held-out test set with a seven-metric suite under a strict paired statistical protocol, we have arrived at several conclusions of practical interest for the world-model adaptation community.

Adaptation is statistically uniform.  The reduction in validation diffusion loss is large (\(\sim\) \(2.8\times\)), uniform across configurations, and achieves perfect paired ordering (\(p\ll 0.001\)). On this aggregate signal alone there is no ambiguity that the pretrained foundation video model benefits substantially from domain-specific LoRA adaptation. Validation loss alone, however, does not discriminate between view-stratified data choices; the configuration differences emerge when generation quality is measured directly.

Exocentric-only adaptation matches or exceeds combined adaptation, with significant gains on LPIPS, PSNR, and DreamSim.  Our headline finding is that exocentric-only training, using only the \(15{,}985\) exocentric clips of the synchronized ego\(+\)exo corpus, matches or exceeds the combined configuration on six of the seven point estimates we report. The exocentric-only adapter wins on R3D-Fréchet (\(35.14\) vs.combined \(36.20\) vs.base \(42.69\)), on JEDi (\(0.829\) vs.combined \(0.892\) vs.base \(1.246\), a \(33.5\%\) relative reduction), on LPIPS (a paired reduction of \(0.058\) at \(100\%\) paired win-rate vs.combined’s \(0.049\) at \(94.5\%\)), on PSNR (an improvement of \(+0.575\) dB vs.\(+0.500\) dB), on Hessel CLIPScore (\(+0.018\) vs.\(+0.017\)), and on DreamSim (a reduction of \(0.062\) vs.\(0.046\)), even though the combined configuration also sees the \(16{,}120\) egocentric clips. The combined configuration edges exo only on SSIM, by \(0.0012\). Direct paired tests between the two adapted configurations (6) confirm that combined is significantly worse than exo on LPIPS, PSNR, and DreamSim, and the two distributional metrics (R3D-Fréchet and JEDi) agree on the ranking under either feature backbone. The corresponding combined-vs-egocentric tests (7) show the asymmetric counterpart: combined is significantly better than ego-only on LPIPS (\(p{=}0.002\)), PSNR (\(p{<}10^{-6}\)), and SSIM (\(p{<}10^{-14}\)). Adding the exocentric subset to the egocentric subset therefore helps, whereas adding the egocentric subset to the exocentric subset hurts. When synchronized ego/exo capture is available, single-view exocentric training is therefore a strong default; combined ego\(+\)exo training does not improve over exo-only training on this corpus. We hypothesize that this reflects the wider field of view and stable reference frame of the exocentric camera, which align well with the inductive prior of a diffusion video model that was pretrained primarily on stable third-person footage. What matters is in-distribution coverage of the deployment viewpoint, not raw clip count.

Egocentric data transfers, but ranks third.  The egocentric-only adapter trained on \(16{,}120\) egocentric clips improves over the pretrained base on every metric at \(p{<}10^{-8}\), including LPIPS (\(+0.041\) at \(90.5\%\) paired win-rate), CLIPScore (\(+0.017\) at \(72.5\%\)), and DreamSim (\(+0.043\) at \(78.0\%\)). The transfer is substantive but consistently weaker than the in-distribution exocentric-only adapter. This pattern may inform data-collection priorities for future embodied world models in retail-like deployment domains.

The adaptation gap is concentrated in the near-horizon prediction window.  The absolute LPIPS gap to the pretrained base is largest at the shortest rollout time we sample (\(t{=}0.5\) s, base mean LPIPS \(0.542\) vs.exocentric-only \(0.460\), a gap of \(0.082\)) and narrows steadily as rollout time grows, since all configurations drift away from the single ground-truth trajectory in the same way. In the \(t{\in}[1.0,2.0]\) s window the paired gap is still substantial: at \(t{=}1.0\) s the exocentric-only configuration reaches a paired LPIPS improvement of \(+0.075\) at \(89\%\) paired win-rate, approximately \(1.3\times\) the full-clip average; the combined and egocentric-only configurations exhibit the same near-horizon amplification at \(+0.068\) and \(+0.061\) respectively. The near-horizon window is precisely the regime in which world models are most directly used for embodied control [8]. We recommend reporting the LPIPS gap at fixed near-horizon time stamps in addition to the full-clip average as a standard reporting convention.

Modern metrics are more sensitive.  The DreamSim paired improvement for the exocentric-only configuration is approximately \(2.7\times\) the LPIPS paired improvement in relative terms (\(23.6\%\) vs.\(8.7\%\)), while the two metrics produce the same configuration ranking; on the distributional side, JEDi’s \(33.5\%\) relative reduction is roughly \(1.9\times\) the corresponding R3D-Fréchet reduction (\(17.7\%\)), again with identical configuration ordering. This is consistent with the design of DreamSim and JEDi to capture mid-level semantic and feature-distribution similarity that the classical metrics miss [51], [52]. We report JEDi as a recent distributional video-quality metric complementary to R3D-Fréchet; the conclusions in this paper do not rely on JEDi alone and are supported by R3D-Fréchet and DreamSim as well. We recommend that domain-specific video evaluations report DreamSim alongside (or in place of) LPIPS, and a V-JEPA-based MMD alongside an I3D/R3D Fréchet distance.

Limitations↩︎

Our study has four limitations we believe deserve explicit mention.

First, we adapt one base model family (Cosmos3-Nano). A natural extension is to evaluate whether the qualitative findings transfer to other foundation video models such as Cosmos-Predict 2.5, Sora, and Stable Video Diffusion. A parallel adaptation effort to Cosmos-Predict 2.5 is ongoing.

Second, our Fréchet-style metric uses a ResNet-3D Kinetics backbone (R3D-Fréchet) rather than the canonical I3D backbone, so absolute values are not directly comparable to I3D-FVD values in the broader literature; see 9 for details and a recipe for recomputing with the canonical I3D backbone.

Third, we do not conduct a human evaluation. The metrics community has documented that human judgment remains the gold standard for video generation quality [51], [52]. Our paired statistical protocol provides a transparent basis for any subsequent human-versus-automatic alignment study, and the modern metric we adopt (DreamSim) is designed precisely to narrow the gap to human judgment, but a direct human comparison would strengthen the conclusions.

Fourth, we use image-and-text conditioning. The natural use case for a world model in embodied control is action conditioning, where the model rolls out a counterfactual based on a proposed action sequence. Cosmos3-Nano does not natively support action conditioning; adding it is a promising direction for downstream evaluation as a true simulator for embodied control.

Beyond these specific limitations, the broader takeaway is that view-stratified studies, paired statistics, and modern perceptual metrics together support reproducible, well-controlled studies of parameter-efficient video world model adaptation. We hope the protocol and findings presented here are useful starting points for the next generation of domain-specialized generative simulators.

7 Reproducibility Details↩︎

7.1 Hardware and Software↩︎

All training and evaluation experiments run on four NVIDIA RTX PRO 6000 Blackwell (\(96\) GB HBM) GPUs in a single host. The software stack is Python 3.12 with CUDA 12.8 and PyTorch 2.11 (cu128 build, including the sm_120 device kernels required by the Blackwell architecture), together with Diffusers 0.39, PEFT 0.19, and Accelerate 1.13. The two modern metrics that require dedicated environments are isolated. DreamSim is run in a virtual environment with transformers\(<\)​4.50 (required by the official DreamSim distribution). JEDi is run in a separate environment containing the official videojedi package together with the V-JEPA ViT-H/16 feature extractor checkpoint released by the JEDi authors; this environment pins the V-JEPA preprocessing utilities to the versions used in the original “Beyond FVD” paper.

7.2 Hyperparameters↩︎

9 lists the hyperparameters shared by the three view-stratified configurations and the second-seed control. All four configurations use identical hyperparameters apart from the training data subset and (for the second-seed control) the random seed.

7.3 Inference Settings↩︎

For every (configuration, test clip) pair we run image-to-video inference with: 189 frames (Cosmos3-Nano’s native maximum, \(\sim 7.9\) seconds at 24 FPS), spatial resolution \(480{\times}832\), the UniPC multistep scheduler with \(\sigma_{\text{shift}}{=}10\), 20 inference steps, classifier-free guidance scale \(6\), negative prompt “low quality, blurry, distorted, glitches, watermark”, deterministic seed \(42\), output FPS 24.

7.4 Split Construction↩︎

We hold out clips from training and partition the held-out pool into a validation set used for adapter selection (\(1{,}388\) clips, from which we sample \(n{=}32\) paired clips for the rectified-flow validation loss every 100 training steps) and a test set used for final evaluation (\(200\) clips, balanced across both egocentric (\(n{=}100\)) and exocentric (\(n{=}100\)) views with rejection sampling for zero overlap with the validation set). The training set (\(32{,}105\) clips), the validation set (\(1{,}388\) clips), and the test set (\(200\) clips) are pairwise disjoint by per-clip unique identifier.

8 Extended Results↩︎

8.1 Exact Validation-Loss \(p\)-Values↩︎

All four LoRA configurations reduce validation diffusion loss on every one of the \(n{=}200\) paired evaluation samples. Under the Wilcoxon signed-rank test the per-row \(p\)-values are \(\sim10^{-105}\) for egocentric-only, \(\sim10^{-104}\) for exocentric-only, \(\sim10^{-104}\) for combined seed-42, and \(\sim10^{-105}\) for combined seed-137. These numbers reflect the nominal \(p\) associated with perfect paired ordering on \(n{=}200\) samples and should be interpreted as “every paired sample improves over the base”; we report \(p\ll 0.001\) in the main text accordingly.

8.2 Cross-View Egocentric Transfer Check↩︎

The main test set is a stratified \(200\)-clip pool covering both views. To isolate the cross-view behaviour of the three adapters, we report paired LPIPS on the egocentric-view subset of the main test set (\(n{=}100\) ego clips), so that the same generations used for the main metric panel are reused. 10 shows that all three adapted configurations significantly improve egocentric LPIPS on this subset, with the exocentric-only adapter producing the largest paired improvement (\(+0.045\) at \(100\%\) paired win-rate) – i.e., the exocentric-only adapter does transfer to the egocentric viewpoint when measured on these clips, and slightly outperforms the egocentric-only adapter (\(+0.039\) at \(96\%\) paired win-rate). The combined configuration is comparable to ego-only (\(+0.038\) at \(98\%\)). This pattern is consistent with the headline result: under matched optimization budget, the exocentric subset is doing the heavy lifting of adaptation, including at cross-view test time.

8.3 Rollout-Horizon Curve↩︎

13 plots mean LPIPS as a function of rollout time on the held-out 200-clip test set, one curve per configuration. All four curves rise monotonically with \(t\): at \(t{=}0.5\) s the base mean LPIPS is \(0.542\) versus \(0.460\) for the exocentric-only configuration (a gap of \(0.082\)); at \(t{=}4.0\) s the base reaches \(0.685\) versus exo \(0.627\) (a smaller gap of \(0.058\)). The absolute gap to the pretrained base is widest at the smallest rollout times and narrows steadily as \(t\) grows, since all configurations drift away from the single ground-truth trajectory in the same way. The exocentric-only configuration retains the smallest LPIPS at every time stamp.

9 Practical Notes↩︎

9.1 R3D-Fréchet vs.Canonical I3D-FVD↩︎

The canonical Fréchet video distance is computed on features extracted by the Inflated 3D ConvNet (I3D) of [44]. A turnkey I3D feature extractor with Kinetics-400 weights is not currently packaged with mainstream deep-learning frameworks; for this paper we use the closely related ResNet-3D-18 video classifier (torchvision.models.video.r3d_18 with KINETICS400_V1 weights) and refer to the resulting metric as R3D-Fréchet to make the deviation explicit. Both feature extractors are pretrained for Kinetics-400 action classification at the same input clip length, so the Fréchet distance retains its intended interpretation as a measure of distributional similarity in a Kinetics-aware embedding. Absolute values, however, are not directly comparable to I3D-FVD numbers from the broader literature. Recomputing the Fréchet-style metric with the canonical I3D backbone is straightforward but requires installing the I3D feature extractor outside the standard framework distribution; we recommend doing so for any direct comparison with prior work.

9.2 Time-Aligned Sampling↩︎

A frame-index-based implementation of LPIPS would mis-align Cosmos3-Nano’s 24 FPS generated output against the 4 FPS ground-truth clips, yielding pairings of frames separated by up to two seconds of wall-clock time. Wall-clock time-aligned sampling (16 evenly-spaced timestamps in \([0,\min(\text{gen\_dur},\text{gt\_dur})]\), dropping \(t{=}0\) to remove the trivial first-frame match) increases the paired LPIPS gap on our test set by approximately \(3.9\times\) over the index-based comparison. We recommend the time-aligned protocol for any video evaluation where the generated and reference frame rates differ.

9.3 Adapter-Merge Verification↩︎

For inference we explicitly merge the trained adapter \(\Delta W = (\alpha/r)\,BA\) into each corresponding base weight matrix and verify the merge numerically. Across all \(36\) transformer blocks the merged weight differs from the base in \(98.83\%\) of its entries, and the per-tensor maximum magnitude of \(\Delta W\) reaches \(17.8\%\) of the base weight magnitude. This sanity check rules out a class of silent-failure modes (adapter loading as a zero increment) that would otherwise produce numerically valid but functionally untrained outputs.

10 Release Policy and Per-Clip Data Availability↩︎

Release.  The release accompanying this paper will include a representative subset of RetailSMV sufficient to reproduce the reported metrics, the full set of evaluation scripts (data preparation, generation, metric computation, paired statistical tests), the generated videos used in the main results, and all per-clip metric files. Full corpus access is controlled: requests will be served on a case-by-case basis through the same channel as the released subset, subject to the privacy and licensing terms under which the corpus was collected (3.1).

Per-clip data.  For each (configuration, test clip) pair we provide per-clip time-aligned LPIPS, PSNR, SSIM, Hessel CLIPScore, and DreamSim, together with the per-clip wall-clock LPIPS series at \(t\in\{0.5,1.0,2.0,3.0,4.0,5.0,6.0,7.0\}\) seconds. We also provide the \(800\) generated videos (four configurations \(\times\) \(200\) test clips), the precomputed R3D feature arrays used to compute R3D-Fréchet, and the V-JEPA ViT-H/16 feature arrays used to compute JEDi (one \((B{,}D)\) array per configuration plus the ground-truth feature array).

11 Contributions and Acknowledgments↩︎

Core Contributors.  Amirreza Rouhi, Rajat Aggarwal, Parikshit Sakurikar, Anoop M.Namboodiri, Sashi P.Reddi.

Contact.  Correspondence and corpus-access requests can be directed to the core contributors at the following addresses:

• amir@dreamvu.ai

• rajat@dreamvu.ai

• parikshit@dreamvu.ai

• anoop@dreamvu.ai

• sashi@dreamvu.ai

References↩︎