1 Introduction↩︎

Recent advances in diffusion-based video generation [1]–[7] have made photorealistic synthesis increasingly accessible. As these models mature, multi-reference image-conditioned video editing (MRVE) emerges as a natural next step, enabling complex edits that coordinate information from multiple visual sources simultaneously. Consider transforming a street scene by applying cyberpunk lighting from one reference while replacing a character with a robot from another. While recent video editing methods have made progress in processing multiple reference images, existing methods struggle to accurately coordinate information from multiple visual sources, often failing to properly incorporate reference content or producing results inconsistent with editing instructions.

We trace this limitation to the lack of explicit reference relationships in existing instructions, which general-purpose vision-language models cannot precisely generate. First, existing training instructions, while containing reference anchors, rely on distant semantic dependencies (e.g., “replace the character with the person from reference image 2”), making it difficult for video editors to establish precise spatial binding between visual attributes and their sources. This leads to systematic failures that intensify with increasing reference count. Second, improving instruction quality requires strong comparative perception: the ability to distinguish visual attributes across multiple references and establish correct source-feature associations. However, recent work has shown that existing multimodal large language models (MLLMs) struggle with comparative video understanding [8], with this challenge becoming more severe in multi-reference scenarios. The lack of an appropriate intermediate representation that explicitly captures reference attribution is a key factor limiting MRVE performance.

To address this, we propose ReBind, a framework that introduces dense instructions with explicit reference relationships as the bridge between visual understanding and video generation. As illustrated in Figure 1, the core design principle is embedding explicit visual reference tokens directly at their corresponding semantic positions in the instruction, eliminating ambiguity by clearly binding each visual attribute to its source. This design is inspired by recent advances in interleaved image-text modeling [9], which leverage the contextual locality of transformers for accurate multi-image understanding. However, generating such instructions with accurate reference attribution requires specialized capability that general-purpose MLLMs lack. We therefore develop ReBind-Instruct, a vision-language model trained through a two-stage pipeline: Structured Instruction Learning via supervised fine-tuning teaches the model to generate structured instructions with explicit visual reference tokens, while Reference Attribution Optimization employs reinforcement learning with tailored rewards to enhance reference attribution accuracy. Building on these dense instructions, we develop ReBind-Edit by extending the LTX-2.3 video generation model to support multi-reference conditioning via cross-attention mechanisms that bind visual features from reference images to their corresponding textual anchors, enabling precise multi-reference coordination.

To evaluate instruction generation quality, we augment the existing UniVBench [10] with human-annotated ground-truth dense instructions for both single- and multi-reference cases. On this testset, ReBind-Instruct substantially outperforms general-purpose MLLMs, demonstrating the effectiveness of our specialized training approach. For video editing quality, evaluation on IntelligentVBench [11] and UniVBench shows that ReBind-Edit achieves competitive performance with closed-source models and state-of-the-art results among open-source methods, with particularly strong performance on tasks requiring multi-reference coordination. Furthermore, our ablation studies confirm that the proposed dense instructions substantially outperform the short instructions widely adopted in prior work, and that higher-quality dense instructions consistently lead to better video editing performance.

Our contributions are summarized as follows:

• We identify a critical deficiency in existing methods. Instead of dense captions, ReBind proposes a novel semantic instruction format that disambiguates reference relationships and enables models to better attend to pixel-level information.

• We develop ReBind-Instruct, a two-stage approach that addresses the fundamental weakness of existing MLLMs in discovering associations among multiple visual elements. Experiments demonstrate that ReBind-Instruct achieves near-parity with closed-source frontier MLLMs in reference attribution accuracy and intent recognition.

• Building upon ReBind-Instruct, ReBind-Edit enables lightweight adaptation of text-to-video models for multi-reference editing capabilities. ReBind-Edit substantially outperforms state-of-the-art open-source video editing models on UniVBench and IntelligentVBench benchmarks.

2 Related Work↩︎

2.1 Video Captioning↩︎

Current video captioning methods [12]–[16] have made substantial progress in generating dense and accurate descriptions for individual videos. ShareGPT4Video [17] leverages GPT-4V to produce high-quality dense captions and improve video instruction data, while LLaVA-Video [18] constructs large-scale synthetic corpora to enhance video understanding. Recent studies further extend video captioning beyond visual-only descriptions: Qwen3-Omni [19] supports joint audio-visual understanding with enhanced multimodal reasoning, and AVoCaDO [20] improves temporal alignment between audio and visual events. ViDiC [8] takes a step toward comparative video understanding by formulating video difference captioning over video pairs. However, existing methods do not address reference attribution in MRVE, where models must identify changes and ground each visual attribute to its corresponding reference source, requiring both comparative perception and explicit reference binding.

Reinforcement learning [21]–[23] has proven effective for optimizing fine-grained video understanding beyond supervised fine-tuning. Video-SALMONN 2 [24] employs direct preference optimization (DPO) [21] to enhance audio-visual reasoning, while DiaDem [25] applies DPO for speaker attribution in dialogue captioning. GRPO-based methods demonstrate effectiveness through carefully designed rewards. VideoCap-R1 [26] optimizes action descriptions with structured reasoning rewards, and AVoCaDO [20] improves temporal alignment through checklist-based rewards. Building on this paradigm, we apply GRPO with specialized rewards targeting reference attribution accuracy, a previously unaddressed objective in multi-reference editing.

2.2 Reference-Conditioned Video Editing↩︎

Text-guided video editing [27]–[36] has enabled semantic manipulation of video content, but it remains limited in fine-grained control over appearance, identity, and localized visual attributes. To improve controllability, reference-conditioned video editing methods [37]–[42] introduce visual conditions to better preserve target appearance and editing fidelity. More recently, unified video models [43]–[46] integrate video understanding, generation, and editing within a single framework, further broadening the scope of reference-conditioned manipulation. Notably, Bernini [47] unifies MLLMs and diffusion models through semantic planning in embedding space, achieving strong performance on video generation and editing with single-reference image-conditioned video editing (RVE) inputs. However, multi-reference image-conditioned video editing (MRVE) remains an open challenge. When generalized to MRVE tasks, they suffer from reference interference, ambiguous source-target correspondence, and inconsistent attribute extraction.

3 Dataset Construction↩︎

The multi-reference image-conditioned video editing (MRVE) task requires training samples consisting of multi-reference images, an input video, and a target video. We construct a synthetic dataset leveraging Bernini [47], a text-guided complex video editing model. As illustrated in Figure 2, our approach begins by collecting target videos from OpenVid-1M [48] and AVSET-10M [49]. We then apply the editing model to these target videos with inverse instructions, thereby generating the corresponding source videos through reverse editing operations.

To ensure dataset quality, we employ a two-stage filtering process: (1) video-to-video cosine similarity to remove low-correspondence pairs, and (2) MLLM verification to eliminate incorrectly edited samples. For multi-reference image construction, we first utilize Qwen3-Omni-30B [50] to identify and describe entities that undergo changes between source and target videos. We then apply SAM3 [51] to extract multi-frame object masks for each entity, followed by Flux2 [52] image inpainting to generate clean reference candidates. Finally, we select high-quality, front-facing images from each entity’s candidate set as the final results. This pipeline yields approximately 100K video editing triplets with clean multi-reference images, consisting of approximately 50% single-reference and 50% multi-reference samples. These samples provide the visual data for our framework, with dense instruction annotations constructed in the corresponding training stages described below.

4 Method↩︎

Our framework consists of two key components: a dense instructor that generates structured instructions from video pairs and reference images, and a video editor that performs precise editing guided by these instructions. We train the dense instructor through a two-stage pipeline: (1) Structured Instruction Learning (Sec. 4.1) to learn the structured instruction format through supervised fine-tuning, and (2) Reference Attribution Optimization (Sec. 4.2) to optimize multiple reward objectives using reinforcement learning with Group Relative Policy Optimization (GRPO). The video editor (Sec. 4.3) is then trained on video editing triplets with dense instructions as text guidance.

4.1 Stage 1: Structured Instruction Learning↩︎

We initialize our ReBind-Instruct from Qwen3-Omni-30B-A3 [19] and fine-tune it on a curated dataset of MRVE samples.

Training Dataset. To train the ReBind-Instruct, we select approximately 40K samples from the constructed triplets and annotate them with ground-truth dense instructions using Gemini 3.1 Pro. Each training instance consists of a source video \(V_{src}\), an edited video \(V_{edit}\), reference images \(\{I_1, \ldots, I_K\}\), and a ground-truth dense instruction \(S_{gt}\). Crucially, these instructions contain explicit visual references using special tokens that establish precise bindings between textual descriptions and their corresponding visual sources.

To ensure annotation quality, we apply machine detection to filter erroneous data through automated quality checks: (1) reference token consistency verification to ensure all visual reference tokens are correctly formatted and match the provided images, and (2) rule-based filtering to detect length anomalies, repetitive content, and format errors. Detailed filtering prompts are provided in the Appendix.

Training Objective. We optimize the standard next-token prediction objective: \[\label{eq:sft} \mathcal{L}_{SFT} = -\sum_{t=1}^{T} \log P_\theta(s_t \mid s_{<t}, V_{src}, V_{edit}, I_1, \ldots, I_K)\tag{1}\] where \(s_t\) denotes the \(t\)-th token in \(S_{gt}\), \(T\) is the sequence length, and \(\theta\) represents the model parameters. This stage teaches the model to generate structured instructions with explicit visual references (e.g., <Image_1>) at semantically appropriate positions by comparing the source and edited videos. After SFT, the model achieves strong capability in generating dense instructions for both single- and multi-reference image-conditioned video editing scenarios.

4.2 Stage 2: Reference Attribution Optimization↩︎

While SFT provides a strong initialization, it cannot directly optimize for task-specific objectives such as content completeness and reference grounding accuracy. To address this, we apply GRPO to further improve ReBind-Instruct by optimizing a composite reward function. We first describe the training data preparation, then introduce the GRPO algorithm and our reward design.

Training Dataset. For the Reference Attribution Optimization stage, we randomly sample approximately 2K video editing samples from the constructed dataset and manually annotate them with ground-truth dense instructions. To enable efficient reward evaluation during GRPO training, we perform offline extraction of reward components from human-annotated instructions using GPT-5.1, including dual checklist keypoints and relation triples.

Group Relative Policy Optimization. GRPO is a reinforcement learning algorithm that optimizes the policy by comparing outputs within groups. For each training instance \((V_{src}, V_{edit}, \{I_1, \ldots, I_K\}, S_{gt})\), we sample \(G\) candidate instructions \(\{S_1, \ldots, S_G\}\) from the current policy \(\pi_\theta\). We then compute a reward \(R(S_g)\) for each candidate and normalize rewards within the group: \[\label{eq:grpo95advantage} A_g = \frac{R(S_g) - \mu_R}{\sigma_R}\tag{2}\] where \(\mu_R\) and \(\sigma_R\) are the mean and standard deviation of rewards within the group. The policy is updated by maximizing: \[\label{eq:grpo95objective} \mathcal{L}_{GRPO} = \mathbb{E}_{S_g \sim \pi_\theta} \left[ A_g \cdot \log \pi_\theta(S_g) \right] - \beta \cdot D_{KL}(\pi_\theta \| \pi_{ref})\tag{3}\] where \(\pi_{ref}\) is the reference policy (the SFT model), \(D_{KL}\) is the KL divergence, and \(\beta\) controls the KL penalty strength. This group-relative formulation provides more stable gradients and naturally handles reward scale variations.

4.2.1 Reward Design↩︎

To ensure both content completeness and reference grounding accuracy in MRVE instructions, we design a reward framework consisting of three core components: (1) dual checklist rewards \(\mathcal{R}_{D}\) comprising \(\mathcal{R}_{pre}\) and \(\mathcal{R}_{diff}\) for content quality evaluation through missing and hallucination analysis, (2) referential consistency reward \(\mathcal{R}_{C}\) for accurate reference attribution through structured triple matching, and (3) length reward \(\mathcal{R}_{L}\) to mitigate repetition collapse and encourage appropriate instruction length.

4.2.2 Dual Checklist Reward↩︎

To enhance instruction completeness and accuracy, we propose dual checklist-based reward that evaluate content coverage through missing and hallucination analysis. The key motivation for dual rewards stems from the fundamental asymmetry in video editing captions: they must simultaneously describe what remains unchanged and what gets modified. These two aspects require distinct evaluation strategies due to their different semantic and structural requirements:

Preserved content refers to elements that maintain temporal continuity from the source video <Video_1>. These descriptions focus on capturing the original dynamic or static features that persist through the editing process, such as “the walking motion from <Video_1>” or “the urban background preserved from the original scene”. The evaluation emphasizes whether the model correctly identifies and anchors unchanged content to the source video.

Edited content refers to elements that are modified by incorporating features from reference images <Image_X>. These descriptions must explicitly attribute the modified features to their specific reference sources, such as “the sunset sky from <Image_1>”. The evaluation requires both identifying the modified content and ensuring correct reference attribution.

Each annotated ground-truth instruction \(S_{gt}\) includes two distinct keypoint sets: preserved content keypoints \(\mathcal{K}_{pre}\) and edited content keypoints \(\mathcal{K}_{diff}\). We merge both sets into a unified keypoint set \(\mathcal{K} = \mathcal{K}_{pre} \cup \mathcal{K}_{diff}\) for evaluation.

Given the unified keypoint set \(\mathcal{K}\) and generated instruction \(S_{gen}\), we employ GPT-5.1 to identify three types of events: (1) missing keypoints that exist in \(\mathcal{K}\) but are absent in \(S_{gen}\), (2) incorrectly described keypoints from \(\mathcal{K}\) that are mentioned but inaccurately portrayed in \(S_{gen}\), and (3) hallucinated content that appears in \(S_{gen}\) but is absent from \(\mathcal{K}\). For each identified missing or incorrect keypoint, we further categorize it as belonging to \(\mathcal{K}_{pre}\) or \(\mathcal{K}_{diff}\). Let \(\mathcal{M}_{pre}\), \(\mathcal{I}_{pre}\) denote missing and incorrect keypoints from preserved content, \(\mathcal{M}_{diff}\), \(\mathcal{I}_{diff}\) denote those from edited content, and \(\mathcal{H}\) denote hallucinated keypoints. We compute separate rewards for each content type: \[\mathcal{R}_{pre} = 1 - \frac{|\mathcal{M}_{pre}| + |\mathcal{I}_{pre}| + |\mathcal{H}|}{|\mathcal{K}_{pre}|}\] \[\mathcal{R}_{diff} = 1 - \frac{|\mathcal{M}_{diff}| + |\mathcal{I}_{diff}| + |\mathcal{H}|}{|\mathcal{K}_{diff}|}\] where \(\mathcal{R}_{pre}\) evaluates preserved content description accuracy and \(\mathcal{R}_{diff}\) evaluates edited content description with reference attribution. The final dual checklist reward is: \(\mathcal{R}_{D} = \mathcal{R}_{pre} + \mathcal{R}_{diff}\).

4.2.3 Referential Consistency Reward↩︎

While checklist-based rewards evaluate content quality by verifying mentions of visual elements, they remain insufficient to assess whether these elements are correctly attributed to their source references. In reference-image coniditioned video editing scenarios, a particularly severe failure mode manifests: cross-reference confusion, wherein the model accurately identifies visual content but assigns it to incorrect reference sources, thereby actively corrupting the editing process with plausible yet fundamentally incorrect directives.

To address this challenge, we introduce a referential consistency reward \(\mathcal{R}_{cons}\) that prevents reference confusion at two levels through structured object-attribute-reference triples. Each anchoring relation is formulated as a triple \(a_i = \langle o_i, d_i, r_i \rangle\) comprising an object identifier, visual attribute, and reference anchor. This design enforces correct attribution both within individual triples and across multiple triples to prevent cross-reference confusion. Ground-truth relations constitute the set \(\mathcal{A}_{gt} = \{a_1^{gt}, \ldots, a_M^{gt}\}\) whilst generated relations form \(\mathcal{A}_{gen} = \{a_1^{gen}, \ldots, a_N^{gen}\}\).

We employ GPT-5.1 to extract triples from both ground-truth and generated captions, then construct a binary matching matrix \(W \in \{0,1\}^{M \times N}\) where each element \(w_{i,j}\) indicates whether triple \(a_i^{gt}\) matches triple \(a_j^{gen}\): \[\label{eq:triple95score} w_{i,j} = \begin{cases} 1, & \text{if } o_i = o_j \text{ and } r_i = r_j \text{ and } d_i = d_j \\ 0, & \text{otherwise} \end{cases}\tag{4}\] A match requires exact agreement on all three components: object identity, reference anchor, and visual descriptor. This strict matching criterion eliminates cross-reference confusion by assigning zero score to cases where the object and descriptor are correct but anchored to the wrong reference. We decompose the evaluation into recall and precision. Recall measures how well the generated instruction covers ground-truth content by finding, for each GT triple, whether any generated triple matches it: \[\label{eq:ref95recall} \text{Recall} = \frac{1}{M} \sum_{i=1}^{M} \max_{j=1}^{N} w_{i,j}\tag{5}\] Precision measures how accurate the generated content is by finding, for each generated triple, whether any GT triple matches it: \[\label{eq:ref95precision} \text{Precision} = \frac{1}{N} \sum_{j=1}^{N} \max_{i=1}^{M} w_{i,j}\tag{6}\] The referential consistency reward combines these components via F1-score: \[\label{eq:ref95consistency95f1} \mathcal{R}_{C} = \frac{2 \cdot \text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}}\tag{7}\]

4.2.4 Length Reward↩︎

To mitigate output repetition collapse and enhance inference efficiency, we design a length-regularized reward that encourages complete captions while penalizing excessive length. Following [20], we adopt a unidirectional penalty structure. The thresholds \(\tau_1 = 512\) and \(\tau_2 = 1024\) are set based on empirical analysis, where \(\tau_1\) is approximately the longest instruction in the ground-truth distribution and \(\tau_2\) is approximately twice the maximum ground-truth length: \[\label{eq:length} \mathcal{R}_{L} = \begin{cases} 1.0, & \text{if } L < \tau_1 \\ 1 - \frac{L - \tau_1}{\tau_2 - \tau_1}, & \text{if } \tau_1 \leq L < \tau_2 \\ 0.0, & \text{otherwise} \end{cases}\tag{8}\] where \(L\) denotes the token count of the generated caption.

4.3 Reference-Conditioned Video Editor↩︎

After training ReBind-Instruct, we develop ReBind-Edit that performs precise video editing guided by dense instructions with explicit reference tokens. We build our editor upon LTX-2.3 [53], a representative text-to-video generation model.

Architecture. We build upon LTX-2.3 [53], a text-to-video diffusion transformer originally designed for text-guided generation. To enable multi-reference image-conditioned video editing, we extend the model to accept a concatenated latent sequence: \(z = [z_{ref_1}, \ldots, z_{ref_K}, z_{src}, z_{edit}]\), where reference images and source video serve as clean conditioning, while the edited video undergoes the diffusion process [54]. The source video \(V_{src}\) includes both visual and audio modalities, with audio providing semantic context to guide visual editing while remaining unmodified in the output. The model incorporates cross-attention layers that bind visual features from reference images to their corresponding textual anchors (<Image_X>, <Video_X>) in the dense instruction, enabling precise extraction of attributes from designated sources as directed by the instruction. This design allows the model to handle variable numbers of references within a unified architecture.

Training. We train the reference-image conditioned model on video editing samples \((V_{src}, V_{edit}, \{I_1, \ldots, I_K\}, S_{gt})\) where \(V_{src}\) is the source video, \(V_{edit}\) is the edited video, \(\{I_1, \ldots, I_K\}\) are reference images, and \(S_{gt}\) is the dense instruction generated by our trained dense captioner. We encode them into latents and construct the concatenated sequence \(z = [z_{ref_1}, \ldots, z_{ref_K}, z_{src}, z_{edit}]\). During training, we apply noise only to the edited video portion \(z_{edit}\), obtaining \(z_{edit,t} = \sqrt{\bar{\alpha}_t} z_{edit} + \sqrt{1-\bar{\alpha}_t} \epsilon\) where \(\epsilon \sim \mathcal{N}(0, I)\). The training objective is: \[\label{eq:editor95unified} \mathcal{L} = \mathbb{E}_{z,t,\epsilon} \left[ \| \epsilon - \epsilon_\theta([z_{ref_1}, \ldots, z_{ref_K}, z_{src}, z_{edit,t}], t, S_{gt}) \|^2 \right]\tag{9}\] where \(\epsilon_\theta\) is the denoising network that processes the full concatenated sequence with cross-attention to the dense instruction \(S_{gt}\).

5 Experiments↩︎

We conduct extensive experiments to validate the effectiveness of our approach across three key dimensions. First, we evaluate whether our specialized two-stage training produces higher-quality reference-grounded instructions compared to general-purpose vision-language models. Second, we investigate whether dense instructions with explicit reference attribution actually improve video editing quality over conventional short instructions through controlled ablation studies. Third, we benchmark our unified video editor against state-of-the-art methods on both single- and multi-reference editing tasks.

5.1 Implementation Details↩︎

ReBind-Instruct. We build ReBind-Instruct upon Qwen3-Omni-30B-A3B-Instruct, a pre-trained vision-language model with audio-visual understanding capabilities. We fully fine-tune the model throughout both training stages. Training proceeds through a two-stage progressive scheme: Stage 1 with supervised fine-tuning on 40K video editing triplets for 3 epochs at batch size 128 and learning rate 1e-5, followed by Stage 2 with GRPO on 2K samples with 8 candidates per sample at learning rate 5e-6 and KL penalty \(\beta=0.04\) on 32 H200 GPUs.

ReBind-Edit. We build ReBind-Edit on top of LTX-2.3 [53], using Gemma-3-12B-IT [63] as the text encoder and its 14B-parameter DiT backbone, which is fully fine-tuned. We train on approximately 100k video editing triplets with a batch size of 8 and a learning rate of 1e-6. For all editing tasks, we generate videos at 720P resolution with 81 frames, following the LTX-2.3 configuration.

5.2 Evaluation Setup↩︎

Benchmarks. We evaluate on IntelligentVBench with single-reference cases covering object addition, removal, replacement, and UniVBench with single-reference and multi-reference cases across eight dimensions. Since UniVBench lacks dense instructions with explicit reference attribution, we manually annotate 1,240 samples in total covering both single-reference and multi-reference scenarios, with human annotators analyzing video pairs and reference images to generate structured instructions with explicit reference attribution, followed by expert review to ensure annotation quality and consistency.

Metrics. For evaluating instruction quality, we propose two metrics. Instruction Quality measures content completeness: we use GPT-5.1 to compare the generated instruction against ground-truth keypoints, identifying missing content, incorrect descriptions, and hallucinated elements, with the score reflecting the ratio of correctly captured keypoints. Reference Accuracy measures reference attribution accuracy through structured triple matching: we extract object-descriptor-anchor triples from both generated and ground-truth instructions, then compute the F1 score based on exact matches where all three components must agree. For video editing quality, we follow IntelligentVBench to evaluate Instruction Following, Condition Preserving, and Visual Quality, and follow UniVBench to evaluate across eight dimensions: Subject, Background, Action, Camera, Color, Lighting, Style, and Relative Position, with GPT-5.1 as the judger.

Baselines. For reference-grounded instruction generation, we compare with general-purpose vision-language models Gemini-2.5-Pro [59], Gemini-3.1-Pro, Qwen2.5-Omni-3B, Qwen2.5-Omni-7B, Qwen3-Omni-Captioner-30B-A3B, Qwen3-Omni-Instruct-30B-A3B, and Gemma4-31B-it [63]. For video editing on IntelligentVBench and UniVBench, we compare with open-source methods UniVideo [44], VINO [57], Kiwi-Edit [58], and OmniWeaving [11], along with closed-source models HappyHorse 1.0 [56] and Kling-o3 [55].

5.3 Reference-Image Conditioned Video Editing Results↩︎

Table 1 presents quantitative results on IntelligentVBench. Our method achieves the best overall average performance among all methods including closed-source systems. Specifically, we outperform the closed-source Kling-o3 model by 0.17 in overall average, and achieve 0.64 improvement over HappyHorse 1.0. Compared to the strongest open-source baseline OmniWeaving, we demonstrate a 0.49 improvement. These results establish that our dense instruction approach with explicit reference attribution is highly effective for video editing. Qualitative results in the Appendix further demonstrate visually coherent edits with accurate attribute extraction from reference images. Table 2 presents results on UniVBench for both reference video editing (RVE) and multi-reference video editing (MRVE). Our method achieves the best overall performance among open-source methods, with 0.41 improvement over the strongest baseline UniVideo (hidden) on RVE and 0.60 on MRVE. These results validate that dense instructions with explicit reference attribution substantially improve editing quality, particularly in multi-reference scenarios where precise coordination across visual sources is critical.

Qualitative results reveal systematic failure modes across baselines: OmniWeaving exhibits cross-reference confusion, Kiwi-Edit shows minimal editing with poor reference consistency, and UniVideo suffers from severe temporal inconsistencies. Even closed-source systems struggle with identity preservation across frames. In contrast, our method produces accurate multi-reference coordination with correct feature attribution and strong temporal consistency, validating the effectiveness of dense instructions with explicit reference tokens.

5.4 Dense Instruction Generation Quality↩︎

Table 3 evaluates our dense instructor against state-of-the-art vision-language models on the annotated UniVBench multi-reference subset. Our method achieves state-of-the-art performance among all models, including closed-source systems. Compared to the strongest open-source baseline Gemma4-31B-it, our method achieves 18.8% overall improvement. Compared to the closed-source Gemini-3.1-Pro, we demonstrate 5.11% overall improvement. This validates our core hypothesis that generating reference-grounded editing instructions with explicit visual anchoring requires specialized training beyond general-purpose MLLM pretraining.

5.5 Ablation Study↩︎

5.5.1 Impact of Dense Instructions on Editing Quality↩︎

To isolate the impact of instruction design and quality on editing performance, we train ReBind-Edit on 50k randomly sampled training cases for 5k steps with three types of instructions: basic instructions from Qwen3-Omni-Instruct, dense instructions with explicit reference tokens from the same Qwen3-Omni-Instruct, and dense instructions from our ReBind-Instruct.

Table 4 shows that instruction structure matters: dense instructions improve editing quality by 0.45 over basic instructions using the same MLLM. Moreover, instruction quality matters: our ReBind-Instruct further improves results by 0.41 over Qwen3-Omni-Instruct with the same dense format. These results demonstrate that higher-quality instructions directly translate to better editing performance.

5.5.2 Impact of Different Reward Components↩︎

In Table 5, we conduct an in-depth analysis of each component within our GRPO training pipeline. Stage 1 supervised fine-tuning yields substantial improvements over the base Qwen3-Omni-Instruct model with 14.28% overall gain, demonstrating the effectiveness of learning structured instructions on curated editing triplets. During Stage 2 GRPO optimization, we progressively add reward components. The dual checklist rewards \(\mathcal{R}_{D}\) contribute 4.00% overall gain, the referential consistency reward \(\mathcal{R}_{C}\) adds 1.42%, and the length reward \(\mathcal{R}_{L}\) contributes 0.78%. These results validate that multi-objective optimization through tailored rewards is essential for reference-grounded instruction generation.

5.6 Conclusion↩︎

We address multi-reference video editing by introducing dense instructions with explicit reference attribution as the intermediate representation. Our key insight is that the bottleneck lies in generating appropriate instructions rather than generation model capacity. We develop a two-stage training pipeline to generate instructions with explicit visual reference tokens completely and accurately. We further design a editor model that binds visual features to textual anchors through cross-attention mechanisms. Experiments demonstrate state-of-the-art performance on both instruction generation and video editing benchmarks, validating that explicit reference grounding is essential for controllable multi-reference video editing.

Appendix↩︎

6 Evaluation Benchmark Details↩︎

To complement the evaluation setup described in Section 5.2, we provide detailed information about the benchmarks used in our experiments.

6.1 IntelligentVBench↩︎

IntelligentVBench [11] is a comprehensive benchmark designed to assess next-level intelligent unified video generation by integrating abstract reasoning and free-form composition. We evaluate our method on the Text-Image-Video-to-Video (TIV2V) subset, which assesses cross-modal compositional editing capabilities. The TIV2V task requires models to extract specific concepts from reference images and integrate them into source videos while preserving temporal consistency and visual fidelity of non-target elements. This subset comprises 210 test cases spanning three operational objectives: inserting an object from a reference image into the source video (71 cases), substituting a specific visual element within the video with one from the reference image (73 cases), and replacing the video’s original background with the scene depicted in the reference image (66 cases). The benchmark employs Gemini-2.5-Pro as an automatic evaluator with three metrics rated on a 1-5 scale: Instruction Following (IF) measures the model’s capacity to seamlessly integrate video and image in strict adherence to instructions, Condition Preserving (CP) evaluates preservation of the source video’s unedited regions and verification that newly introduced elements faithfully conform to reference images, and Visual Quality (VQ) assesses aesthetic quality, temporal consistency, and motion naturalness. Overall performance is measured by AVG = (IF + CP + VQ)/3 and MIN = Min(IF, CP, VQ).

6.2 UniVBench↩︎

UniVBench [10] is the first unified benchmark providing comprehensive evaluation across video understanding, generation, and editing tasks with multi-shot support and copyright-free content. For reference-guided video editing evaluation, UniVBench employs an agentic evaluation system UniV-Eval that decomposes videos into shot-level units and performs fine-grained assessment across nine major category groups: Subject, Relative Position, Actions, Background, Color, Lighting, Style, Atmosphere, and Camera. These categories are further decomposed into 21 specific sub-dimensions that enable diagnostic feedback at the shot level. The evaluation agent performs per-category comparisons between model outputs and input tuples (source video, reference images, text instructions), producing structured weakness checklists that highlight fine-grained deficiencies. Final scores are aggregated along multiple evaluation dimensions, with each dimension scored on a scale from 0 to 10. While UniVBench originally uses Seed-1.6 as the evaluation LLM, we adopt GPT-5.1 for our experiments to ensure more reliable and consistent evaluation quality. UniVBench contains diverse test cases covering both single-reference and multi-reference video editing scenarios. For our evaluation, we manually annotate 1,240 samples (620 single-reference and 620 multi-reference) with dense instructions featuring explicit reference attribution to enable fair comparison of instruction generation quality and video editing performance.

7 Baseline Methods Details↩︎

To provide context for the comparative evaluation in Section 5, we present detailed descriptions of all baseline methods compared in our experiments.

UniVideo [44] is a unified framework that integrates video understanding, generation, and editing within a single architecture. It adopts a dual-stream design combining a Multimodal Large Language Model (MLLM) for instruction understanding with a Multimodal DiT (MMDiT) for video generation. UniVideo supports two distinct conditioning modes for reference image integration: (1) query-based mode extracts reference features through learned query tokens and injects them via cross-attention, enabling flexible composition, and (2) hidden-state-based mode directly incorporates the MLLM’s hidden states into the generation process, providing richer semantic guidance. These two modes offer different trade-offs between compositional flexibility and semantic fidelity.

Kiwi-Edit [58] is a unified video editing framework designed for instruction-reference-following tasks with single reference images. The method synergizes learnable queries and latent visual features for reference semantic guidance, processing reference images through dual pathways: learnable query tokens for flexible composition and latent visual features for precise visual control. Features are fused with the video generation backbone through cross-attention mechanisms. While originally designed for single-reference scenarios, we evaluate its performance on multi-reference editing tasks to assess its generalization capability.

OmniWeaving [11] is a unified video generation model that combines a multimodal large language model with a latent diffusion model for video synthesis. The framework integrates video generation and editing tasks within a single architecture, trained on massive-scale pretraining data encompassing compositional and reasoning-augmented scenarios. The model employs attention-based feature fusion mechanisms to process interleaved text, multi-image, and video inputs. We evaluate its generalization capability on multi-reference video editing tasks.

VACE [39] is an all-in-one framework for video creation and editing built on Diffusion Transformers. VACE unifies multiple video synthesis tasks including reference-to-video generation, video-to-video editing, and masked video-to-video editing through a Video Condition Unit (VCU) that organizes diverse task inputs into a unified interface. The framework employs a Context Adapter structure to inject task-specific concepts via formalized temporal and spatial representations. We evaluate two variants: VACE-Wan2.1 built on Wan2.1 and VACE-LTX built on LTX.

VINO [57] is a unified visual generator that performs image and video generation and editing within a single framework. VINO couples a vision-language model with a Multimodal Diffusion Transformer (MMDiT), where multimodal inputs (text, images, videos) are encoded as interleaved conditioning tokens to guide the diffusion process. This design supports multi-reference grounding, long-form instruction following, and coherent identity preservation across static and dynamic content without modality-specific architectural components.

8 Additional Implementation Details↩︎

To ensure reproducibility of the results presented in Section 5, we provide comprehensive training hyperparameters for all three training stages described in Section 4. This section details the optimization settings, learning rate schedules, parallelization strategies, and GRPO-specific parameters used in our experiments.

Table 6 presents the detailed training hyperparameters across all three training stages. Stage 1 uses supervised fine-tuning on ReBind-Instruct with a learning rate of \(1.0 \times 10^{-5}\) and cosine learning rate scheduling with warmup for 3 epochs. Stage 2 applies GRPO reinforcement learning to further optimize ReBind-Instruct with GRPO-specific parameters such as beta=0.04 for KL penalty control and 8 generations per training step. Stage 3 trains ReBind-Edit using dense instructions from ReBind-Instruct with the same base learning rate of \(1.0 \times 10^{-5}\) but with a warmup-constant-cosine scheduler for 15K steps. All stages use mixed precision training and employ gradient checkpointing for memory efficiency.

9 Details of Training Dataset↩︎

In this section, we present detailed specifications of our data collection, filtering, and annotation procedures.

9.1 Target Video Selection Criteria↩︎

Target videos are collected from OpenVid-1M and AVSET-10M based on content suitability for editing operations. We prioritize videos with clear subjects and stable camera motion to facilitate reliable reverse editing. Specifically, videos are filtered to ensure: (1) minimum resolution of 720P and duration between 2-10 seconds to balance visual quality with computational efficiency, (2) presence of identifiable entities (people, objects, or scenes) suitable for attribute-level editing, and (3) absence of extreme motion blur or occlusion that could compromise editing quality. Videos containing predominantly text overlays or with severe compression artifacts are excluded through automated quality checks.

9.2 Inverse Instruction Design↩︎

To generate source videos through reverse editing, we design inverse instructions that specify attribute transformations opposite to the desired editing direction. For each target video, we first identify editable attributes using automated captioning models, then construct inverse instructions following templates such as "change the [entity] to have [inverse attribute]" or "replace the [entity] with one having [contrasting property]." For example, if the target video shows a person wearing red clothing, the inverse instruction might specify changing to blue clothing. This ensures the generated source video provides a meaningful starting point for forward editing tasks.

9.3 Quality Filtering Thresholds↩︎

In the video-to-video similarity filtering stage, we compute cosine similarity between CLIP [64] embeddings extracted from source and target videos. Pairs with similarity below 0.3 indicate failed edits where the source and target are semantically unrelated, while similarity above 0.95 suggests insufficient editing variation. Both extremes are filtered to ensure the dataset contains meaningful editing challenges. For MLLM verification, GPT-5.1 is prompted to evaluate whether the visual changes between source and target align with the inverse editing instruction. Samples where the model identifies semantic inconsistencies, incomplete edits, or visual artifacts are flagged and excluded. This two-stage process removes approximately 15% of initial pairs.

9.4 Reference Image Selection Criteria↩︎

From the multi-frame mask candidates generated by SAM3 and inpainted by Flux2, we select final reference images based on visual quality metrics. Specifically, we prioritize: (1) front-facing views where the entity is clearly visible with minimal occlusion (occlusion ratio below 20%), (2) high visual clarity measured by Laplacian variance to exclude blurry frames, and (3) completeness where the entity is fully within the frame boundary. For each entity, we select the highest-quality image as reference. In multi-reference scenarios involving 2-5 reference images per sample.

9.5 Dataset Statistics↩︎

The final dataset contains approximately 100K video editing triplets with the following distribution: 52.3% single-reference samples (one entity edited) and 47.7% multi-reference samples (2-3 entities edited simultaneously). Videos range from 2-10 seconds in duration with resolutions between 480p and 1080p. Content categories include people (49.2%), animals (12.6%), objects (32.8%), and scenes (5.4%). Each sample is associated with 1-5 reference images and corresponding dense instruction annotations generated through the two-stage training pipeline described in Section 3.2.

9.6 Human Annotation for GRPO Training↩︎

To provide high-quality supervision for Stage 2 GRPO training, we recruit eight experienced annotators to manually create dense instructions for 2K samples from the synthetic dataset. Annotators are provided with the source video, target video, and reference images, and are tasked with writing dense instructions that: (1) explicitly bind visual attributes to reference images using standardized tokens (e.g., <image_1>, <image_2>), (2) clearly distinguish between preserved and modified elements, and (3) provide sufficient detail to guide the editing operation. While these ground-truth instructions are human-annotated, auxiliary training signals such as checklists and reward scores are automatically generated by GPT-5.1 for scalability.

9.7 Human Annotation for UniVBench Evaluation↩︎

To evaluate instruction generation performance on UniVBench, we manually annotate 1,240 samples from the benchmark dataset. While UniVBench provides basic instructions specifying high-level editing intents, these lack explicit reference attribution and detailed descriptions. Our annotators directly observe the source videos, target videos, and reference images to write new dense instructions from scratch, ensuring: (1) explicit reference tokens bind each visual attribute to its corresponding reference image, (2) the preserve-vs-change distinction is clearly articulated, and (3) instruction density and format match our training data. This process generates gold-standard dense instructions for UniVBench, enabling fair evaluation of our model’s instruction generation capabilities.

10 Training Curve of ReBind-Edit↩︎

To validate the training stability and convergence of ReBind-Edit reported in Section 5.4.1, we present the training loss curve on the 50k subset used for ablation experiments. This curve demonstrates that the model reaches stable convergence by 5000 steps, justifying our choice of this checkpoint for fair comparison across different instruction types.

Figure 6 shows the training loss curve of ReBind-Edit when trained on 50k randomly sampled training cases with dense instructions and explicit reference tokens. The loss converges rapidly in early training and reaches a stable plateau after approximately 3000 steps, with minimal fluctuation through 5000 steps. This convergence pattern validates our choice of using 5000-step checkpoints for the ablation experiments of instruction type, ensuring all instruction design variants are evaluated at a well-converged and stable training state for fair comparison.

11 Reward Curves of ReBind-Instruct↩︎

To provide insights into the GRPO training dynamics of ReBind-Instruct described in Section 4.2, we present the reward curves across all training iterations. These curves demonstrate the effectiveness of our reward design, showing consistent improvement in dual checklist rewards, stable referential consistency, appropriate length control, and decreasing load balancing loss.

Figure 7 shows the training dynamics of ReBind-Instruct across four key metrics. The Dual Checklist Reward, which evaluates the balance between preserving unchanged content and accurately modifying specified elements, demonstrates consistent improvement throughout training, indicating effective learning of the preserve-vs-change distinction. The Referential Consistency Reward maintains high performance, validating that the model successfully learns coherent mappings between reference images and target attributes. The Length Reward remains consistently high, confirming that generated instructions meet the desired token length constraints. The Load Balancing Loss shows a gradual decrease, reflecting improved stability in the model’s attention mechanisms during training.

12 User Study↩︎

To complement automated evaluation with human judgment, we conduct a user study with 15 participants on 50 randomly sampled cases from UniVBench. For each case, participants are shown the source video, reference images, editing instruction, and the anonymized outputs from all methods. They are asked to rank the results for each of the four evaluation criteria: Instruction Following, Source Video Preserving, Reference Image Preserving, and Visual Quality. We compare ReBind-Edit against three open-source baselines: UniVideo [44], Kiwi-Edit [58], and OmniWeaving [11]. Preference scores are computed from the rankings, with higher scores indicating better performance.

As shown in Figure 8, ReBind-Edit achieves the highest preference scores across all four evaluation dimensions. The advantage is most pronounced in Reference Image Preserving, validating the effectiveness of our dense instructions with explicit reference relationships for accurate multi-reference coordination. ReBind-Edit significantly outperforms all baselines, demonstrating superior performance in both instruction following and visual quality.

13 More Qualitative Results↩︎

To provide a more comprehensive view of ReBind’s performance across diverse editing scenarios, we present additional qualitative comparisons on both single-reference image-conditioned video editing (RVE) and multi-reference image-conditioned video editing (MRVE) tasks. These examples further demonstrate the effectiveness of our dense instructions with explicit reference relationships in achieving precise visual attribute extraction and accurate multi-reference coordination.

Single-Reference Editing. Figure 9 presents additional comparisons on UniVBench for single-reference editing tasks, where we compare ReBind-Edit against UniVideo, Kiwi-Edit, OmniWeaving, and the closed-source Kling-o3. These examples reveal common challenges in reference-conditioned video editing: UniVideo and Kiwi-Edit sometimes struggle with complete instruction execution, where certain editing operations may not be fully realized in the output; OmniWeaving faces challenges in balancing editing execution with identity consistency across frames; Kling-o3 occasionally exhibits temporal instabilities in object generation during initial frames. In contrast, our method leverages explicit reference attribution in dense instructions to achieve more consistent editing results, with accurate attribute extraction from reference images, improved temporal coherence throughout the video sequence, and faithful execution of editing operations while preserving the original video’s dynamics.

Figure 10 showcases additional results on IntelligentVBench, covering diverse editing operations including local add, background replacement, and local replacement. These examples validate that our approach generalizes well across different editing types, consistently delivering visually coherent results that accurately follow both the editing instructions and reference visual characteristics.

Multi-Reference Editing. Figure 11 provides supplementary multi-reference editing results from UniVBench. These cases involve coordinating visual attributes from multiple reference sources simultaneously, which presents significant challenges for cross-reference attribution. The results demonstrate that ReBind-Edit successfully handles complex multi-reference scenarios by correctly binding each visual attribute to its designated source through explicit reference tokens in dense instructions. Our method avoids the cross-reference confusion that commonly affects baseline approaches, producing edited videos where each modified element accurately reflects its corresponding reference image.

14 MLLM Prompts for Instruction Filtering↩︎

To ensure the quality of generated dense instructions, we apply automated filtering through GPT-5.1 to detect and remove erroneous annotations. The filtering process checks for three categories of errors: reference token consistency, content quality, and format correctness.

15 MLLM Prompts for UniVBench Evaluation↩︎

For evaluating video editing quality on UniVBench, we employ GPT-5.1 as the judge model to assess edited videos across eight dimensions: Subject, Background, Action, Camera, Color, Lighting, Style, and Relative Position. The evaluation balances edit success with temporal consistency, recognizing that successful edit execution is the primary goal while maintaining appropriate consistency when the edit permits.

References↩︎