From Draft to Draft-Free: One-Step Video Object Removal via Privileged Distillation and Fast Planting
July 16, 2026
Abstract
Video object removal is a fundamental yet challenging task in video editing. Despite recent progress, existing methods typically fall into two categories. Traditional approaches based on optical flow or attention mechanisms often introduce noticeable artifacts and yield unnatural results. In contrast, diffusion-based methods improve visual realism but demand multiple denoising steps, limiting their practicality. To address these issues, we propose From-Draft-to-Draft-Free (D2DF), a framework that distills the ability of transforming coarse drafts into refined videos into a one-step video generation model. Within D2DF, a teacher model is trained to refine low-quality removal results (“drafts”) into high-fidelity videos by multiple steps. Then, through Prior-Privileged Consistency Distillation (PPCD), we distill this capability into a student model that performs one-step removal conditioned on the draft. To eliminate draft dependency, we introduce a Self-Guided Fast Planting (SGFP) module based on our Temporal Masked Transformer that autonomously generates scene-consistent pseudo-drafts in latent space, enabling a fully draft-free one-step model. Extensive experiments show that both draft-conditioned and draft-free versions achieve state-of-the-art performance on multiple metrics, surpassing traditional and multi-step generative methods in both quality and efficiency. The denoising process for a single video takes only about 1 second.
1 Introduction↩︎
Video object removal (VOR) aims to remove undesired objects from videos and reconstruct the background in the occluded regions. As an essential task in video inpainting[1]–[3], it has a wide range of applications. The challenge is intrinsically difficult, often requiring globally consistent spatial–temporal content generation[4]–[6] for regions that cannot be propagated from adjacent frames.
Traditional VOR methods rely on optical-flow propagation or shallow transformer architectures [7]–[12]. While they propagate background textures, their limited generative capacity yields structural distortions and blurry reconstructions, especially for large occlusions (Fig. 1 (b)). This “draft" output contains noticeable flaws.
The emergence of large video diffusion models [13]–[18] offers a path to solving this “generative capacity" bottleneck, significantly enhancing realism. However, two critical problems remain. First, Hallucination: weak priors cause diffusion models to”hallucinate"—generating plausible but semantically incorrect content (Fig. 1 (c)(d)), failing deterministic reconstruction. Second, Slowness: diffusion inference is intrinsically slow (tens of DDIM steps [19]), hindering practical deployment.
To solve the hallucination issue, we observe that flow-based methods efficiently propagate reliable pixel information across frames. This process is fast and fills parts of the masked region with valid content. While their “draft" output can be blurry (Fig. 1 (b)), it provides a strong, partially reconstructed starting point. We use this”draft" to guide a diffusion “refiner" to avoid hallucination. To solve the slowness issue, we argue that the full multi-step refinement is unnecessary given this strong”draft" condition. The task is simplified enough that the “refiner" can be distilled into a single, efficient step. Our ultimate goal, however, is an autonomous model that learns this paradigm but discards the external”draft" prior at inference.
Based on this insight, we propose the “From Draft to Draft-Free” framework (D2DF), which progressively derives a draft-guided one-step refiner (D2DF-DG) and a fully draft-free one-step generator (D2DF-DF). Our framework (Fig. 1) has three stages:
Stage I: The Draft-Guided Diffusion Teacher. We first train a multi-step diffusion teacher (D-LDM) conditioned on external “drafts" (e.g., ProPainter [7]). The teacher refines the flawed draft into a high-quality object removal video through 50 steps.
Stage : The Draft-Guided One-Step Refiner (D2DF-DG). Our goal is to distill the 50-step teacher into a 1-step student (D2DF-DG). A naive consistency distillation [20]–[22] would fail for the reason that when conditioned on the flawed draft, the teacher must reconstruct from varying levels of damage. This forces it onto an unstable solution trajectory that significantly deviates from the ideal path to the ground truth (GT). As Fig.2 illustrates, this noisy path provides an inconsistent and Suboptimal Target for the student, making convergence difficult. To solve this, we introduce Prior-Privileged Consistency Distillation (PPCD). We replace the flawed draft with the GT as the condition of the teacher. This “truth-injected" strategy compels the teacher to generate a perfectly consistent and stable trajectory. This ideal path, which stays aligned with the GT, provides a”Golden” Target for the student, dramatically improving distillation effectiveness. Crucially, the student (D2DF-DG) still accepts the draft as input, but now learns a clean, one-step mapping to this “golden" output.
Stage : The Draft-Free One-Step Generator (D2DF-DF). Despite the impressive performance of D2DF-DG, it still relies on external drafts. To eliminate this dependency, we design the Self-Guided Fast Planting (SGFP) module, a lightweight network based on Temporal Masked Transformer (TMT) that constructs a “pseudo-draft" in the latent space. This differs fundamentally from drafts in explicit pixel-space from typical methods like ProPainter[7]. We then use PPCD again to distill D2DF-DG (as teacher) into our final student, D2DF-DF. This model inherits the one-step speed but is now fully autonomous, achieving end-to-end generation without external priors.
Our D2DF-DF achieves superior performance on three benchmarks [23]–[25], exhibiting strong robustness, especially under large-mask scenarios. Crucially, our models set a new standard for efficiency: D2DF-DF achieves state-of-the-art results while being over 40 times faster than competing SOTA diffusion methods like ROSE[16]. This combination of quality, speed, and autonomy represents a significant step forward for practical video editing.
Our contributions can be summarized as follows:
\(\bullet\) We propose a novel From-Draft-to-Draft-Free (D2DF) framework, a progressive methodology to decouple latency and dependency, producing two models: a high-efficiency one-step refiner (D2DF-DG) and a fully autonomous one-step generator (D2DF-DF).
\(\bullet\) In D2DF, we introduce Prior-Privileged Consistency Distillation (PPCD), a novel “truth-injected" distillation where a teacher uses GT to create a golden target.
\(\bullet\) We design the Self-Guided Fast Planting (SGFP) module, a lightweight latent-space prior generator, combined with PPCD, to create a fully autonomous, end-to-end, one-step model.
2 Related Work↩︎
2.1 Diffusion-based Video Generation↩︎
Diffusion models have recently achieved remarkable success in visual generation by iteratively denoising noisy inputs [26]–[32]. Following advances in text-to-image diffusion, recent works extend this paradigm to video generation. Text-to-video (T2V) models [14], [33]–[37] integrate temporal modules into 2D UNet architectures, while image-to-video (I2V) methods such as DynamiCrafter [38] and PixelDance [39] demonstrate strong motion and scene modeling ability. More recently, Transformer-based diffusion models (e.g., DiT, SORA-like [15], [40]) further improve temporal reasoning and visual fidelity.
2.2 Video Inpainting↩︎
Most existing video inpainting approaches adopt a two-stage framework. In the first stage, prior information is extracted from the input video, such as motion cues [4], [41], [42], homography transformations[43] or low-resolution reconstructions [44]. In the second stage, based on these priors, the missing regions are filled by a carefully designed generative network. Typical architectures include image inpainting models [10], [42], [45], 3D convolutional networks [41], and Transformer-based frameworks [9]. Recent studies have combined the advantages with the spatio-temporal modeling power of video Transformers [7]–[10], leading to a series of two-stage hybrid methods. Recently, researchers have introduced video diffusion models for video inpainting [6], [16], [17], [46]–[48]. These models substantially improve the realism and coherence of the generated videos, yet they still suffer from hallucination in the reconstructed regions. To address these issues, our work introduces the SGFP module to provide strong priors, together with a Prior-Privileged Consistency Distillation (PPCD) framework that enables one-step video generation, effectively balancing quality and efficiency.
2.3 Consistency Model↩︎
Diffusion models achieve impressive generative performance but suffer from slow inference due to their multi-step sampling process. To accelerate generation, several strategies have been proposed, such as improved ODE solvers [49], neural operators [50], and model distillation [51]–[53]. Among them, consistency distillation [20] has emerged as an effective solution. Instead of relying on iterative denoising, it trains a student consistency model to directly predict clean samples from intermediate noisy states, enabling one-step generation while preserving sample quality. Subsequent works, such as Latent Consistency Models (LCM) [21], [54], extend this paradigm into latent space for higher efficiency and lower memory cost. Recent improvements [22], [55], [56] further enhance stability and fidelity, making consistency distillation a promising framework for efficient generative modeling. However, consistency models still face challenges in adapting to task-specific applications, especially for video generation. To address this, we propose Prior-Privileged Consistency Distillation (PPCD), which provides more stable and consistent solution trajectories for video generation tasks.
3 Method↩︎
3.1 Preliminaries↩︎
Latent Diffusion Models (LDM)[13] establish the generative modeling process in the latent space. The training objective matches the predicted denoised sample \(\mathbf{x}_0\) with the ground truth through a simplified variational bound: \[\mathcal{L}_{LDM} = \mathbb{E}_{x, t, \epsilon \sim \mathcal{N}(0,\mathbf{I})} \left[\|\boldsymbol{\epsilon} - \boldsymbol{\epsilon}_\theta(\mathbf{x}_t, t)\|_2^2\right], \label{eq:ddpm95loss}\tag{1}\] where \(\mathbf{x}_t\) is the noisy input obtained through the forward diffusion process with timestep \(t\), and \(\boldsymbol{\epsilon}_\theta\) is a neural network that predicts the noise component. This objective effectively establishes a progressive denoising mapping that transforms pure noise \(\mathbf{x}_T\) into clean data \(\mathbf{x}_0\) through a Markov chain of \(T\) steps. This mapping can be formulated as a parameterized reverse process \(\mathbf{p}_\theta:(\mathbf{x}_t,t) \longmapsto \mathbf{x}_{t-1}\). However, this requires an iterative denoising process.
Consistency Model [20] attempts to establish mappings through a consistency function, defined as \(\mathbf{f}:(\mathbf{x}_t,t) \longmapsto \mathbf{x}_\delta\), where \(\delta\) is a fixed small positive number. Its consistency is reflected in one of the key properties of this function: \[\mathbf{f}(\mathbf{x}_t,t)=\mathbf{f}(\mathbf{x}_{t'},t'),\quad \forall t,t'\in[\epsilon,T]. \label{eq:self95consistency}\tag{2}\] Ideally, the consistency function established through learnable neural networks \(f_\theta\) enables the consistency model to recover the original data \(\mathbf{x}_0\) from the initial distribution \(\mathbf{x}_T \sim \mathcal{N}(\mathbf{0}, T^2 \boldsymbol{I})\) in one step of denoising. Learnable parameters \(\theta\) are trained via Consistency Distillation [20], [22], [56]. The distillation goal is to minimize the output discrepancy between random adjacent data points, thereby ensuring self-consistency in the sense of probability. Formally, the consistency loss is defined as follows:
\[\mathcal{L}(\mathbf{\theta}, \mathbf{\theta}^-;\Phi)=\mathbb{E}_{\boldsymbol{x},t} \left[d\left(\mathbf{f}_\mathbf{\theta}(\mathbf{x}_{t_{n+1}},t_{n+1}),\mathbf{f}_{\mathbf{\theta}^{-}}(\hat{\mathbf{x}}^\phi_{t_n},t_n)\right)\right], \label{eq:consistency95loss}\tag{3}\]
where \(0 = t_1 < t_2 < \cdots < t_N = T\), and \(n\) is uniformly distributed over the set \(\{1, 2, \cdots, N - 1\}\). \(\Phi(\cdot)\) denotes the one-step ODE [49] solver applied to PF-ODE[57], the model parameter \(\mathbf{\theta^-}\) is obtained from the exponential moving average (EMA) of \(\mathbf{\theta}\), and \(d(\cdot,\cdot)\) is a chosen metric function for measuring the distance between two samples. \(\hat{x}^{\phi}_{t_n}\) is determined using the equation \(\hat{\mathbf{x}}^{\phi}_{t_n} := \Phi(\mathbf{x}_{t_{n+1}}, t_{n+1}, t_n; \phi)\). \(\hat{\mathbf{x}}^{\phi}_{t_n}\) is estimated by a teacher model with an ODE solver in consistency distillation.
3.2 Stage : Draft-Guided Diffusion Teacher↩︎
Our first stage simplifies the task into a refinement process. We introduce a strong conditional prior (draft) from existing methods (e.g., ProPainter [7]). This process establishes a sufficiently powerful teacher network to provide a mapping from draft to refined video in multiple steps. Thus, we train a Draft-Guided Diffusion Teacher model (D-LDM), denoted as \(\phi\). This model is conditioned on both the draft latent \(\mathbf{x}_\text{draft}\) and the masked video latent \(\mathbf{x}_{0}^M\). At timestep \(t\), the model input \(I(\mathbf{x}_t)\) is defined as the concatenation of the noised input \(\mathbf{x}_t\) and the conditional information: \[I(\mathbf{x}_t)= Cat(\mathbf{x}_t,\mathbf{x}_0^{M}+\mathbf{x}_{\text{draft}}),\] where \(\mathbf{x}_t\) denotes the noised input through the forward diffusion process and \(Cat(\cdot)\) denotes the latents concatenation.
The D-LDM model is then trained by substituting this conditional input \(I(\mathbf{x}_t)\) for \(\mathbf{x}_t\) in the LDM training objective (Eq.@eq:eq:ddpm95loss ). The objective for the teacher network \(\boldsymbol{\epsilon}_\phi\) becomes: \[\mathcal{L}_{D-LDM} = \mathbb{E}_{x, t, \epsilon \sim \mathcal{N}(0,\mathbf{I})} \left[\|\boldsymbol{\epsilon} - \boldsymbol{\epsilon}_\phi(I(\mathbf{x}_t), t)\|_2^2\right].\] This strong draft conditioning significantly reduces the mapping complexity, making a one-step approximation feasible. The resulting D-LDM \(\phi\) achieves strong generative performance via iterative (e.g., 50-step) DDIM sampling.
3.3 Stage : Prior-Privileged Consistency Distillation for Draft-Guided One-Step Refiner↩︎
In the second stage, we distill the D-LDM teacher model (\(\phi\)) into a one-step, draft-guided student model, D2DF-DG. While the theory of consistency distillation (CD) is promising, it suffers from high sampling variance from the teacher model, especially when conditioned on imperfect drafts, which destabilizes the distillation process.
To overcome this, we introduce Prior-Privileged Consistency Distillation (PPCD). Standard distillation from a teacher, conditioned on flawed drafts, leads to unstable and suboptimal trajectories. Our key idea is to inject the ground-truth video as a “privileged" prior into the teacher model \(\phi\) during distillation only. This provides an ideal,”golden" target trajectory and ensures the mapping is anchored in a convergent solve space.
Formally, the teacher model serves as a trajectory provider. To ensure it provides a stable and reliable consistency trajectory, we replace the draft latent \(\mathbf{x}_{\text{draft}}\) with the ground-truth latent \(\mathbf{x}_0\) as shown in Fig.3 (PPCD). The teacher’s privileged input at timestep \(t\) becomes: \[\mathbf{I}^{pp}(\mathbf{x}_t) = Cat(\mathbf{x}_t, \mathbf{x}_0^{M} + \mathbf{x}_0).\] Given this privileged input, the teacher network produces a denoised latent \(\hat{\mathbf{x}}^{\phi}_{t_n} = \Phi(I^{pp}(\mathbf{x}_{t_{n+1}}), t_{n+1}, t_n; \phi)\) at the next timestep along the diffusion trajectory.
Based on the principle of consistency distillation (Eq.@eq:eq:consistency95loss ), the student network \(\mathbf{f}_{\mathbf{\theta}}\) (and its EMA target \(\mathbf{f}_{\mathbf{\theta}^{-}}\)) is trained to match the consistency property. However, the student still operates using the draft as input, as it will at inference time. We obtain the target latent and prediction latent: \[\begin{align} \hat{\mathbf{x}}^{\phi}_{0} & = \mathbf{f}_{\mathbf{\theta}^{-}}(Cat(\hat{\mathbf{x}}^\phi_{t_n},\mathbf{x}_0^M + \mathbf{x}_{\text{draft}}),t_n), \\ \mathbf{x}^{pred}_{0} & = \mathbf{f}_{\mathbf{\theta}}(Cat(\hat{\mathbf{x}}^\phi_{t_{n+1}},\mathbf{x}_0^M + \mathbf{x}_{\text{draft}}),t_{n+1}), \end{align}\] where \(\mathbf{f}_{\mathbf{\theta}^{-}}\) denotes the target model and \(\theta^-\) is an EMA version of the student network \(\theta\). The meanings of \(t_n\) and \(t_{n+1}\) are the same as in Eq.@eq:eq:consistency95loss . Ultimately, our PPCD utilizes the following objectives to optimize the student network \(\theta\):
\[\mathcal{L}_{\text{ppcd}} = \mathbb{E}_{\boldsymbol{x},t} \left[d\left(\mathbf{x}^{pred}_{0},\hat{\mathbf{x}}^{\phi}_{0}\right)\right] + \alpha \mathbb{E}_{\boldsymbol{x},t} \left[d\left(\mathbf{x}^{pred}_{0},\mathbf{x}_{0}\right)\right],\]
where \(d(\cdot,\cdot)\) is the mean square error of two samples and \(\alpha\) is a hyperparameter. The first term enforces consistency with the privileged trajectory, while the second term adds a direct ground-truth constraint.
Using the trained D-LDM as the teacher, we apply PPCD to obtain D2DF-DG, a one-step, draft-conditioned object removal refiner model.
3.4 Stage : Self-Guided Fast Planting (SGFP) for Draft-Free One-Step Generator↩︎
The D2DF-DG model from Stage is fast but still depends on drafts generated by external methods, incurring inference overhead. In our final stage, we eliminate this dependency by proposing the Self-Guided Fast Planting (SGFP) module, resulting in the final draft-free model, D2DF-DF.
The SGFP module (Fig.4) learns to reconstruct a “pseudo-draft" directly in latent space using spatio-temporal information from unoccluded regions. The module is built upon our Temporal Masked Transformer (TMT) architecture, which aims to rapidly reconstruct masked regions. Given a masked draft latent \(\mathbf{x}_{0}^M \in \mathbb{R}^{C \times T \times H \times W}\) and a binary mask \(\mathbf{M} \in \{0, 1\}^{1 \times T \times H \times W}\) (where 0 denotes masked regions and 1 denotes known regions), the objective of SGFP is to get a completed latent tensor \(\mathbf{x}_{recon}\).
To handle the high-dimensional video latents while maintaining computational tractability, we first partition the spatial dimensions \((H, W)\) into a series of non-overlapping patches of size \(P \times P\). This reshapes the input into a sequence of \(N = T \times (H/P) \times (W/P)\) tokens, where each token is a vector of dimension \(C \cdot P^2\). Each patch vector \(\mathbf{p}_i\) is then embedded into a lower-dimensional space \(\mathbb{R}^{D_e}\) using a linear layer, combined with a positional embedding: \(\mathbf{z}_i = \text{Linear}(\mathbf{p}_i) + \mathbf{e}_{pos}(i) ,\) where \(D_e\) is the embedding dimension and \(\mathbf{z}_i\) is the input to the TMT layers. The term \(\mathbf{e}_{pos}(i)\) is a learnable positional encoding generated by a small MLP from the patch’s normalized 3D coordinates \((t, h, w)\), providing spatiotemporal awareness.
The key to the “self-guided” nature of SGFP lies in the construction of its self-attention mechanism. The computation must rely solely on known, unmasked information. We first compute a mask ratio \(r_i = \text{Mean}(\mathbf{M}_{patch, i})\) for each patch. If \(r_i\) exceeds a threshold \(\tau\), the patch is considered “invalid”. In the self-attention calculation, all patches \(\mathbf{z}_i\) can act as Queries, but only “valid” patches (i.e., \(r_j \le \tau\)) are permitted to serve as Keys and Values. This is enforced via an attention mask \(\mathbf{A}_{mask}\): \[\mathbf{A}_{mask}(i, j) = \begin{cases} 0 & \text{if } r_j \le \tau \text{ and } \text{dist}(i, j) \le R \\ -\infty & \text{otherwise} \end{cases},\] where \(\text{dist}(i, j) \le R\) is an optional spatial window constraint that further restricts attention to a \((2R+1) \times (2R+1)\) spatial neighborhood of the query patch \(i\) (across all time frames \(T\)), reducing complexity and reinforcing local priors. After \(L\) Transformer layers, the output sequence \(\mathbf{z}_i^{(L)}\) is projected back to the original patch dimension \(\mathbb{R}^{C \cdot P^2}\) and “un-folded” to reconstruct the latent variable \(\mathbf{x}_{recon} \in \mathbb{R}^{C \times T \times H \times W}\). Finally, the pseudo-draft \(\mathbf{x}_{\text{draft}}^{p}\) is generated by composing the reconstruction and the original input using the mask \(\mathbf{M}\), ensuring the fidelity of unmasked regions: \[\mathbf{x}_{\text{draft}}^p = \mathbf{x}_{recon} \odot (1 - \mathbf{M}) + \mathbf{x}_{0}^M \odot \mathbf{M} .\]
We then treat the D2DF-DG model (from Stage ) as the new teacher and employ the PPCD framework again to distill the final draft-free student, D2DF-DF, which now incorporates the SGFP module. We trained the SGFP together with the subsequent generation module, incorporating constraints for pseudo-drafts: \[\mathcal{L}_{\text{draft}} = \mathbb{E}_{\boldsymbol{x},t} \left[d\left(\mathbf{x}^{p}_{draft},\mathbf{x}_{0}\right)\right].\] By introducing SGFP, we obtain D2DF-DF, an end-to-end, draft-free video object removal model. The entire progressive pipeline is summarized in Figure 3.
4 Experiments↩︎
4.1 Experiment Setup↩︎
Implementation Details. We employ CogVideoX-5B-I2V[14] (480\(\times\)720 resolution) as the pre-trained weights for training the first-stage D-LDM network. The D-LDM model is trained with a learning rate of \(5\times10^{-5}\) and uses 50 DDIM steps for inference. The “draft" is obtained using the ProPainter[7]. In the PPCD distillation, the hyperparameter \(\alpha\) is set to 1, with a batch size of 4 and a learning rate of \(2\times10^{-5}\). Sampling timesteps \(t_1,t_2,...,t_N\) are performed uniformly at 50-step intervals over 1k timesteps. The EMA decay rate of the target network is set to 0.99. For the SGFP module, the number of layers \(L\) was set to 4, the spatial radius \(R\) to 2, the patch size \(P\) to 10, the embedding dimension \(D_e\) to 512, and the mask threshold \(\tau\) to 0.7. In the third-stage training, the ratio of \(\mathcal{L}_{\text{draft}}\) to \(\mathcal{L}_{\text{ppcd}}\) is 1:5, and the learning rate for the SGFP module parameters is \(1\times 10^{-4}\). We train only the DiT component with full-parameter SFT.
Datasets. DAVIS[58] and YouTube-VOS[59] are widely used for video inpainting with stationary random masks, but they do not provide ground-truth videos after object removal. Therefore, they are less suitable for VOR evaluation unless synthetic or subjective protocols are introduced. Based on this, we evaluate our model on three datasets: RORD [23], ROVI [24], and VPLM [25]. RORD is a large-scale real-world dataset specifically designed for object removal, containing 2,761 distinct scenes for training and 343 scenes for testing. ROVI includes 5,650 videos with a total of 9,091 object removal instances, among which 458 videos (758 objects) are reserved for testing. From the training splits of RORD and ROVI, we extract 25-frame video segments to form our training set, resulting in approximately 30k samples. For evaluation, we adopt the VPLM dataset, which provides 60 video-based object removal tasks. All three datasets provide triplets of original video, object mask, and ground-truth removed video.
Metrics. To quantitatively evaluate the performance of video object removal, we employ four established metrics: PSNR [60], SSIM [61], VFID [62], and LPIPS [63]. PSNR and SSIM are widely used for measuring low-level pixel-wise similarity between the generated video and the ground truth. To further assess perceptual quality, we use LPIPS, which leverages deep features to better align with human perception. We use VFID, an adaptation of FID for videos, to evaluate the overall temporal consistency and visual realism of the completed video sequences. Additionally, we also introduced flow warping error[64] as a metric for measuring temporal consistency in the zero-shot validation. We strictly follow the settings of ProPainter [7] for calculating metrics.
| Models | RORD | ROVI | VPLM | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2-5 (lr)6-9 (l)10-13 | PSNR \(\uparrow\) | SSIM \(\uparrow\) | VFID \(\downarrow\) | LPIPS \(\downarrow\) | PSNR \(\uparrow\) | SSIM \(\uparrow\) | VFID \(\downarrow\) | LPIPS \(\downarrow\) | PSNR \(\uparrow\) | SSIM \(\uparrow\) | VFID \(\downarrow\) | LPIPS \(\downarrow\) |
| FLoED [18] | 26.06 | 0.8263 | 0.471 | 0.1557 | 34.42 | 0.9225 | 0.138 | 0.0763 | 34.85 | 0.9149 | 0.493 | 0.0816 |
| E\(^2\)FGVI [8] | 26.95 | 0.8421 | 0.311 | 0.1340 | 38.01 | 0.9693 | 0.096 | 0.0337 | 38.29 | 0.9652 | 0.424 | 0.0390 |
| ROSE [16] | 27.94 | 0.8959 | 0.270 | 0.1105 | 35.72 | 0.9642 | 0.099 | 0.0379 | 36.19 | 0.9520 | 0.378 | 0.0473 |
| FuseFormer [11] | 27.99 | 0.8659 | 0.306 | 0.1431 | 41.21 | 0.9813 | 0.064 | 0.0292 | 41.24 | 0.9814 | 0.236 | 0.0364 |
| DiffuEraser [17] | 28.64 | 0.8865 | 0.268 | 0.1205 | 38.79 | 0.9769 | 0.093 | 0.0323 | 38.76 | 0.9719 | 0.298 | 0.0421 |
| FGT [9] | 30.13 | 0.8972 | 0.271 | 0.1042 | 37.80 | 0.9725 | 0.089 | 0.0325 | 38.51 | 0.9725 | 0.350 | 0.0368 |
| ProPainter [7] | 30.97 | 0.9132 | 0.230 | 0.1020 | 40.86 | 0.9759 | 0.069 | 0.0263 | 41.15 | 0.9808 | 0.241 | 0.0374 |
| MiniMax-Remover [65] | 30.60 | 0.9166 | 0.224 | 0.0947 | 38.65 | 0.9775 | 0.069 | 0.0318 | 39.47 | 0.9744 | 0.246 | 0.0381 |
| D2DF-DG(Ours) | 32.20 | 0.9318 | 0.251 | 0.0902 | 42.41 | 0.9888 | 0.062 | 0.0227 | 43.31 | 0.9877 | 0.191 | 0.0337 |
| D2DF-DF(Ours) | 31.56 | 0.9202 | 0.268 | 0.1001 | 42.25 | 0.9889 | 0.063 | 0.0282 | 43.05 | 0.9864 | 0.196 | 0.0348 |
2.5pt
(a) PSNR
Figure 5: 
.
Figure 6: 
.
4.2 Comparisons↩︎
Quantitative Evaluation. We report quantitative results on three datasets. We compare the proposed D2DF-DG and D2DF-DF with previous video object removal methods, including FuseFormer [11], FGT [9], E\(^2\)FGVI [8], FLoED [18], DiffuEraser[17], ROSE [16] and ProPainter [7]. Among them, FLoED, DiffuEraser, and ROSE are methods based on diffusion models. As shown in Table.[tbl:tab:compare], D2DF-DG achieves the best performance on most metrics and consistently delivers strong results across all three benchmarks. Although D2DF-DF exhibits slightly lower performance compared to D2DF-DG, it still leads state-of-the-art methods in PSNR. This fully demonstrates the powerful capability of our approach in object removal. Additionally, we observe a significant number of large objects in the RORD dataset. Therefore, we select the best models from diffusion-based (DiffuEraser) and traditional approaches (Propainter) and compare their performance at different mask ratios. As shown in Fig.[fig:compare95psnr95ssim], our models demonstrate only a marginal advantage over Propainter and DiffuEraser within the 0-20% mask ratio range. However, as the mask ratio increases, our models show a more pronounced advantage in both PSNR and SSIM metrics. This highlights the robustness of our model and its superiority in removing large-area objects.
Qualitative Evaluation. For qualitative evaluation, we select diffusion-based methods ROSE [16] and DiffuEraser [17] alongside the lightweight transformer-based ProPainter [7] for comparison. As shown in Fig.7, our method D2DF-DF demonstrates strong capabilities for reconstructing background regions. When removing objects from larger areas in the last two rows, D2DF-DF still delivers highly stable performance. However, other methods exhibit issues such as inconsistent texture information, blurring, and artifacts. This highlights the powerful performance of our one-step denoising model in video object removal. The D2DF-DG delivers even more powerful results. Due to space limitations, you can refer to the appendix for details.
Efficiency Comparison with Diffusion-based Methods. Tab.6 compares the inference times of different diffusion-based methods. We test each model on a NVIDIA A100 GPU to generate 25 frames of 480\(\times\)720 video, measuring the time required for both prior extraction and denoising. Results indicate that DiffuEraser [17], also utilizing ProPainter, achieves performance extremely close to our D2DF-DG. However, when switching to our SGFP module for prior extraction, we observed that this module completes the task in just 0.02 seconds. Consequently, D2DF-DF delivers the fastest inference time while maintaining superior performance. Including the time for encoding and decoding latents, our complete processing time is within 10 seconds.
Qualitative Analysis Under Extreme Occlusion Conditions. In this section, we will discuss how different models perform under extreme occlusion conditions. When objects in a video have a limited range of motion or occupy a large area, the background regions occluded by these objects cannot be obtained through frame propagation. We define this scenario as extreme occlusion. RORD [23] employs a polygonal mask annotation method, resulting in a dataset with an exceptionally high number of extreme occlusions, making it highly challenging. We selected the state-of-the-art methods ProPainter[7] and MiniMax-Remover[65], based on optical flow and diffusion models for comparison. As shown in Fig.8, for regions where information cannot propagate between frames, ProPainter directly produces blurred textures, while MiniMax-Remover (6 steps) exhibits noise artifacts. In contrast, our D2DF-DF model achieves excellent background reproduction with just one step. This demonstrates a unique advantage of our model under extreme occlusion conditions.
| Method | Draft | Flow | PSNR\(\uparrow\) | SSIM\(\uparrow\) | LPIPS\(\downarrow\) | \(E_{\mathrm{warp}}\downarrow(\times10^{-4})\) |
|---|---|---|---|---|---|---|
| FuseFormer [11] | – | 25.05 | 0.9155 | 0.2020 | 0.70 | |
| ProPainter [7] | – | 25.42 | 0.9203 | 0.1268 | 1.10 | |
| ROSE [16] | – | 26.45 | 0.9286 | 0.0860 | 0.55 | |
| MiniMax-Remover [65] | – | 26.63 | 0.9324 | 0.0854 | 0.37 | |
| D2DF-DG(Ours) | FuseFormer | 27.08 | 0.9421 | 0.0831 | 0.41 | |
| D2DF-DG(Ours) | ProPainter | 26.96 | 0.9429 | 0.0884 | 0.93 | |
| D2DF-DF(Ours) | – | 26.57 | 0.9365 | 0.0896 | 0.27 |
4.3 Robustness and Generalization Analysis↩︎
Zero-shot Evaluation on Camera-Bench. To further evaluate the generalization ability of our method, we conduct zero-shot experiments on Camera-Bench[66], as shown in Tab. 2. Without training on this benchmark, both D2DF-DG and D2DF-DF remain highly competitive against existing methods. It is worth noting that our framework is not inherently tied to optical flow. Whether optical flow is involved depends on the draft source used by D2DF-DG. Although D2DF-DG is trained with ProPainter drafts, it can still effectively refine FuseFormer drafts, which are generated without optical flow. This suggests that the cross-domain robustness of our method is not limited to dataset transfer, but also extends to different types of draft priors. Furthermore, D2DF-DF removes the external draft dependency entirely, making the final model fully free from optical-flow-related concerns during inference.
Draft-source Robustness. We further evaluate D2DF-DG with drafts from different sources. As shown in Fig. 9, although our model is trained with ProPainter drafts, it can consistently refine drafts generated by DiffuEraser, ProPainter, and FuseFormer, producing similarly high-quality removal results. This demonstrates that D2DF-DG does not overfit to a specific draft distribution, but learns a robust refinement ability across heterogeneous draft priors. Notably, the effectiveness on FuseFormer drafts also indicates that our framework can work with flow-free draft sources, while D2DF-DF further removes the draft dependency entirely.
4.4 Ablation Study↩︎
Study of Prior-Privileged Consistency Distillation. In Tab.¿tbl:tab:ablation95ppcd?, when we train directly using the LDM approach, the model behaves as a standard LDM model, yielding suboptimal results in a single step. Consequently, employing Consistency Distillation (CD) significantly improves performance. However, introducing privileged prior knowledge (replacing the draft conditions with ground truth) further enhances results. This indicates that our ground-truth prior indeed yields a more stable consistency trajectory.
0.70 @ccccccc@ &
(lr)1-3 (lr)4-7 D.T. & CD & PPCD & PSNR\(\uparrow\) & SSIM\(\uparrow\) & VFID\(\downarrow\) & LPIPS\(\downarrow\)
✔ & & & 31.54 & 0.9192 & 0.271 & 0.1014
& ✔ & & 31.85 & 0.9296 & 0.257 & 0.1002
& & ✔ & 32.20 & 0.9318 & 0.251 & 0.0902
| Components | Metrics | |||||
|---|---|---|---|---|---|---|
| 1-4 (lr)5-7 TL | VP | SW | MR | PSNR\(\uparrow\) | SSIM\(\uparrow\) | LPIPS\(\downarrow\) |
| 29.36 | 0.8920 | 0.1426 | ||||
| ✔ | 30.73 | 0.9171 | 0.1149 | |||
| ✔ | ✔ | 30.78 | 0.9228 | 0.1093 | ||
| ✔ | ✔ | ✔ | 31.47 | 0.9197 | 0.1070 | |
| ✔ | ✔ | ✔ | ✔ | 31.56 | 0.9202 | 0.1001 |
Study of Self-Guided Fast Planting Module. To investigate the effectiveness of our proposed SGFP module, we conduct detailed ablation experiments on the RORD dataset. As shown in Tab.3, the first row represents the baseline model, indicating the performance without any prior knowledge. Adding the transformer layer (TL) yields an initial improvement in model performance. Excluding masked regions (VP) results in a slight further enhancement. Introducing the spatial window (SW) significantly boosts model effectiveness. Finally, reconstructing only the masked region (MR) achieves optimal model performance. These ablation experiments demonstrate the rationality and effectiveness of each design step. To validate that the latents generated by SGFP indeed contain meaningful background information, we additionally trained a diagnostic decoder. This decoder is solely used for visualization. As shown in Fig.10 (c), our pseudo-drafts exhibit blocky filling because they are learned from many patch latents. Despite its visual coarseness compared to the draft in Fig.10 (b), this decoding result demonstrates that SGFP successfully captures key background color tones and structures, which suffice as effective prior information.
Study of Three-stage Distillation Framework. To validate the effectiveness of the three-stage framework, we conducted experiments as shown in Table.4. As the table indicates, the performance achieved when the teacher network is LDM was not as good as that obtained with D2DF-DG as the teacher network. This shows the trained D2DF-DG model can provide more stable and consistent solution trajectories than the diffusion D-LDM. Furthermore, the comparison between CD and PPCD also demonstrated the superiority of the PPCD method.
| Training Settings | Metrics | |||||
|---|---|---|---|---|---|---|
| 1-4 (lr)5-7 Student | Teacher | CD | PPCD | PSNR\(\uparrow\) | SSIM\(\uparrow\) | LPIPS\(\downarrow\) |
| D2DF-DF | D-LDM | ✔ | 31.3268 | 0.9179 | 0.1025 | |
| D-LDM | ✔ | 31.4484 | 0.9197 | 0.1014 | ||
| D2DF-DG | ✔ | 31.3257 | 0.9173 | 0.1036 | ||
| D2DF-DG | ✔ | 31.5646 | 0.9202 | 0.1001 | ||
4.5 Analysis of Failure Cases↩︎
In this section, we will discuss cases where the model fails to remove objects and analyze the reasons for these failures. We observe that when multiple objects exist in the video and these objects exhibit dynamic and complex occlusion relationships, the model is unable to fully reconstruct the occluded objects. As shown in Fig.[fig:failurecase], we compared the state-of-the-art optical flow-based method ProPainter[7] and the state-of-the-art diffusion model-based method MiniMax-Remover[65]. The results indicate that neither method can fully restore the occluded “dolphin” in this example. Our two models, D2DF-DG and D2DF-DF, also produce imperfect results under one-step inference settings. However, they retain significantly more context compared to MiniMax-Remover. This also demonstrates the superiority of our approach.
5 Conclusion↩︎
In this paper, we present D2DF, a unified framework that progressively evolves from a draft-guided diffusion teacher to a fully draft-free one-step video object removal model. By introducing Prior-Privileged Consistency Distillation (PPCD), we effectively stabilize the distillation process and enable high-quality one-step generation. Furthermore, the proposed Self-Guided Fast Planting (SGFP) module eliminates the dependency on external draft inputs by generating scene-consistent pseudo-drafts directly in latent space. Our method not only excels in visual quality and temporal coherence but also achieves remarkable efficiency. However, the blurring that occurs in certain scenarios with the one-step denoising is an enhancement needed in future work.
Acknowledgements.↩︎
Zizhao Chen and Ping Wei acknowledge the support of the National Natural Science Foundation of China (No. U23B2060, No. 62495092). Mengmeng Wang acknowledges the support of the National Natural Science Foundation of China (No. 62403429). We also acknowledge the computational resources provided by SGIT AI Lab, State Grid Corporation of China.