1 Introduction↩︎

Video generation models [1]–[6] are becoming increasingly powerful in generating realistic videos [7]; however, their slow inference speed and high compute requirements (3 to 23 minutes for \({\sim}\)​5-second videos on a single A100 GPU) pose an impediment to their wide adoption [8]. As these models scale to generate even higher resolutions, the challenges associated with slow inference speed and high computation requirements will be further exacerbated. The primary reason for these slow inference speeds and high computation needs is the core inference loop of these video generation models [1]–[6]: a computationally expensive diffusion backbone that runs a sequential denoising procedure, dominating generation time.

To alleviate this bottleneck, several approaches have been proposed [9]–[11]. These generation-accelerating approaches can be classified into two categories: training-based [10], [12]–[16] and training-free [9], [11], [17]–[19] acceleration strategies. Due to the significant overhead [11], [20], [21] associated with training-based approaches, which generally entail substantial compute, data, and cost, training-free acceleration methods have become particularly appealing. Among training-free methods, caching methods, due to their lightweight computation requirements, are being rapidly deployed. Caching methods leverage similarity between intermediate states and reuse them to reduce computational redundancy. Caching strategies have rapidly evolved from early uniform caching strategies that used fixed intervals to sophisticated methods [11], [20]–[23] which determine their caching strategy based on the similarity predicted at each step, thus adapting caching to the behavior of the diffusion process.

All of the dynamic caching strategies—such as TeaCache [11], EasyCache [21], and DiCache [20]—share a common structure: they use an accumulated drift metric that estimates the change in model output. If this accumulated drift is below a user-provided threshold, they skip the expensive model run and reuse the previous output with some correction. If the accumulated drift exceeds the threshold, they run the full model and reset the accumulated drift to zero.

Choosing this threshold presents a seemingly inherent tradeoff between output quality and inference latency. A high threshold caches more aggressively, yielding faster generation but lower quality. A low threshold caches conservatively, preserving quality but sacrificing speed.

This raises a fundamental question: is this tradeoff between output quality and inference latency actually fundamental? In this work, we show that it is not. It is instead an artifact of using a fixed threshold throughout the sequential denoising procedure. When the threshold is chosen adaptively, the quality-latency tradeoff can be improved relative to any fixed-threshold dynamic caching method. Figure 1 shows that a dynamic threshold method can push the Pareto frontier beyond that of any fixed-threshold method.

We attribute the power of adaptive threshold schemes to the existence of critical steps in the sequential denoising procedure. By avoiding aggressive caching during these critical steps, our method mitigates the quality loss that fixed-threshold dynamic caching methods typically exhibit. This means one can design an adaptive scheme that identifies critical steps and adjusts the threshold accordingly. Based on these findings, we propose ACID, a simple but powerful adaptive threshold caching mechanism that generalizes across different dynamic caching methods and models, while imposing low computational overhead. ACID examines the rate of change of the respective method’s metric to detect critical steps and adjust thresholds dynamically, improving performance without sacrificing output quality.

Our experiments show that ACID achieves reduced inference latency while maintaining output quality on video generation tasks. In particular, on TeaCache and HunyuanVideo, ACID achieves up to \(2.16\times\) speedup over the no-caching baseline, and up to \(38\%\) additional speedup over the conservative fixed-threshold baseline with negligible (\(<0.3\) dB PSNR, \(<0.01\) SSIM, \(<0.01\) LPIPS) quality degradation.

We summarize our contributions below:

• We identify the existence of critical steps in the sequential denoising procedure and show that video generation caching methods must account for them.

• We design ACID, an adaptive threshold dynamic caching mechanism that switches between low and high thresholds based on the second derivative of each method’s metric. This signals critical steps in the denoising process, enabling significant performance improvements without sacrificing output quality.

• We provide empirical evaluation across several dynamic caching methods (TeaCache, EasyCache, DiCache) and several diffusion models (HunyuanVideo, Wan 2.1, CogVideoX).

2 Related Work↩︎

2.0.0.1 Diffusion Models.

While U-Net-based diffusion models like Stable Diffusion [24] and SDXL [25] successfully catalyzed high-quality image and video generation [26]–[28], their limited architectural capacity bottlenecks the training and deployment of massive models. To eliminate this constraint, recent frameworks have rapidly shifted toward the Diffusion Transformer (DiT) backbone, exploiting its structural flexibility to support next-generation foundation models [29]–[32]. However, despite their theoretical promise, scaling these DiT backbones introduces substantial computational overhead.

2.0.0.2 Diffusion Model Acceleration.

As DiT backbones scale in capacity and complexity, their runtime increases proportionately, exacerbating the notoriously slow inference speeds of diffusion models and impeding real-world deployment. To alleviate this latency bottleneck, several acceleration paradigms have emerged. Recent work has extensively explored efficient attention [10], [33], sparse attention [18], [19], distillation [12]–[14], quantization [10], and improved SDE/ODE solvers [9]. Alongside these methods, caching-based feature reuse strategies offer a particularly compelling, orthogonal path forward, which we detail next.

2.0.0.3 Caching-based feature reuse strategies.

Caching methods have gained attention recently due to their lightweight nature. DeepCache [34] and Faster Diffusion [35] reuse U-Net features across timesteps to reduce computational redundancy. FORA [36] and \(\Delta\)-DiT [37] extend this idea to transformer-based architectures. PAB [38] broadcasts attention features to later timesteps in a pyramid-style method based on various block characteristics. AdaCache [22] modifies residual reuse methods based on content complexity. FasterCache [39] proposes to cache for both the conditional and unconditional branch of Classifier Free Guidance [40]. TeaCache [11], EasyCache [21], and DiCache [20] represent the most recent state-of-the-art dynamic caching approaches; we examine their shared structure and its key limitation in the paragraphs below.

2.0.0.4 The Fixed Threshold of Dynamic Feature Caching Methods.

Many recent dynamic caching-based feature reuse strategies make caching decisions based on the diffusion process’s state at runtime. Methods like TeaCache, EasyCache, and DiCache share a common algorithmic skeleton: a running accumulated drift variable is initialized to zero, an estimate of output change is added at each step, and this accumulated value is compared against a fixed threshold \(\tau\). When the accumulated drift is below \(\tau\), the expensive model forward pass is skipped and the previous output is reused with some correction. When the accumulated drift exceeds \(\tau\), the full model is run and the accumulator is reset. The methods differ in how they estimate drift and how they reconstruct the output on skipped steps, which we delve into below.

2.0.0.5 TeaCache.

TeaCache estimates drift using the L1 distance between consecutive values of the Timestep Embedding Modulated Noisy Input (TEMNI), a combination of the text embedding, timestep embedding, and current noisy input. Rather than measuring how much the model output changed, which would require running the model, TeaCache measures how much the model’s inputs changed, exploiting the empirical observation that large input changes correlate with large output changes. On skipped steps, TeaCache reuses the previous output with a stored residual correction.

2.0.0.6 EasyCache.

EasyCache estimates drift using the model’s transformation rate \(k\), defined as the ratio of output change to input change at the most recently computed step. The authors observe that \(k\) is unstable early in the denoising process but quickly stabilizes, reflecting the model settling into a roughly linear input-output regime. Under this regime, EasyCache estimates the current step’s output change as \(k \times \Delta\text{input} / \|v_{t-1}\|\), where \(\Delta\text{input}\) is the current input change, \(\|v_{t-1}\|\) is the magnitude of the previous output, and \(k\) is the transformation rate of the last fully computed step. This normalized estimate is accumulated against \(\tau\). On skipped steps, EasyCache reconstructs the output as the current input plus the offset \(\Delta = \text{output} - \text{input}\) from the last fully computed step, which is justified because a stable \(k\) implies the model’s additive contribution is approximately constant.

2.0.0.7 DiCache.

DiCache replaces the input-level proxy used by TeaCache and EasyCache with a feature-level proxy. At each step, DiCache runs only the first few transformer blocks of the model as a shallow probe and measures how much those features changed. Because these early layers are cheap to compute, this provides a direct estimate of whether the full model’s output would change significantly, without paying the cost of a full forward pass. On skipped steps, DiCache uses the trajectory of shallow probe features across multiple recent steps to interpolate a better estimate of the current deep output.

Despite these differences, all three methods share a critical limitation: the fixed user-provided threshold is held constant throughout the entire denoising process. This means the caching schedule cannot respond to the diffusion process’s varying dynamics. Our work identifies this fixed threshold as the source of the quality-latency tradeoff observed in prior methods, and proposes an adaptive and modular threshold mechanism on top of methods like TeaCache, EasyCache, and DiCache that modulates \(\tau\) based on the local rate of change of each method’s metric, enabling more aggressive caching in stable regions and more conservative caching at critical steps.

3 ACID↩︎

In this section, we introduce ACID, a method for adaptively varying caching frequency.

3.1 Preliminaries↩︎

3.1.0.1 Denoising Diffusion Models.

At inference time, diffusion models generate content through several sequential denoising steps. The core idea is to start with random noise and iteratively refine it until it approximates a sample from the target distribution. During training (the forward diffusion process), Gaussian noise is incrementally added over \(T\) steps to a sample datapoint \(x_0\) from a data distribution: \[x_t = \sqrt{\alpha_t} \, x_{t-1} + \sqrt{1 - \alpha_t} \, z_t, \quad t = 1, \ldots, T\] where \(\alpha_t \in [0, 1]\) governs the noise level and \(z_t \sim \mathcal{N}(0, I)\) represents Gaussian noise. As \(t\) increases, \(x_t\) becomes progressively noisier, ultimately approaching a standard normal distribution at \(t = T\). The reverse diffusion process, which is what we perform at inference time, is designed to reconstruct the original sample from the noise: \[p_\theta(x_{t-1} \mid x_t) = \mathcal{N}\!\left(x_{t-1};\, \mu_\theta(x_t, t),\, \Sigma_\theta(x_t, t)\right)\] where \(\mu_\theta\) and \(\Sigma_\theta\) are learned parameters defining the mean and covariance.

3.2 Analysis↩︎

3.2.0.1 Analyzing Caching Metrics.

To understand the limitations of fixed-threshold caching, we analyze how caching signals evolve throughout the denoising process. We begin with TeaCache’s \(\Delta\)TEMNI as a representative signal, since TeaCache is one of the earliest dynamic caching methods for video diffusion models and thus a natural starting point for analysis. We find that these structural properties generalize across other caching signals, motivating the general adaptive framework we propose.

Preliminary analysis of the \(\Delta\)TEMNI metric from TeaCache reveals two empirical properties of caching signals. First, these metrics remain relatively consistent across diverse prompts, as illustrated in Figure 2. Second, the structural behavior of these signals is model-dependent, as shown in Figure 3; specifically, HunyuanVideo and Wan 2.1 exhibit a characteristic U-shaped pattern, whereas CogVideoX demonstrates a distinctly different distribution. Based on these observations, we evaluate our proposed solution across three models (HunyuanVideo, Wan 2.1, and CogVideoX), which together cover both signal pattern types and ensure robustness against their varying behaviors.

As Figures 2 and 3 show, there are certain parts of the diffusion process where \(\Delta\)output and \(\Delta\)TEMNI are relatively high, and other parts where \(\Delta\)TEMNI is low. Dynamic feature caching methods already take this into account: when the predicted \(\Delta\)output is greater, they cache less often (because they hit the fixed threshold more often), and when the predicted \(\Delta\)output is smaller, they cache more often (they hit the fixed threshold less often). But while dynamic caching methods do find a better speed–quality tradeoff than the no-caching baseline, we find that this tradeoff is not optimal: the degree to which they cache more in stable zones or cache less in unstable zones is suboptimal—a limitation imposed by the single fixed threshold. We find that using different thresholds for different parts of the diffusion process—such as low thresholds for unstable regions and high thresholds for stable regions—leads to a better speed–quality tradeoff.

We could attribute the gains from adaptive thresholding to a suboptimal metric for TeaCache, but we importantly find the same gains from adaptive thresholding for EasyCache and DiCache as well, which use entirely different caching signals. This suggests the limitation is structural: any method that applies a single fixed threshold throughout the denoising process will face this suboptimality, regardless of the signal it uses.

To find a method with a better speed–quality tradeoff than any fixed-threshold approach, we ran a small experiment. Using TeaCache on HunyuanVideo, we compared a low fixed threshold, a high fixed threshold, and a range of two-threshold configurations in which we varied the transition point between the low and high threshold. We repeated this across two additional open-source models: Wan 2.1 and CogVideoX. Figure 4 shows the \(\Delta\)TEMNI signal throughout the denoising process for each model, with dashed lines marking the transition points at which a dual-threshold configuration had a better speed–quality tradeoff than both fixed-threshold baselines.

We then asked: what signal determines where these optimal transition points fall? Examining the second derivative of the TEMNI signal (Figure 5), we found that the optimal transition points coincide with the step at which the second derivative stabilizes—transitioning from a variable regime to a stable one. This suggests that the rate of change of the caching signal is a natural indicator of when to switch thresholds.

3.3 Adaptive Threshold Caching↩︎

ACID extends the single-threshold framework of methods like TeaCache, EasyCache, and DiCache with a dual-threshold strategy. Rather than applying a single fixed threshold \(\tau\), we maintain two thresholds: a high threshold \(\tau_{\text{high}}\) and a low threshold \(\tau_{\text{low}}\). The active threshold is selected based on the rate of change of the underlying signal of the dynamic caching method (\(\Delta\)TEMNI for TeaCache, \(k \times \Delta\text{input} / \|v_{t-1}\|\) for EasyCache, and \(\Delta\)probe features for DiCache).

When the rate of change is within [\(-\delta\), \(\delta\)], indicating a stable phase of the diffusion process, we apply \(\tau_{\text{high}}\), permitting more aggressive caching. When the rate of change escapes this [\(-\delta\), \(\delta\)] safe zone, indicating a more dynamic and quality-critical phase, we apply \(\tau_{\text{low}}\), restricting caching to preserve output fidelity.

This reflects the key observation that certain parts of the diffusion process are more important to preserve than others, and that the diffusion process cannot be treated as uniform throughout.

4 Experiments↩︎

4.1 Experimental Setup↩︎

4.1.0.1 Base models.

We evaluate on three open-source video generation diffusion models: HunyuanVideo T2V 720P [6], Wan 2.1 T2V 1.3B [4], and CogVideoX 1.5-5B [41].

4.1.0.2 Compared methods.

We apply our adaptive threshold framework on top of three base caching methods: TeaCache (evaluated on HunyuanVideo, Wan 2.1, CogVideoX), EasyCache (evaluated on HunyuanVideo, Wan 2.1), and DiCache (evaluated on HunyuanVideo, Wan 2.1). EasyCache and DiCache are not evaluated on CogVideoX because neither method provides an official implementation for that model; we restrict our evaluation to officially supported configurations to avoid confounding results from self-implemented ports. For each method–model combination, we compare a no-caching baseline, the fixed-threshold method with several threshold values spanning a low-to-high range, and our adaptive dual-threshold variant ACID with several low/high threshold values.

4.1.0.3 Evaluation metrics.

Following TeaCache, we evaluate along two dimensions: inference efficiency and visual quality. For inference efficiency, we report inference latency and speedup, defined as the ratio of the no-caching baseline latency to the method latency for the same model. For visual quality, we use LPIPS, PSNR, and SSIM, computed by comparing cached-method outputs against the no-caching baseline.

4.1.0.4 Implementation details.

All experiments were carried out on NVIDIA A100 80GB PCIe GPUs with PyTorch. We selected 33 prompts from VBench’s [42] prompt set: 3 prompts per group across 11 groups, where each group corresponds to one or more VBench evaluation dimensions, together spanning all 16 VBench dimensions. For each method–model combination, we generated 33 videos per configuration (one per prompt) across all evaluated threshold settings, using a fixed seed throughout.

4.2 Main Results↩︎

The Pareto plots in Appendix 6 show the efficiency and visual quality tradeoff offered by fixed-threshold methods, and how our adaptive dual-threshold method consistently expands the fixed-threshold Pareto frontier for each of the seven caching method and diffusion model combinations.

Table 1 shows the efficiency and visual quality of notable adaptive mode points for each feature caching method and diffusion model combination, comparing them to the baseline mode, the fixed-threshold mode using the low threshold of the adaptive mode, and the fixed-threshold mode using the high threshold of the adaptive mode.

4.2.0.1 TeaCache.

For TeaCache and HunyuanVideo, the adaptive mode achieves a 2.16\(\times\) speedup, between the low-threshold fixed mode’s speedup of 1.57\(\times\) and the high-threshold fixed mode’s speedup of 3.20\(\times\), while offering fidelity scores very similar to those of the low-threshold mode.

For TeaCache and Wan 2.1, the adaptive mode achieves a speedup of 1.96\(\times\), between the low-threshold fixed mode’s speedup of 1.42\(\times\) and the high-threshold fixed mode’s speedup of 2.86\(\times\), while retaining quality far closer to the low-threshold mode than to the high-threshold mode.

For TeaCache and CogVideoX, the adaptive mode achieves a speedup of 1.87\(\times\), between the low-threshold fixed mode’s speedup of 1.41\(\times\) and the high-threshold fixed mode’s speedup of 2.19\(\times\), again retaining quality far closer to the low-threshold mode than to the high-threshold mode.

Figure 7 shows how ACID’s adaptive modes surpass the Pareto frontier curves for all three diffusion models when used with TeaCache.

4.2.0.2 EasyCache.

For EasyCache and HunyuanVideo, our adaptive mode achieves a speedup of 2.39\(\times\), much closer to the high-threshold fixed mode’s speedup of 2.55\(\times\) than to the low-threshold fixed mode’s speedup of 2.06\(\times\), with quality metrics falling between those of the two fixed modes.

For EasyCache and Wan 2.1, our adaptive mode achieves a speedup of 1.92\(\times\), falling between the low- and high-threshold fixed mode speedups of 1.73\(\times\) and 2.12\(\times\), while retaining quality far closer to the low-threshold mode than to the high-threshold mode.

Figure 8 shows how several of ACID’s adaptive threshold modes surpass the Pareto frontier curves for both diffusion models when used with EasyCache.

We also note that EasyCache, by running the full model for the first few diffusion steps, already supports our main idea: varying the threshold across the diffusion process—in this case using a threshold of 0 (full computation) at the beginning and a higher threshold later—leads to a better quality–efficiency tradeoff.

4.2.0.3 DiCache.

For DiCache and HunyuanVideo, the adaptive mode achieves a speedup of 2.04\(\times\), falling between the low- and high-threshold speedups of 1.92\(\times\) and 3.02\(\times\), while achieving visual quality very close to the low-threshold mode and much better than the high-threshold mode.

For DiCache and Wan 2.1, the adaptive mode achieves a speedup of 2.73\(\times\), falling between the low- and high-threshold speedups of 2.27\(\times\) and 3.97\(\times\), while achieving visual quality very close to the low-threshold mode and much better than the high-threshold mode.

Figure 9 in Appendix 6 shows how several of ACID’s adaptive modes surpass the Pareto frontier curve for both diffusion models when used with DiCache.

5 Conclusion↩︎

We proposed ACID, a dynamic dual-threshold caching method for accelerating video diffusion model inference. By applying a high caching threshold during stable phases of the diffusion process and a low threshold during quality-critical phases, identified via the rate of change of each method’s caching signal, our method improves the Pareto frontier in the video quality vs.generation speed space over any fixed-threshold dynamic caching method. Experiments across three open-source video generation models and three caching methods confirm that ACID achieves near-low-threshold visual quality with substantially higher inference speedup.

6 Pareto Frontier Plots↩︎

Figures 7–9 show Pareto frontier plots for all evaluated method–model combinations. Each plot compares fixed-threshold configurations across a range of thresholds against our adaptive dual-threshold variants. In each case, adaptive points expand the frontier beyond what any fixed threshold achieves.

7 Detailed Per-Combo Results↩︎

All configurations evaluated for each method \(\times\) model combination are listed below, sorted by increasing speedup. Bold rows are adaptive modes. Fidelity metrics (PSNR, SSIM, LPIPS) are computed against the uncached baseline. \(\tau\) denotes the caching threshold for all methods (cache-reuse residual for TeaCache/EasyCache; feature-difference score for DiCache). Subscripts \(\ell\)/\(h\) mark the low/high threshold in adaptive modes. Adaptive rows that share the same threshold pair differ in the stability bound \(\delta\).

5pt

5pt

5pt

5pt

5pt

5pt

5pt

References↩︎