1 Introduction↩︎

Diffusion Large Language Models (dLLMs) have emerged as a compelling alternative to autoregressive models (ARMS), demonstrating strong performance in text generation tasks. Notable examples such as LLaDA [1], [2] and Dream [3] highlight the rapid progress in this direction. A key advantage of dLLMs lies in their bidirectional attention mechanisms, which enhance scalability and enable superior performance in handling complex scenarios, such as the “reversal curse” [4], where traditional ARMs often struggle. This allows dLLMs to capture richer contextual dependencies in challenging scenarios.

However, despite their strong performance in certain domains, dLLMs face a fundamental challenge: their computational complexity scales as \(\mathcal{O}(L^3)\) with respect to sequence length \(L\), significantly exceeding the \(\mathcal{O}(L^2)\) cost of autoregressive models (ARs). This cubic scaling imposes a severe bottleneck for long-sequence and real-time generation tasks, limiting the practical deployability of dLLMs in latency-sensitive applications. The root cause lies in the non-autoregressive nature of dLLMs, where each denoising step requires updating all tokens in parallel across the full sequence. Besides, this paradigm hinders the caching of key-value activations from previous steps, rendering dLLMs incompatible with the widely used KV-Cache mechanism.

1.0.0.1 Key observations.

To address this issue, recent work has explored strategies for dLLM acceleration. For example, [5]–[7] reduce redundancy by caching internal token representations across decoding steps. Concurrently, [8] accelerates inference by enabling parallel unmasking of multiple tokens within a single step. These methods implicitly rely on specific token properties, such as feature stability and confidence, to identify opportunities for optimization. However, they all rely on a static strategy across all layers and decoding steps, applying the same caching or unmasking criteria throughout the model and generation process, thus overlooking the dynamic nature of token behavior during generation.

As illustrated in Figure 1 (a-d), the token properties vary across different layers and steps. The frequency of changes in the internal features of tokens differs across layers, while the distributions of token confidence fluctuate across decoding steps. The static strategies adopted by existing methods may fail to account for this dynamic behavior, leading to performance degradation. Therefore, this observation prompts a critical question: how to design an adaptive method that dynamically aligns with the model’s intrinsic layer-wise and step-wise token dynamics to improve the efficiency?

1.0.0.2 Our solution.

In this work, we propose Dynamic-dLLM, a training-free framework for accelerating dLLM inference. Dynamic-dLLM consists of two key components: Dynamic Cache Updating (DCU) and Adaptive Parallel Decoding (APD).

Specifically, as tokens may exhibit heterogeneous dynamics across layers, instead of a static cache updating strategy across all layers, we propose Dynamic Cache Updating (DCU) that allocates cache-update budgets adaptively, ensuring that layers requiring frequent updates are prioritized, while computational overhead is reduced in stable layers. In addition, the existing parallel decoding strategy with fixed thresholds risks committing to tokens prematurely, as confidence estimates can shift over time, leading to error propagation. To mitigate this, we introduce Adaptive Parallel Decoding (APD) that dynamically calibrates decoding thresholds by tracking the evolving distribution of prediction confidence, achieving a decent trade-off between the degradation of generation quality caused by a low threshold and the inefficiency resulting from a high threshold.

Extensive experiments across LLaDA-8B-Instruct, LLaDA-1.5, Dream-v0-7B-Instruct, and benchmarks covering mathematics, science, coding, and general tasks demonstrate the effectiveness and strong generalization capabilities of the proposed method. Notably, Dynamic-dLLM achieves a maximum acceleration of up to 4.48×, with an average speedup exceeding 3× while still maintaining performance, making it a plug-and-play training-free solution for enhancing the efficiency of dLLMs without compromising performance. In summary, our contributions are as follows:

• In this study, we observe that the variations across layers and decoding steps of dLLM may undermine the effectiveness of existing static rule-based acceleration methods.

• We propose Dynamic-dLLM, a training-free framework composed of Dynamic Cache Updating (DCU) and Adaptive Parallel Decoding (APD), DCU adaptively allocates cache-update budgets across layers, while APD dynamically calibrates decoding thresholds across steps, jointly enabling efficient yet robust acceleration of dLLMs.

• Extensive experiments across diverse models and tasks show that Dynamic-dLLM substantially improves inference efficiency while preserving the accuracy, outperforming state-of-the-art acceleration methods.

2 Background and Motivation↩︎

2.1 Preliminaries of dLLM↩︎

In this section, we introduce preliminaries regarding the inference process of dLLM [1]. The introduction of related work is presented in the Appendix 9 due to the page limit.

Given a prompt of length \(L_{\text{prompt}}\) tokens and a target generation length of \(L_{\text{gen}}\) tokens, let \(L = L_{\text{prompt}} + L_{\text{gen}}\). The dLLM generates the output in \(T\) iterative decoding steps, producing approximately \(L_{\text{gen}} / T\) tokens per step. Let \(\mathcal{V}\) denote the model’s vocabulary, and let \([\text{MASK}] \in \mathcal{V}\) be a special placeholder token indicating positions to be predicted. Denote by \(\mathbf{x}^t \in \mathcal{V}^L\) the token sequence at step \(t\), where \(t = T, T-1, \dots, 0\). The initial sequence is constructed as: \[\small \mathbf{x}^T = (x_0, \dots, x_{L_{\text{prompt}}-1}, [\text{MASK}], \dots, [\text{MASK}]),\] where \(x_i\) are the given prompt tokens. At each step \(t\), the mask predictor \(f_\theta\) computes a distribution over the vocabulary for each position: \[\small \mathbf{z}^t = f_\theta(\mathbf{x}^t) \in \mathbb{R}^{L \times |\mathcal{V}|}.\] Using greedy decoding, we can obtain the most probable token at each masked position: \[\small \hat{x}_i^t = \underset{v \in \mathcal{V}}{\arg\max} \left( \mathrm{Softmax}(\mathbf{z}_i^t) \right)_v, \quad \text{if } x_i^t = [\text{MASK}].\] A transition function \(S\) then updates the sequence to \(\mathbf{x}^{t-1}\) by selectively replacing tokens based on confidence scores, re-masking low-confidence predictions to refine them in subsequent steps: \(\mathbf{x}^{t-1} = S(\hat{\mathbf{x}}^t, \mathbf{x}^t, t).\) The final output sequence \(\mathbf{x}^0\) is yielded when \(t = 0\).

2.2 Key Observations↩︎

Despite recent progress in accelerating diffusion-style LLMs (dLLMs) [5]–[8], two critical inefficiencies remain unaddressed.

2.2.0.1 Layer-wise Cache Update Needs Vary Significantly.

Existing methods exploit temporal redundancy by reusing cached intermediate features (e.g., query, key, value, attention output, FFN output) from the previous step for a subset of tokens, assuming high feature similarity across steps. However, as illustrated in Figure 1 (a-d), the proportion of tokens requiring cache updates varies substantially across layers, increasing monotonically from shallow to deep layers. This suggests that uniform or heuristic caching strategies are suboptimal. Instead, a layer-adaptive cache update policy is essential for dynamically allocating computation budgets where they matter most.

2.2.0.2 Static Thresholding Hinders Parallel Decoding Efficacy.

Parallel decoding strategy (e.g., [8]) unmask tokens once their confidence exceeds a fixed threshold. Yet, as shown in Figure 1 (e), the token with the highest confidence at an early step may not be the desired output and will be revised later, often replaced by its “runner-up” prediction with the second-highest confidence initially. Conversely, tokens whose top prediction exhibits clear dominance over alternatives, i.e., low entropy or large margin, can be safely finalized earlier, even if absolute confidence remains below a static threshold. Therefore, to enable earlier commitment to stable predictions, thereby expediting convergence without compromising accuracy, exploring the feasibility of a dynamic per-token threshold, adjusting adaptively based on the predicted distribution (e.g., entropy or probability margin), becomes essential.

3 Method↩︎

To overcome the limitations of existing approaches, we propose Dynamic-dLLM, a training-free acceleration framework that dynamically optimizes dLLM inference along two dimensions: cache-update management and parallel decoding scheduling.

Regarding cache-update management, we introduce a dynamic allocation mechanism for managing cache updates, recognizing the varying dynamics across layers. This approach dynamically distributes the update budget among layers, prioritizing layers that require more frequent cache updates. On the other hand, for optimizing the parallel decoding, we replace fixed confidence thresholds with an adaptive per-token unmasking strategy, based on the predicted distribution of each token. This strategy facilitates early commitment to confident predictions while postponing uncertain ones, achieving a more balanced trade-off between speed and output quality.

The overview is presented in Figure 2. Sections 3.1 and 3.2 detail each component, respectively.

3.1 Dynamic Cache Updating↩︎

Recent works [5]–[7] update a fixed or uniform number of token caches across all layers. However, as demonstrated in Section 2.2, the demand for cache updates varies significantly across layers. This observation motivates the need for a dynamic allocation strategy that adapts the cache-update budget per layer according to the specific requirement.

In this section, we propose the Dynamic Cache Updating (DCU) strategy, which selectively updates only those tokens whose representations undergo significant changes between consecutive inference steps. Prior work [5] identifies such tokens by measuring the cosine similarity between the current and cached Value vectors. While effective, this approach incurs non-negligible computational overhead due to the explicit recomputation and comparison of Value vectors. Ideally, if token dynamics could be estimated without recomputing these vectors, cached values could be safely reused, thereby reducing redundancy.

Inspired by [9], who observed a strong correlation between model inputs and outputs in diffusion transformers (DiT) [10], we investigate the relationship between layer inputs and intermediate features in dLLMs. As shown in Figure 3, the features cached (e.g., Key, Value, Attention Output, and FFN Output) exhibit high correlation with the corresponding layer inputs. This implies that changes in layer inputs across steps serve as a reliable proxy for the underlying dynamics of intermediate activations. Consequently, input-level differences can effectively inform cache-update decisions without accessing or recomputing the cached features themselves.

3.1.0.1 Layer-Adaptive Cache Budget Allocation.

To dynamically allocate the cache update budget across layers, we first define a token-level dissimilarity metric, \(d_i^{t,l}\), estimating the change in the representation of token \(x_i\) at layer \(l\) between consecutive inference steps \(t\) and \(t+1\). This metric is computed using the cosine distance between the normalized token inputs at the respective steps: \[\small d_i^{t,l} = 1 - \frac{(\mathbf{x}_{i}^{t,l})^\top\mathbf{x}_{i}^{t+1, l}}{\|\mathbf{x}_{i}^{t,l}\|\|\mathbf{x}_{i}^{t+1, l}\|}\] A higher value of \(d_i^{t,l}\) denotes a greater change in the token’s representation, suggesting a higher need for cache update. Then, we aggregate the token-level variations into a layer-wise metric \(s^{t,l}\). This metric represents the average change in token representations within layer \(l\): \[\small s^{t,l} = \frac{1}{N} \sum_{i=0}^{N-1} d_i^{t,l},\] where \(N\) is the sequence length. Subsequently, the cache update budget for layer \(l\) at step \(t\), denoted as \(B^{t,l}_{\text{layer}}\), is then allocated proportionally to its measured dynamism at the previous step (\(t+1\)), \(s^{t+1,l}\). This allocation is normalized across all layers using the total available budget, \(B_{\text{layer}} \times \texttt{LayerNum}\): \[\small B^{t,l}_{\text{layer}} = (B_{\text{layer}} \times \texttt{LayerNum}) \cdot \frac{s^{t+1,l}}{\sum_{k=0}^{\texttt{LayerNum}-1} s^{t+1,k}}.\] For each layer \(l\), the set of tokens scheduled for cache update at step \(t\), denoted \(\mathcal{U}^{t,l}\), is initialized as an empty set at the start of the step: \(\mathcal{U}^{t,l} \gets \emptyset\). Then, layer \(l\) identifies the set \(\mathcal{S}^{t,l}\) comprising the top-\(B^{t,l}_{\text{layer}}\) tokens with the highest variation \(d^{t,l}_i\). These selected tokens are then added to the update set: \(\mathcal{U}^{t,l} \gets \mathcal{U}^{t,l} \cup \mathcal{S}^{t,l}\).

3.1.0.2 Token Stuck in the Mud.

Nevertheless, the layer-adaptive cache budget allocation strategy may potentially make some tokens stuck in the mud. Specifically, if a token \(x_i\) is not selected for an update in layer \(l\), its cached representation remains unchanged. Consequently, its input to layer \(l+1\) also remains static, leading to a zero variation score \(d_i^{t,l+1} = 0\) for that layer. As the allocation strategy prioritizes tokens with high \(d_i^{t,l+1}\), the token \(x_i\) will only be updated in layer \(l+1\) if the number of tokens exhibiting non-zero variation is insufficient to fill the allocated budget \(B_{\text{layer}}^{t,l+1}\). Should this occur, and if \(x_i\) is again not selected (e.g., chosen randomly among the low-priority tokens), it will remain unchanged entering layer \(l+2\), perpetuating the cycle. We refer to this phenomenon, where a token fails to be updated across multiple consecutive layers due to consistently low variation scores induced by prior missed updates, as a token becoming stuck in the mud.

3.1.0.3 Mandatory Update Window.

As illustrated in Figure 4, there exists a spatial locality in the update pattern: tokens surrounding the one unmasked in the previous step (the key token) are statistically more likely to be updated in the current step. Let the position of the key token be \(p\). To mitigate the risk of the next key token (the token with the highest confidence to be unmasked in the current step) becoming stuck in the mud, we introduce a Mandatory Update Window. This mechanism ensures that a local region around the key token is always updated, regardless of the adaptive budget allocation. Formally, we define a window of fixed size \(B_{\text{window}}\) centered on the key token’s position \(p\). The set of token positions covered by this window at a given step is \(\left[ p - \frac{B_{\text{window}}}{2}, p + \frac{B_{\text{window}}}{2} \right]\). For each layer \(l\), the caches for all tokens within this window are compulsorily added to the layer’s update set \(\mathcal{U}^{t,l}\): \[\small \mathcal{U}^{t,l} \gets \mathcal{U}^{t,l} \cup \left\{ x_i \,\middle|\, p - \frac{B_{\text{window}}}{2} \le i \le p + \frac{B_{\text{window}}}{2} \right\}.\] This updated set \(\mathcal{U}^{t,l}\) then constitutes the final list of tokens whose caches will be recomputed for layer \(l\) in the current step. By ensuring continuous updates within this local window, we reduce the likelihood of critical tokens being overlooked and retain the response to local changes. The global budget is subsequently distributed adaptively among the remaining tokens based on the layer-specific variation metrics \(s^{t,l}\) for the following step.

3.2 Adaptive Parallel Decoding↩︎

Section 2.2 highlights that the peak confidence of a token can vary significantly across decoding steps in dLLMs. This inherent dynamism poses a challenge for fixed-threshold parallel decoding methods [8], which rely on a static criterion and consequently suffer from decoding inaccuracies due to mispredictions at certain steps.

To address this, we introduce the Adaptive Parallel Decoding mechanism that dynamically adjusts the masking threshold for each token based on its local prediction stability. Each token \(x_i\) starts with an initial threshold \(\tau^{T}_i\). The threshold at step \(t\), denoted \(\tau^t_i\), is adapted from the threshold used at the previous step \(t+1\), \(\tau^{t+1}_i\).

3.2.0.1 Adaptive Threshold via Confidence Concentration.

The core idea is to modulate the threshold based on the concentration of the token’s predicted probability distribution. Intuitively, a diffuse distribution (a small gap between the highest and second-highest probabilities) suggests lower confidence in the current prediction, warranting a stricter (higher) threshold to reduce unnecessary updates. Conversely, a concentrated distribution indicates stability, permitting a reduced threshold for early decoding. Let \(\mathbf{z}_{i}^t\) be the probability distribution over the vocabulary \(\mathcal{V}\) for token \(x_i\) at step \(t\), the index of the most likely token is: \(u = \arg\max_{v \in \mathcal{V}} \left( \mathbf{z}_{i}^t \right)_v.\) Thus, the concentration of this distribution is quantified using the second-highest probability score: \[c_i^t = 1 - \max_{v \in \mathcal{V} \setminus \{u\}} \left( \mathbf{z}_{i}^t \right)_v.\] A larger value of \(c_i^t\) signifies a more peaked and confident distribution. Based on this measure, the decoding threshold for token \(x_i\) at step \(t\) is adjusted as follows: \[\small \tau^t_i = \tau^{t+1}_i - \alpha \cdot c_{i}^t,\] where \(\alpha\) is a positive hyperparameter controlling the sensitivity of the threshold adaptation. This formulation ensures that tokens with highly concentrated distributions (large \(c_i^t\)) have their thresholds decreased, allowing for early decoding, while tokens with diffused distributions have increased thresholds to prevent decoding errors.

3.2.0.2 Integration with Temporal Instability.

In addition, the magnitude of historical shifts in a token’s confidence distribution provides a strong signal for its likelihood of future revision. We quantify this shift via the cosine distance between the token’s confidence distributions at adjacent steps: \[\small H_{i}^t = 1 - \frac{(\mathbf{z}_{i}^t)^\top \mathbf{z}_{i}^{t+1}}{\|\mathbf{z}_{i}^t\| \, \|\mathbf{z}_{i}^{t+1}\|}. \label{EQ10}\tag{1}\] A larger \(H_i^t\) indicates greater instability in the prediction, suggesting that the token may still be undergoing refinement and thus warrants a stricter (higher) threshold to prevent early decoding. By combining \(c_i^t\) and \(H_i^t\), the decoding threshold for token \(x_i\) at step \(t\) is updated as: \[\small \tau^t_i = \tau^{t+1}_i - \alpha \cdot c_{i}^t + \beta \cdot H_{i}^t,\] where \(\alpha, \beta \geq 0\) are hyperparameters balancing the influence of prediction confidence and temporal instability.

Algorithms 6 and 7 in the Appendix outline the core mechanisms of Dynamic-dLLM for accelerating dynamic LLMs (dLLMs) via Feature-Caching and Parallel Decoding, respectively. By explicitly accounting for dynamism along both the layer and step dimensions, Dynamic-dLLM minimizes redundant computation and thereby significantly accelerates the inference process of dLLMs.

4 Experiment↩︎

4.1 Experiment Settings↩︎

We assessed the performance of Dynamic-dLLM using three typical dLLMs as baselines: LLaDA-8B-Instruct, LLaDA-1.5 and Dream-7B-Instruct. If not otherwise specified, we default \(B_{\text{layer}}\) to 32, and \(B_{\text{window}}\) to 32. More experimental details are shown in Appendix 7.

To comprehensively evaluate a model’s performance and efficiency, we employ two key metrics: accuracy on benchmarks and throughput, with the latter measured in Tokens Per Second (TPS). The benchmarks includes MMLU (5-shot)[11], ARC-challenge (ARC-c, 0-shot)[12], GPQA (5-shot)[13], GSM8k (4-shot)[14], and HumanEval (HE, 0-shot)[15]. For fair comparison, we divided the methods into two groups, one using Feature-Cache[5], [6], [8] and the other using KV-Cache and parallel decoding[8]. All experiments were performed on NVIDIA Pro6000 GPUs.

4.2 Main Results↩︎

Our results (baseline vs. alternative methods vs. our Dynamic-dLLM) are presented in Table 1, 2, and 3. These results show that Dynamic-dLLM not only achieves the most significant throughput improvement but also maintains performance.

With only feature cache enabled, Dynamic-dLLM delivers substantial speedups for high-priority tasks without accuracy degradation. It achieves notable throughput boosts on benchmarks with an average speedup of over 2.5× across all evaluated tasks for LLaDA-8B-Instruct , while maintaining accuracy. When combined with parallel decoding, Dynamic-dLLM scales speedups. For LLaDA-8B-Instruct on GSM8k, throughput hits 37.29 TPS (4.48× faster than the baseline’s 8.32 TPS), with average speedup across tasks reaching 3.21× and robust accuracy.

This superiority persists across models: LLaDA-1.5 achieves 4.46× speedup on GSM8k (37.02 vs. 8.30 TPS) with near-baseline accuracy (60.67% vs. 61.08%); Dream-v0-7B-Instruct gains 3.91× speedup on GSM8k (31.48 vs. 8.05 TPS). These cross-model results demonstrate its generalization capabilities.

4.3 Ablation Studies↩︎

In this section, we present the ablation studies regarding the core designs of our method.

Impact of \(B_{\text{layer}}\) on Accuracy and Throughput. As shown in Figure 5 (a), we fix the \(B_{\text{window}}\) to 32 and do not use parallel decoding, and explore the impact of \(B_{\text{layer}}\) on accuracy and throughput. With the gradual increase of \(B_{\text{window}}\), the accuracy shows an upward trend, reaching a plateau around 32. On the other hand, the throughput also rapidly decreases with the increase of \(B_{\text{window}}\). Based on observations, a value of 32 for \(B_{\text{window}}\) is a more trade-off choice.

Impact of \(B_{\text{window}}\) on Accuracy and Throughput. Similarly, we discussed the impact of \(B_{\text{window}}\) on accuracy and throughput in Figure 5 (b). \(B_{\text{window}}\) is fixed to 32 and parallel decoding is disabled. The impact of \(B_{\text{window}}\) on accuracy and throughput is roughly the same as that of \(B_{\text{layer}}\), but the smaller \(B_{\text{window}}\) has a more severe reduction in accuracy than \(B_{\text{layer}}\). To ensure that the accuracy is basically on par with the baseline, we have chosen 32 as the optimal value for \(B_{\text{window}}\).

Dynamic Threshold vs. Fixed Threshold. We discussed the difference between fixed threshold and dynamic threshold in Figure 5 (c). The accuracy of both is the same under all initialization. However, dynamic thresholds bring fewer inference steps than fixed thresholds in higher initialization. With the maximum initialization of 0.9, which does not excessively descend performance, dynamic thresholds can reduce inference steps by approximately 30% compared to the fixed thresholds.

5 Concluding Remarks↩︎

5.0.0.1 Summary.

We present Dynamic-dLLM, a training-free framework for accelerating diffusion LLMs by adapting to the dynamic behavior of tokens across layers and decoding steps. By introducing dynamic cache updating and adaptive parallel decoding, our method significantly reduces redundant computation while preserving generation quality. Extensive experiments across various models and benchmarks covering mathematics, science, coding, and general tasks demonstrate the effectiveness and strong generalization capabilities of the proposed method. In summary, Dynamic-dLLM offers a general, plug-and-play solution toward efficient dLLM inference, highlighting the importance of adaptive strategies in non-autoregressive generation.

5.0.0.2 Limitation & Future Work.

While Dynamic-dLLM demonstrates strong performance across standard language generation benchmarks, its capabilities in multi-modal understanding and complex reasoning scenarios remain largely unexplored. In particular, the model’s current design is tailored to unimodal textual inputs, and it is unclear how its core mechanisms generalize to settings involving heterogeneous data modalities (e.g., vision, audio, or structured knowledge). Future work should investigate how these principles can be reformulated or extended to address the unique challenges of cross-modal alignment, representation fusion, and modality-specific computational demands. Such extensions could unlock new avenues for building more flexible and efficient foundation models capable of robust reasoning across diverse input types.

Acknowledgement↩︎

This work was supported by the Guangdong Basic and Applied Basic Research Foundation (2025A1515011546) and by the Shenzhen Science and Technology Program (JCYJ20240813105901003, KJZD20240903102901003, ZDCY20250901113000001).

6 Algorithm Supplement↩︎

The pseudocode of Dynamic-dLLM is shown in Algorithm 6 and 7.

7 Experiment Details↩︎

7.1 Benchmarks and Settings↩︎

Table 4 shows the specific setups for each benchmark, involving the count of decoding steps, block length, and generation length. The benchmarks encompass MMLU (5-shot), ARC-C (0-shot), GSM8K (4-shot), Math (4-shot), and HumanEval (0-shot). To examine the generalization and robustness of diverse approaches, we reduce task-dependent hyperparameter adjustments and instead use a uniform setup for all benchmarks except HumanEval. Owing to its unique task characteristic, HumanEval demands a greater number of decoding steps and a longer generation length.

7.2 Implementation Details↩︎

We offer a thorough explanation of the parameter setups for the compared methods dKV-Cache and dLLM-Cache across various models. According to the suggested configurations in the dKV-Cache paper, we set the cache update interval to 8 for the LLaDA series and to 4 for the Dream series.

For dLLM-Cache, the paper presents multiple parameter configurations, where \(K_p\) denotes the prompt refresh interval and \(K_r\) represents the response refresh interval. specifically, For LLaDA-8B-Instruct: \(K_p = 50\), \(K_r = 7\); For LLaDA-1.5: \(K_p = 100\), \(K_r = 6\); For Dream-v0-7B-Instruct: \(K_p = 50\), \(K_r = 2\).

In addition, for Adaptive Parallel Decoding (APD) in Dynamic-dLLM, we set \(\alpha=0.001\) and \(\beta=0.0008\) based on extensive statistical analysis.

8 Example Description↩︎

As shown in Figure 8, in the absence of candidate, a fixed threshold can actually hinder early decoding of correct predictions, while Adaptive Parallel Decoding monitors the status of candidates in real time and ends unnecessary steps early.

9 Related Work↩︎

Large Language Models. Driven by transformer-based architectures and large-scale pretraining, Large Language Models (LLMs) have achieved remarkable success, demonstrating exceptional capabilities[16]–[23]. While these models primarily excel in textual processing[24]–[27], their robust architectural foundation has paved the way for various functional extensions[20], [27]–[30], such as semantic segmentation and object detection[30]–[35]. One notable derivative branch is the development of Multimodal Large Language Models (MLLMs)[36]–[39], which expand the core LLM utility by integrating modality encoders to process inputs beyond text, such as image, audio, and video. Through this extension, the reasoning power of the central LLM backbone has been adapted to assist in other domains, including traditional computer vision tasks, as illustrated by specific applications like LISA[40] and LISA++[41] in reasoning segmentation.

Diffusion Large Language Models. Diffusion models, which excel in continuous data generation through iterative denoising processes [42], [43], have recently shown promising potential in natural language processing. Unlike their success in image domains [44], [45], adapting these models to text generation faces fundamental challenges arising from the discrete token space and sequential dependencies inherent in language. A key advancement in addressing these issues comes from discrete diffusion frameworks, particularly Masked Diffusion Models (MDMs) that operate by progressively refining sequences through context-aware mask prediction [46], [47]. Recent methodological innovations have significantly expanded the capabilities of diffusion-based language models. Scaling efforts have produced foundation models like LLaDA[1], an 8B parameter bidirectional Transformer trained from scratch, and Dream[3] which leverages pre-trained autoregressive weights, both achieving performance parity with similarly sized autoregressive models.

Inference acceleration methods of dLLMs. Multiple studies have investigated strategies for speeding up discrete diffusion large language models (dLLMs). Some studies using feature caching cut down on computational costs by caching the internal features of tokens across different diffusion steps. dLLM-Cache [5] selects a fixed proportion of tokens for cache update for each layer by sorting the cosine similarity of Value vector between adjacent steps. dKV-Cache [6] puts the tokens decoded at each step into the cache and doeses not update them in later steps. Fast-dLLM [8] puts tokens outside the current block to the cache and updates tokens within the current block. The similarity of these methods is that they adopt the same cache update strategy for all layers, which ignores the different requirements of each layer. In addition, Fast-dLLM proposes the parallel decoding strategy, unmasking the tokens with confidence exceeding a predetermined threshold, which has difficulty balancing quality and efficiency.

10 Experimental Supplements↩︎

10.1 Stability Analysis of Hyperparameters↩︎

Stability Analysis of \(B_{layer}\) and \(B_{window}\). To investigate the stability of parameter settings across diverse scenarios, we conducted additional experiments. As shown in Table 5, when the sequence length exceeds 1k, the accuracy of the default settings (\(B_{layer}=32\), \(B_{window}=32\)) exhibits a slight decline. However, with appropriate increases in these two parameters, the accuracy gradually recovers. Furthermore, when the sum of \(B_{layer}\) and \(B_{window}\) is fixed, different proportional allocations between them lead to varying impacts on performance. Through these experiments, we confirmed that setting \(B_{layer}\) equal to \(B_{window}\) yields optimal results. To ensure the method’s stability across different generation lengths, we propose an auto-tuning strategy: \(B_{layer}=B_{window}=\frac{1}{8} \times gen\_len\) (where \(gen\_len\) denotes the generation length).

Stability Analysis of \(\alpha\) and \(\beta\). For the hyperparameters \(\alpha=0.001\) and \(\beta=0.0008\), we performed experiments under different output length settings (256, 512, and 1024), while maintaining \(B_{window}=\frac{1}{8} \times gen\_len\) and \(B_{layer}=\frac{1}{8} \times gen\_len\) (see Table 6). The results indicate that there is almost no degradation in accuracy across these different generation lengths, verifying the strong stability of \(\alpha\) and \(\beta\) .

As per Equation 1 , \(\alpha\) and \(\beta\) regulate threshold adaptation to prediction confidence and temporal instability, respectively. We conducted ablation experiments on GSM8K (\(gen\_len\)=256), with results in Table 7 (score=accuracy; NIS=Number of Inference Steps, fewer = faster). The optimal range for \(\alpha/\beta\) is the \(10^{-3}\) order of magnitude. The model is sensitive to order-of-magnitude scaling (severe quality loss with over-scaling) but stable to small variations within this range.

10.2 Analysis of Additional Latency↩︎

We present comprehensive metrics in Table 8, which details the performance scores (accuracy), average single-inference latency, and additional latency introduced by DCU and APD separately—all evaluated on the GSM8K dataset across different generation lengths (\(gen\_len\): 256, 512, 1024). As shown in the Table 8, both DCU and APD introduce minimal additional latency, and this overhead exhibits a clear linear scaling trend with generation length, which only accounts for a small portion of the inference latency, far less than the gain it brings.

4pt

10.3 Performance in Low-resource Hardware↩︎

To address low-resource hardware deployment, we tested our method with the LLaDA-8B-Instruct model on the GSM8K and HumanEval datasets using an RTX 4090 GPU. As shown in the Table [4090], our method still maintains a strong speed-accuracy trade-off under such cost-constrained settings, demonstrating its practicality for deployment on non-high-end hardware.

References↩︎