1 Introduction↩︎

Large language models are typically decoded autoregressively [1]–[3], where each generated token requires a new forward pass of the target model. This sequential process becomes a major latency bottleneck, especially for long-form generation and interactive applications. Speculative decoding mitigates this problem by using a lightweight draft model to propose multiple future tokens, which are then verified in parallel by the target model while preserving the target distribution [4], [5].

The efficiency of speculative decoding depends heavily on the drafter [6]–[10]. Traditional autoregressive drafters maintain strong local consistency but still generate draft tokens sequentially [8], [11]. Recent block-parallel drafters, such as DFlash, improve drafting efficiency by predicting an entire block of tokens in one pass [12]–[14]. However, this parallelism also introduces a limitation: tokens inside the same draft block are predicted largely independently, without explicitly conditioning on previously drafted tokens in that block [14]–[16]. As a result, DFlash provides strong long-context predictions from the prefix, but lacks explicit short-context causal modeling within the draft block.

Several recent methods, including Domino and DSpark, attempt to address this issue by introducing intra-block causality into block-parallel drafting [15]–[18]. These methods show that modeling causal dependencies inside the draft block can improve acceptance. However, they usually require training a new draft model from scratch or jointly training the draft backbone with additional causal components. This makes them costly to apply when a DFlash-style drafter has already been trained, and limits their flexibility across different draft checkpoints.

In this work, we propose DeLS-Spec, a decoupled long-short context speculative decoding method. Instead of modifying or retraining the DFlash backbone [14], DeLS-Spec keeps DFlash fixed as a long-context expert and introduces a lightweight local head as a short-context expert. The DFlash expert captures semantic and task-level information from the full prefix, while the local head captures causal dependencies from the already drafted tokens inside the current block.

A key property of DeLS-Spec is its decoupled training. The local head is trained independently with a standard next-token prediction objective on plain text data. It does not require target-model hidden states, DFlash hidden states, or joint optimization with the speculative decoding pipeline. This makes training extremely cheap and allows the local head to be attached to existing DFlash-style checkpoints after they have been trained. At inference time, DeLS-Spec combines the long-context logits from DFlash and the short-context logits from the local head [19]. Since the local head is not trained together with a specific DFlash checkpoint, it is not one-to-one tied to the DFlash weights during inference. This makes the method more modular and flexible: a trained local head can be reused or transferred across compatible DFlash checkpoints, rather than requiring a new draft model to be trained for each setting.

We evaluate DeLS-Spec on Qwen3 models [3] across math, code, and dialogue benchmarks. Experiments show that DeLS-Spec consistently improves both decoding speedup and average acceptance length over DFlash. These results demonstrate that decoupling long-context block-parallel drafting from short-context causal correction is an efficient and practical way to enhance existing speculative decoding systems.

Our contributions are summarized as follows:

• We propose DeLS-Spec, a decoupled long-short context speculative decoding method that improves DFlash-style block-parallel drafting without retraining the DFlash backbone.

• We introduce a lightweight local head that provides intra-block causal information and can be trained independently with a standard next-token prediction objective.

• DeLS-Spec consistently enhances DFlash on Qwen3 models across math, code, and dialogue benchmarks, while offering modular flexibility across different checkpoints.

2 Related Work↩︎

2.0.0.1 Speculative decoding and classical drafters.

Speculative decoding accelerates autoregressive inference by using a lightweight drafter to propose multiple tokens and a target model to verify them in parallel while preserving the target distribution [4], [5]. Early blockwise decoding and tree-based verification reduce sequential decoding rounds or expand candidate coverage [20], [21]. Later drafters improve proposal quality through target-attached heads, sequentially dependent heads, feature-level drafting, dynamic draft trees, recurrent drafting, and distillation [6]–[11], [22]. However, many drafters still require sequential drafting, feature updates, or tree construction, making drafting cost grow with the speculative budget.

2.0.0.2 Parallel and diffusion-based drafting.

Parallel generation reduces drafting latency by predicting multiple positions simultaneously, but also weakens token dependency, as observed in non-autoregressive and diffusion-style language generation [23]–[27]. Recent speculative drafters therefore use diffusion or parallel prediction to generate draft blocks efficiently [12], [13], [28]–[30]. DFlash is most relevant to our work: it uses a lightweight block-diffusion drafter with target hidden-state injection to obtain strong long-context conditioning and generate a whole block in one pass [14]. Follow-up methods build draft trees or improve training objectives for such parallel drafters [31], [32]. Nevertheless, DFlash-style block-parallel prediction mainly models position-wise long-context distributions and lacks explicit conditioning on the locally drafted prefix, which can hurt later-token acceptance.

2.0.0.3 Intra-block causality and parallel-drafter correction.

Several methods address the missing local causality in parallel drafting. Hydra introduces dependency among draft heads [7]; Domino corrects DFlash-style logits with a GRU causal encoder and low-rank residual head [15]; DSpark adds semi-autoregressive Markov/RNN heads and confidence scheduling [17]; JetSpec uses tree-causal attention for branch-wise causal conditioning [18]; and TreeFlash approximates local autoregressive distributions on top of DFlash [16]. D-PACE further shows that accepted length is tied to position-dependent prefix acceptance [32]. Although structured non-autoregressive decoding and CTC-based drafting can introduce dependencies or alignments [33], [34], they are less compatible with the exact per-position probabilities required by standard lossless verification. Unlike these unified correction pipelines, DeLS-Spec keeps the trained DFlash drafter fixed as a long-context expert, independently trains a lightweight short-context expert for intra-block causality, and fuses their logits with unigram-prior correction to avoid double-counting frequency bias.

3 Preliminaries↩︎

3.1 Parallel Draft Model: DFlash↩︎

Let \(y=x_{<k}\) denote the observed long context, and let \(x_k,x_{k+1},\ldots,x_{k+s}\) denote the block of future tokens to be drafted. Parallel draft models such as DFlash [14] perform block-parallel drafting by predicting multiple future tokens in a single forward pass. Specifically, for each position inside the draft block, the model predicts \[p_L(x_i \mid y), \qquad i\in\{k,\ldots,k+s\}.\] Although DFlash is described as a diffusion language model, its practical drafting procedure consists of one forward pass followed by one sampling step, which can be viewed as a diffusion language model with a single denoising step. This design significantly reduces draft latency compared with autoregressive draft models. However, because tokens within the block are predicted in parallel, each position is unaware of the tokens generated at other positions. As a result, the causal dependency inside the draft block is weak, which limits the accepted length during speculative decoding.

3.2 Introducing Intra-Block Causality↩︎

Recent approaches such as Domino [15] and DSpark [17] address this issue by introducing intra-block causal modeling on top of the parallel DFlash backbone. They attach a lightweight causal correction head after the parallel backbone and directly optimize \[\mathcal{L}_{\mathrm{cc}} = -\sum_{i=k+1}^{k+s} \log p(x_i \mid y,z_i), \qquad z_i=x_{k:i-1}.\] where \(z_i\) denotes the local prefix within the draft block. Although this causal correction improves local consistency, it is typically trained as part of the complete draft pipeline. In particular, it depends on the target model and requires joint training or fine-tuning of the parallel DFlash backbone and the causal correction head. Therefore, migrating such causal dependency mechanisms to an already trained DFlash model can be expensive, especially when the model size increases.

4 Method↩︎

4.1 Decoupled Long-Short Contexts↩︎

We propose DeLS-Spec, a decoupled long-short context modeling method for speculative drafting. The key observation is that an already trained DFlash model provides a long-context conditional distribution \(p_L(x_i\mid y)\), which captures semantic and task-level constraints from the full prefix \(y\). What is missing is the short-range causal dependency induced by the local block prefix \(z_i\). Therefore, a natural starting point is a product-of-experts [19] formulation: \[p(x_i \mid y,z_i) \approx p(x_i \mid y)\,p(x_i \mid z_i).\] In this view, the long-context expert provides global semantic constraints, while the short-context expert provides local coherence constraints within the draft block. However, this factorization is not an exact equality. More precisely, we have \[\begin{align} p(x_i \mid y,z_i) &= \frac{ p(x_i \mid y)\,p(x_i \mid z_i) }{ p(x_i) } \cdot \frac{ p(y)\,p(z_i) }{ p(y,z_i) } \cdot \frac{ p(y,z_i \mid x_i) }{ p(y \mid x_i)\,p(z_i \mid x_i) } . \end{align}\] The proof of this factorization is provided in Appendix 9. The second factor, \[\frac{p(y)\,p(z_i)}{p(y,z_i)},\] does not depend on the candidate token \(x_i\) and can therefore be absorbed into the normalization constant when applying softmax or ranking candidate tokens. Thus, the token-dependent part can be written as \[p(x_i\mid y,z_i) \propto \frac{ p(x_i \mid y)\,p(x_i \mid z_i) }{ p(x_i) } \cdot \frac{ p(y,z_i \mid x_i) }{ p(y \mid x_i)\,p(z_i \mid x_i) } .\]

This decomposition yields two important implications. First, directly multiplying \(p(x_i\mid y)\) and \(p(x_i\mid z_i)\) double-counts the unigram prior of the token \(x_i\), since both conditional distributions already contain the intrinsic frequency bias of \(x_i\). Without correction, the product would over-emphasize high-frequency tokens. Therefore, the unigram prior \(p(x_i)\) should be divided out.

Second, the residual term \[R(x_i;y,z_i) = \frac{ p(y,z_i \mid x_i) }{ p(y \mid x_i)\,p(z_i \mid x_i) }\] captures the interaction between the long context \(y\) and the local prefix \(z_i\) that cannot be modeled by two independent conditional experts. Fully modeling this residual may improve the approximation quality, but doing so requires joint access to the target model, the parallel draft backbone, and the causal correction module. This corresponds to the design philosophy of methods such as Domino and DSpark, which attempt to learn such residual interactions through end-to-end training.

In contrast, DeLS-Spec deliberately ignores this residual term in exchange for modularity and training efficiency. We approximate \[\log p(x_i\mid y,z_i) \approx \log p_L(x_i\mid y) + \log p_S(x_i\mid z_i) - \log p_P(x_i),\] where \(p_L\) is the long-context distribution provided by the trained DFlash model, \(p_S\) is an independently trained short-context model, and \(p_P\) is the unigram prior estimated from the training corpus.

4.2 Independent Short-Context Training↩︎

A key advantage of DeLS-Spec is that the short-context model can be trained completely independently. It does not require hidden states from the target model, nor does it require access to the DFlash draft model. The only requirement is that it shares the same tokenizer as the target model. In our implementation, \(p_S(x_i\mid z_i)\) is parameterized by a lightweight local RNN head and trained on plain text corpora with the standard next-token prediction objective: \[\mathcal{L}_S = -\sum_{i=k+1}^{k+s} \log p_S(x_i \mid z_i), \qquad z_i=x_{k:i-1}.\] This training objective encourages the short-context model to capture local causal dependencies inside a draft block, independent of the long-context drafting model.

4.3 Logit Fusion at Inference↩︎

During inference, DeLS-Spec is directly attached to an already trained DFlash draft model. Let \(\ell_L(x_i\mid y)\) denote the long logits produced by DFlash, \(\ell_S(x_i\mid z_i)\) denote the short logits produced by the local RNN head, and \(\ell_P(x_i)\) denote the log unigram prior estimated from the training corpus. We fuse these logits as \[\ell(x_i) = \ell_L(x_i\mid y) + \alpha\,\ell_S(x_i\mid z_i) - \beta\,\ell_P(x_i),\] where \(\alpha\) and \(\beta\) are calibration coefficients. The theoretical decomposition corresponds to \(\alpha=\beta=1\). In practice, however, the long logits, short logits, and unigram prior are obtained from different sources and may have different scales. Therefore, \(\alpha\) and \(\beta\) are introduced to calibrate their relative contributions. The unigram prior term \(\ell_P(x_i)\) is computed once from corpus statistics and introduces no additional inference-time model computation, its estimation details are provided in Appendix 8. The complete inference procedure is summarized in Figure 1 and Algorithm 2.

5 Experiments↩︎

5.1 Experimental Setup↩︎

5.1.0.1 Models and Benchmarks.

We evaluate DeLS-Spec on Qwen3-4B and Qwen3-8B [3]. For the DFlash backbone [14], we use the officially released checkpoints z-lab/Qwen3-4B-DFlash-b16 and z-lab/Qwen3-8B-DFlash-b16, both with a draft block size of 16. We evaluate on math reasoning, code generation, and open-ended dialogue benchmarks. The math benchmarks include GSM8K [35], MATH-500 [36], and AIME25 [37]; the code benchmarks include HumanEval [38], MBPP [39], and LiveCodeBench [40]; and the dialogue benchmarks include MT-Bench [41] and Alpaca [42]. We report end-to-end decoding speedup over autoregressive decoding and the average acceptance length \(\tau\).

5.1.0.2 Training Data.

We use the DFlash authors’ Qwen3-4B-Instruct-100K data [14] for the 4B setting and the Domino authors’ Qwen3-8B-ShareGPT data [15] for the 8B setting. These corpora are used both to train the local head and to estimate the unigram prior. Unlike methods that jointly fine-tune the draft pipeline, DeLS-Spec only trains the local head with the standard next-token prediction objective on the training corpus. The DFlash backbone used during evaluation is kept fixed and directly loaded from the released z-lab checkpoints.

5.1.0.3 Baselines.

We compare DeLS-Spec with autoregressive decoding and representative speculative decoding baselines, including EAGLE-3, DART, and DFlash [10], [13], [14]. DFlash is the block-parallel backbone on which DeLS-Spec is built. We also include additional comparisons with Domino-style fine-tuning and other DFlash settings where applicable.

5.1.0.4 Implementation.

Unless otherwise specified, all results are evaluated with the Hugging Face Transformers backend [43]. All training and evaluation runs are conducted on a single NVIDIA L20 GPU. The local head is implemented as a GRU-based RNN by default. For DeLS-Spec inference, we set \(\alpha=\beta=0.3\) unless otherwise stated. Following Domino, we optimize the inference implementation with CUDA Graphs and fused Triton kernels [44] to reduce the overhead of the local-head decoding loop. Detailed local-head training hyperparameters are provided in Table 4.

5.2 Main Results↩︎

Table ¿tbl:tab:main-results? presents the main results on Qwen3-4B and Qwen3-8B across math, code, and chat benchmarks. DFlash already outperforms tree-based baselines such as EAGLE-3 and DART by a large margin. Built upon this strong baseline, DeLS-Spec further improves both decoding speedup and average acceptance length \(\tau\) in almost all settings.

The gains are particularly clear on math and code tasks, where local token dependencies are stronger. For Qwen3-4B at temperature \(0\), DeLS-Spec improves the speedup of DFlash from \(4.74\times\) to \(5.02\times\) on MBPP, from \(6.09\times\) to \(6.35\times\) on MATH-500, and from \(5.69\times\) to \(5.95\times\) on AIME25. Meanwhile, the acceptance length increases by up to \(0.44\) on AIME25 and \(0.40\) on MATH-500. Under stochastic decoding, DeLS-Spec also brings notable improvements, e.g., on Qwen3-4B at temperature \(1\), HumanEval speedup increases from \(4.61\times\) to \(4.85\times\), with \(\tau\) improved from \(5.84\) to \(6.21\). Similar trends hold for Qwen3-8B, where DeLS-Spec improves \(\tau\) by \(0.33\) on HumanEval and MBPP at temperature \(0\), and by \(0.29\) on MT-Bench at temperature \(1\).

These results show that DeLS-Spec complements DFlash by improving the local consistency of draft tokens. The local head enables more drafted tokens to be accepted during verification, especially on math and code benchmarks, while preserving the original DFlash parallel drafting process.

5.3 General Adaptability of the DeLS-Spec Local Head↩︎

We further test the general adaptability of the local head on two DFlash block-7 checkpoints from DSpark [17]: deepseek-ai/dflash_qwen3_4b_block7 and deepseek-ai/dflash_qwen3_8b_block7. This setting examines whether the local head can transfer beyond our default DFlash configuration. Since the local head is trained with block size \(16\), it can be directly used with DFlash models whose block size is no larger than \(16\).

As shown in Table ¿tbl:tab:dspark-dflash-block7-results?, DeLS-Spec consistently improves these DSpark checkpoints. On the 4B checkpoint, DeLS-Spec brings speedup gains on MATH-500, AIME25, and HumanEval, improving DFlash by \(0.26\times\) on each benchmark. It also increases the acceptance length by \(0.33\) on HumanEval and \(0.31\) on MATH-500. On the 8B checkpoint, DeLS-Spec improves \(\tau\) by \(0.32\) on GSM8K and MATH-500, and by \(0.31\) on LCB, while also increasing speedup across all benchmarks.

These consistent improvements on independently released block-7 checkpoints demonstrate that the DeLS-Spec local head is not tied to a specific DFlash implementation. Instead, it can serve as a plug-in module that improves local draft-token modeling for different DFlash-style draft models.

5.4 Effect of \(\alpha\) and \(\beta\)↩︎

We further study the effect of the two combination hyperparameters, \(\alpha\) and \(\beta\), by performing a grid search on Qwen3-4B. Figure 3 summarizes the results. The left panel shows that \(\alpha\) and \(\beta\) should be increased jointly to achieve the best performance, rather than tuning either one alone. The heatmap also exhibits a clear diagonal trend, suggesting that in practice \(\alpha\) and \(\beta\) can simply be set to the same value, with values below \(0.5\) yielding the best performance. In particular, the best region lies around \(\alpha \approx 0.3\) and \(\beta \approx 0.3\), indicating that a balanced contribution from the local head and the unigram-prior correction is important.

To better highlight the importance of subtracting the unigram prior, the middle panel normalizes each row by subtracting the corresponding value at \(\beta=0\). This row-wise comparison makes the effect of \(\beta\) more explicit: once \(\alpha\) is properly set, using a positive \(\beta\) consistently improves the acceptance length over the \(\beta=0\) case. The right panel further summarizes this trend by plotting, for each \(\alpha\), the improvement of the best \(\beta\) over \(\beta=0\). We observe consistent gains across almost all \(\alpha\) values, which confirms that unigram-prior subtraction is a key component of the proposed formulation.

5.5 Impact of the Omitted Residual Term↩︎

We next quantify how much performance is lost by omitting the residual term \(R(x_i; y, z_i)\) in DeLS-Spec. Table 1 compares several variants in terms of average acceptance length \(\tau\). DeLS-Spec (Markov) follows the Markov head used in DSpark [17], where the logits of each draft token are corrected only based on the immediately preceding token. In our framework, this corresponds to generating the short-context logits using only the previous token. Domino-FT freezes the DFlash backbone and trains only the Domino head [15], using the same training data and number of epochs as our DeLS-Spec local head.

Although Domino-FT achieves the highest average \(\tau\), the gap between DeLS-Spec and Domino-FT is small. On average, DFlash obtains \(\tau=6.04\), DeLS-Spec (RNN) improves it to \(6.35\), while Domino-FT reaches \(6.45\). Thus, omitting the residual term only leads to a \(0.10\) drop in average acceptance length compared with Domino-FT, while DeLS-Spec still captures \(75.69\%\) of Domino-FT’s improvement over DFlash. The \(\tau\) gain ratio is computed as \(\frac{\tau_{\text{DeLS-Spec}}-\tau_{\text{DFlash}}}{\tau_{\text{Domino-FT}}-\tau_{\text{DFlash}}}.\)

The gains are also consistent across benchmarks. DeLS-Spec (RNN) recovers more than \(80\%\) of Domino-FT’s improvement on AIME25 and HumanEval, and even surpasses Domino-FT on LCB with a \(\tau\) gain ratio of \(105.56\%\). In addition, DeLS-Spec (RNN) consistently outperforms DeLS-Spec (Markov), improving the average \(\tau\) from \(6.28\) to \(6.35\). This shows that modeling richer short-range dependencies with an RNN local head is more effective than a one-step Markov correction, while still avoiding the full residual modeling cost of Domino-FT.

5.6 Do Learnable \(\alpha\) and \(\beta\) Help?↩︎

We further investigate whether learning the fusion weights \(\alpha\) and \(\beta\) can bring additional gains over fixed values. As shown in Table 2, \(\alpha=\beta=0\) corresponds to the original DFlash baseline, while \(\alpha=\beta=0.3\) is our default setting. For the learnable variant, we initialize \(\alpha\) and \(\beta\) to \(0.3\) and optimize them for 10K steps on the training data. During this process, the target model, DFlash, and the local head are all loaded but frozen, and only \(\alpha\) and \(\beta\) are updated with the cross-entropy loss.

The learnable variant consistently improves over the DFlash baseline, increasing the average acceptance length from \(6.33\) to \(6.55\). This confirms that assigning positive weights to the local correction and unigram-prior subtraction is beneficial. However, it is still worse than our simple fixed setting \(\alpha=\beta=0.3\), which achieves the best average \(\tau\) of \(6.65\) and outperforms the learnable variant on every benchmark. For example, the fixed setting improves \(\tau\) over the learnable variant by \(0.20\) on MATH-500, \(0.13\) on GSM8K and MBPP, and \(0.06\) on LCB.

Appendix 10 reports the learned values of \(\alpha\) and \(\beta\) in Table 5. They are consistently positive and increase with draft-token position, which supports the usefulness of both terms. Nevertheless, directly optimizing these weights with cross-entropy does not translate to better acceptance length on downstream benchmarks. This suggests that the fixed \(\alpha=\beta=0.3\) setting provides a better trade-off between effectiveness, robustness, and simplicity, avoiding the need to load all components and perform extra tuning.

5.7 Training Cost Comparison↩︎

We compare the training cost of Domino-FT and DeLS-Spec on a single NVIDIA L20 GPU with 48GB memory. As shown in Table 3, all methods are trained with batch size \(1\), accumulation steps \(4\), and one epoch. For Domino-FT and DeLS-Spec (RNN), we sample 256 anchors from each example to construct draft blocks. In contrast, DeLS-Spec (Markov) does not rely on anchors or block construction, and is trained with dense token-level supervision directly on the corpus.

10.0pt

DeLS-Spec is substantially more efficient than Domino-FT. On Qwen3-4B, Domino-FT requires 13.4 hours and 42.6GB VRAM, while DeLS-Spec (RNN) only takes 1.1 hours and 9.0GB VRAM, reducing training time by over \(12\times\) and memory usage by about \(4.7\times\). The Markov variant is even cheaper, requiring only 0.4 hours and 6.5GB VRAM. The gap becomes more pronounced on Qwen3-8B: Domino-FT runs out of memory on a 48GB L20 GPU, whereas DeLS-Spec (RNN) still fits comfortably with 10.1GB VRAM and finishes training in 1.1 hours. DeLS-Spec (Markov) further reduces the cost to 0.5 hours and 5.9GB VRAM.

These results highlight the practical advantage of DeLS-Spec. The key reason is that the DeLS-Spec local head can be trained independently of both the target model and DFlash. In contrast, Domino-FT requires the target model to produce hidden states and DFlash to provide base logits, while the Domino head only learns the residual correction logits. As a result, Domino-FT must keep multiple large components involved during training, leading to much higher memory usage and training time. DeLS-Spec avoids this dependency by training a lightweight local head separately, making it significantly easier to apply under limited hardware resources.

6 Conclusion↩︎

We presented DeLS-Spec, a decoupled long-short context method for block-parallel speculative decoding. DeLS-Spec keeps DFlash fixed as a long-context expert and adds a lightweight local head as a short-context expert. By combining their logits at inference time, it introduces intra-block causal information without retraining the draft model from scratch. The local head can be independently trained with a standard next-token prediction objective, making training cheap and modular. Since it is not one-to-one tied to a specific DFlash checkpoint, it can also be flexibly attached to compatible DFlash-style drafters. Experiments on Qwen3 models show that DeLS-Spec consistently improves speedup and average acceptance length over DFlash, demonstrating an efficient way to enhance existing block-parallel speculative decoding systems.

7 Limitations and Future Work↩︎

While DeLS-Spec demonstrates significant efficiency and performance gains, our approach has several limitations that present opportunities for future research:

Omission of Residual Interactions. To achieve extreme training efficiency and modularity, DeLS-Spec deliberately omits the residual interaction term between the long context and the local short context. As a result, while it recovers the majority of performance improvements, its average acceptance length is slightly bounded and falls marginally short of end-to-end jointly trained methods like Domino, which explicitly learn this full residual interaction.

Generalization to Other Parallel Drafters. Our current empirical evaluation is primarily built upon the DFlash architecture. However, the theoretical framework of DeLS-Spec is fundamentally agnostic to the underlying draft model. Since the local head operates independently, future work will explore applying and validating DeLS-Spec on a broader range of non-autoregressive and parallel draft models beyond DFlash.

Adaptive Logit Fusion Strategies. Currently, DeLS-Spec employs a straightforward logit fusion mechanism with fixed hyperparameters (\(\alpha\) and \(\beta\)). We believe the fusion process can be further optimized. Future research could explore dynamic fusion strategies, such as adaptively adjusting the local head and unigram-prior weights based on the token-level entropy or confidence scores of the long-context logits, allowing for a more context-aware balance between global semantics and local causality.

Mitigating Exposure Mismatch in Local Heads. As observed in our analysis, directly learning the fusion weights (\(\alpha\) and \(\beta\)) via cross-entropy yields suboptimal inference performance compared to using fixed values. We attribute this to an exposure mismatch (or teacher-forcing bias) during training, where the local head is conditioned on ground-truth prefixes rather than its own drafted tokens. Future work could investigate techniques such as scheduled sampling or alignment-based tuning to bridge this train-test gap, potentially unlocking the full capability of learnable fusion parameters.

8 Implementation Details↩︎

8.0.0.1 Local Head Implementation.

We instantiate \(p_S(x_i\mid z_i)\) with a lightweight local head over the local draft prefix. Since the local head is evaluated autoregressively during inference, we use a low-rank bottleneck to reduce the sequential computation cost, following the design choice in Domino and DSpark [15], [17]. For the RNN variant, token ids are mapped by the frozen target embedding table and passed through a one-layer bias-free GRU; the final hidden state is projected to a low-rank state with SiLU activation and then mapped to vocabulary logits: \[r_i=\mathrm{SiLU}\!\left(W_{\mathrm{rank}} \mathrm{GRU}(E_{\mathrm{tgt}}(z_i))\right), \qquad \ell_S^{\mathrm{RNN}}(x_i\mid z_i)=W_{\mathrm{vocab}}r_i .\] For the Markov variant, we directly look up a low-rank state from the most recent token and multiply it with a low-rank vocabulary head: \[r_i=E_M(x_{i-1}), \qquad \ell_S^{\mathrm{Markov}}(x_i\mid z_i)=r_i W_M^\top ,\] where \(E_M,W_M\in\mathbb{R}^{|\mathcal{V}|\times d_r}\). Both variants output full-vocabulary logits and are trained with the same next-token prediction objective as the short-context model. The hyperparameters for the local head are summarized in Table 4.

8.0.0.2 Unigram Prior Estimation.

We construct a fixed unigram log-prior over target tokens using the same tokenizer and chat-template preprocessing as training. For each training sequence, we count only tokens whose loss mask equals one, i.e., assistant-response tokens that contribute to the supervised objective. Let \(c(v)\) denote the resulting count for vocabulary item \(v\). We form a smoothed prior \[\tilde{p}_0(v) = \frac{c(v)+\lambda}{\sum_{v'}(c(v')+\lambda)},\] where we use additive smoothing with \(\lambda=1.0\) by default. We then use \(b_0(v)=\log \tilde{p}_0(v)\) as a fixed vocabulary-level bias.

9 Proof of the Product-of-Experts Factorization↩︎

In Section 4.1, we introduce the exact factorization of the conditional probability \(p(x_i \mid y, z_i)\). Here, we provide the step-by-step derivation for this decomposition.

Starting from Bayes’ theorem, the conditional probability of the candidate token \(x_i\) given the long context \(y\) and the short context \(z_i\) can be written as: \[p(x_i \mid y, z_i) = \frac{p(y, z_i \mid x_i)\,p(x_i)}{p(y, z_i)}.\]

We can multiply both the numerator and the denominator by the product of the marginals \(p(y)\,p(z_i)\) and the conditional probabilities \(p(y \mid x_i)\,p(z_i \mid x_i)\): \[p(x_i \mid y, z_i) = \frac{p(y, z_i \mid x_i)\,p(x_i)}{p(y, z_i)} \cdot \frac{p(y \mid x_i)\,p(z_i \mid x_i)}{p(y \mid x_i)\,p(z_i \mid x_i)} \cdot \frac{p(y)\,p(z_i)}{p(y)\,p(z_i)}.\]

By rearranging the terms, we group them into three distinct factors: \[p(x_i \mid y, z_i) = \left[ \frac{p(y \mid x_i)\,p(z_i \mid x_i)\,p(x_i)}{p(y)\,p(z_i)} \right] \cdot \frac{p(y)\,p(z_i)}{p(y, z_i)} \cdot \frac{p(y, z_i \mid x_i)}{p(y \mid x_i)\,p(z_i \mid x_i)}. \label{eq:proof95rearranged}\tag{1}\]

Recall that by applying Bayes’ theorem to the individual contexts, we can express the conditional probabilities of \(x_i\) as: \[p(x_i \mid y) = \frac{p(y \mid x_i)\,p(x_i)}{p(y)} \quad \text{and} \quad p(x_i \mid z_i) = \frac{p(z_i \mid x_i)\,p(x_i)}{p(z_i)}.\]

Multiplying these two equations together and dividing by the unigram prior \(p(x_i)\) yields the exact expression inside the square brackets of Equation 1 : \[\frac{p(x_i \mid y)\,p(x_i \mid z_i)}{p(x_i)} = \frac{p(y \mid x_i)\,p(z_i \mid x_i)\,p(x_i)}{p(y)\,p(z_i)}.\]

Substituting this equivalence back into Equation 1 gives the final exact equality: \[\begin{align} p(x_i \mid y, z_i) &= \frac{ p(x_i \mid y)\,p(x_i \mid z_i) }{ p(x_i) } \cdot \frac{ p(y)\,p(z_i) }{ p(y, z_i) } \cdot \frac{ p(y, z_i \mid x_i) }{ p(y \mid x_i)\,p(z_i \mid x_i) } . \end{align}\] This completes the proof.

10 Analysis of Learnable \(\alpha\) and \(\beta\)↩︎

Table 5 reports the learned values of \(\alpha\) and \(\beta\) at different draft-token positions. We observe that both weights become relatively large at later positions. For example, \(\alpha\) increases from \(0.547\) at position \(2\) to around \(0.762\) after position \(12\), while \(\beta\) also increases from \(0.512\) to around \(0.613\). This trend suggests that the learned fusion weights assign increasingly higher importance to the local head and the unigram-prior correction for later draft tokens.

A possible explanation is the mismatch introduced by teacher forcing during training [45]. When optimizing \(\alpha\) and \(\beta\) on the training data, later positions are conditioned on longer ground-truth prefixes. Under this setting, the local head can exploit more accurate short-context information and may appear more reliable than DFlash for predicting later draft tokens. As a result, the optimization tends to assign larger \(\alpha\) and \(\beta\) values to later positions.

However, this advantage may not fully transfer to inference. During speculative decoding, later draft tokens are conditioned on previously drafted tokens rather than ground-truth tokens, which introduces exposure bias. Therefore, overly large learned weights at later positions may overestimate the reliability of the local correction and lead to suboptimal acceptance length. This explains why the learnable variant in Table 2 improves over the DFlash baseline but still underperforms the fixed setting \(\alpha=\beta=0.3\). The fixed setting is less sensitive to the teacher-forcing bias and provides a more robust choice for inference.

11 Comparison of Acceptance Rate Results↩︎

Table 6 provides a position-wise view of the acceptance behavior on Qwen3-4B at temperature \(0\). Unlike the average acceptance length \(\tau\) reported in Table ¿tbl:tab:main-results?, the quantity \(rate_i=P(\mathrm{acceptance\;length}\ge i)\) measures the probability that verification accepts at least the first \(i\) draft tokens. It therefore describes the tail of the accepted draft length distribution: improvements at larger positions indicate that the method more often preserves longer consecutive draft spans, which is the direct source of higher average acceptance length and decoding speedup.

The results show that DeLS-Spec improves the acceptance curve over DFlash across almost all nontrivial positions. The first position is accepted by both methods by definition, and the second position is nearly unchanged (\(81.45\%\) versus \(81.63\%\)). From position \(3\) onward, however, DeLS-Spec consistently achieves higher acceptance rates. The gains are largest in the middle of the draft block, reaching about \(2.6\) percentage points around positions \(5\)–\(6\), and remain positive even at the end of the block, with a \(1.40\) percentage-point improvement at position \(16\). This pattern supports the main hypothesis of DeLS-Spec: the local head and unigram-prior correction do not merely improve isolated early predictions, but make the draft sequence locally more coherent so that verification can accept longer prefixes.

References↩︎

---

1. Corresponding author↩︎