1 Introduction↩︎

Retrieval-augmented generation (RAG) has become an essential paradigm for grounding large language models (LLMs) in external knowledge [@gao2023retrieval], enabling them to answer domain-specific and knowledge-intensive queries beyond what is stored in their parameters. Initially developed for open-domain question answering (QA) [@karpukhin-etal-2020-dense; @lewis2020retrieval], RAG has become an integral component of workflows that require resolving information from multiple sources, from open-domain deep-research agents that synthesize evidence across the web [@du2026deepresearch; @java2026characterizing] to enterprise systems that retrieve from heterogeneous data such as reports, emails, and chats [@choubey-etal-2025-benchmarking; @abaskohi2026drbench].

Despite these advancements, a central challenge for RAG systems is the resolution of conflicts among the knowledge sources [@xu-etal-2024-knowledge-conflicts]. At inference time, the LLM can be overly reliant on its internal memory acquired during pre-training, even when the retrieved context provides newer or more task-specific evidence [@chen-etal-2022-rich; @xie2024adaptive]. One popular training-free approach to context-memory conflict is contrastive decoding (CD) [@li-etal-2023-contrastive], which modulates generation by contrasting model predictions under different conditioning contexts [@shi-etal-2024-trusting; @wang-etal-2025-adacad; @khandelwal-etal-2025-cocoa]. Rather than fine-tuning the LLMs, contrastive decoding adjusts token probabilities based on the difference between output distributions with and without the context. However, this formulation treats the retrieved evidence as a single context and does not resolve conflicts within the retrieved context itself.

In multi-document QA, the model needs to decide the relevance of each document to the query and which evidence to use [@wan-etal-2024-evidence; @jin-etal-2024-tug]. The retrieved evidence may contain answer-supporting evidence alongside irrelevant distractors, stale information, and plausible but incorrect claims. In this setting, the decoding problem is not only about choosing the next token, but also about choosing the evidence source that should inform that token. We use confidence as the complement of uncertainty and distinguish two levels: document-level confidence, which estimates whether a retrieved document appears sufficient for answering the question, and token-level confidence, which estimates whether a document-conditioned stream yields a sharp local next-token distribution. Token-level confidence has been used to capture how confidently the model predicts under a given context [@malinin2021uncertainty; @jiang-etal-2023-active; @fadeeva-etal-2024-fact; @nguyen2026probabilities], but it cannot determine whether the context itself is sufficient to answer the question. Document-level confidence addresses the complementary question of which retrieved source contains answer-bearing evidence [@kadavath2022language; @singh-etal-2023-tree; @wan-etal-2024-evidence; @joren2025sufficient], but it does not ensure that the model can generate from that source with confidence. Effective multi-document RAG therefore requires both source selection and generation confidence.

Most existing RAG and conflict-resolution benchmarks do not directly isolate this joint capability. For example, retrieval-augmented QA benchmarks [@joshi-etal-2017-triviaqa; @kwiatkowski-etal-2019-natural; @mallen-etal-2023-trust; @zhang-etal-2024-retrievalqa] mainly test whether models can ground answers in retrieved context despite noisy retrievals, while conflict-resolution benchmarks [@leeambigdocs; @wangretrieval] introduce contradictory evidence through altered public facts or ambiguous entity references, such that resolving the conflict may depend on the model’s internal world knowledge rather than on confidence over the retrieved evidence itself. This challenge is especially pronounced in enterprise settings, where relevant facts are often private or organization-specific, and retrieved evidence can be heterogeneous, time-sensitive, and internally conflicting [@choubey-etal-2025-benchmarking; @abaskohi2026drbench].

Motivated by these gaps, we introduce an enterprise benchmark for multi-document question answering and propose a novel decoding method. To address the evaluation gap, we propose DRQA, a factual-conflict QA benchmark derived from the recent DRBench [@abaskohi2026drbench] consisting of synthetic enterprise-specific facts grounded in company profiles, user personas, and deep-research tasks, where the answer cannot be inferred from the model’s internal memory. Instead, each answer must be recovered from a retrieved bundle containing correct, temporal, misinformation, and noise documents, requiring models to identify the evidence source that supports the answer. To address the decoding gap, we propose Dual-Confidence Contrastive Decoding (DCCD), a training-free contrastive decoding method that uses dual-confidence signals: document-level confidence estimates whether a document appears sufficient for answering the question, while token-level confidence estimates whether the model can produce a sharp next-token prediction from that document. DCCD uses these signals to select positive and negative document-conditioned streams and scales the contrastive term by their confidence margin, amplifying high-confidence evidence while suppressing conflicting or unsupported alternatives. Across multiple RAG benchmarks, DCCD achieves the best average performance, with the largest gains on DRQA.

We summarize our contributions as follows. (1) We introduce DRQA, a factual-conflict QA benchmark derived from enterprise deep-research scenarios, where each query is paired with a retrieved bundle containing one correct document and distractors that are misinformation, stale temporal, or noise. (2) We propose Dual-Confidence Contrastive Decoding (DCCD), a training-free contrastive decoding method that combines document-level answerability confidence with token-level generation confidence to perform confidence-gated document-level contrast. (3) We show that DCCD improves over full-context and contrastive decoding baselines across standard multi-document QA benchmarks, with especially large gains on DRQA, highlighting the importance of source-aware contrast when retrieved evidence is internally conflicting.

2 Related Work↩︎

2.1 Retrieval-Augmented Question Answering↩︎

Despite their capabilities, LLMs remain prone to hallucination and are limited by the knowledge acquired during training, making it necessary to ground generation in verifiable external evidence. Retrieval-augmented generation (RAG) dynamically incorporates information through dense passage retrieval [@karpukhin-etal-2020-dense], and has been shown to improve open-domain question answering when the model’s internal knowledge is insufficient [@lewis2020retrieval]. Consequently, models are typically evaluated on open-domain QA benchmarks on their ability to answer factual questions using retrieved passages from public corpora. For example, [@izacard-grave-2021-leveraging] use real Google queries and Wikipedia answers to evaluate the ability of the model to ground responses in retrieved evidence, PopQA [@mallen-etal-2023-trust] further probes the boundary between internal and retrieved knowledge by focusing on long-tail facts for which retrieval is often necessary, while RetrievalQA [@zhang-etal-2024-retrievalqa] evaluates adaptive RAG behavior under retrieved contexts of varying usefulness.

Since knowledge conflict can also arise within the retrieved context itself [@xu-etal-2024-knowledge-conflicts], conflict-oriented benchmarks have been proposed to evaluate the models’ ability to arbitrate among conflicting evidence. However, existing benchmarks are constructed by altering public facts or referencing ambiguous entities [@leeambigdocs; @wangretrieval]. In these settings, models may still be able to resolve the conflict using internal world knowledge (e.g., recognizing the true public fact or the intended entity) rather than by reasoning over the retrieved text itself. In parallel, deep-research benchmarks evaluate broader systems that gather and synthesize evidence across many sources, including open-domain web research and enterprise settings [@du2026deepresearch; @java2026characterizing; @abaskohi2026drbench]. Our proposed DRQA dataset fills this gap by combining conflict-oriented factual QA with enterprise deep-research scenarios, where the answer is unlikely to be available from internal memory and must instead be recovered by identifying which retrieved document supports the answer.

2.2 Contrastive Decoding↩︎

Contrastive decoding (CD) [@li-etal-2023-contrastive] was originally proposed as a training-free strategy that selects tokens with high expert-model probability and low amateur-model probability, using their probability discrepancy to favor fluent and informative continuations. Context-Aware Decoding (CAD) [@shi-etal-2024-trusting] adapts this idea to improve context faithfulness in RAG by contrasting the LLM with and without retrieved context, so that generation follows retrieved evidence rather than prior knowledge. Further work makes the contrastive weight adaptive [@zhao-etal-2024-enhancing; @kim-etal-2024-adaptive]. For example, AdaCAD [@wang-etal-2025-adacad] adjusts the contrastive weight at each decoding step using the Jensen-Shannon divergence (JSD) between contextual and zero-shot distributions, while CoCoA [@khandelwal-etal-2025-cocoa] introduces token-level confidence-aware adaptation to address conflict between contextual and internal knowledge. Most similar to our work is Dynamic Contrastive Decoding (DVD) proposed by [@jin-etal-2024-dvd], which targets multi-document QA by contrasting document-conditioned predictions and dynamically increasing the influence of documents estimated to be useful for the current generation step. However, DVD does not explicitly distinguish document-level support from token-level generation confidence.

2.3 Confidence Estimation↩︎

Current work on confidence estimation has primarily focused on confidence in the model’s output distribution [@geng-etal-2024-survey]. Entropy-based methods estimate predictive uncertainty for autoregressive generation [@malinin2021uncertainty; @jiang-etal-2023-active], and token-level uncertainty has been used for hallucination detection and fact checking [@fadeeva-etal-2024-fact]. Recent probability-only methods motivate lightweight top-\(k\) token statistics for uncertainty estimation [@nguyen2026probabilities], where our token-level confidence follows the Dirichlet uncertainty framework of [@malinin-gales-2018-predictive] by mapping the top-\(k\) logits of each stream to Dirichlet concentration parameters and computing the expected entropy. In parallel, document-level confidence addresses the complementary problem of whether an evidence source is sufficient to answer the question. Prior work has studied the models’ ability to self-evaluate whether they know an answer [@kadavath2022language], identify convincing evidence under conflicting sources [@wan-etal-2024-evidence], and estimate the sufficiency of sources in retrieved evidence [@joren2025sufficient; @hwang-etal-2025-retrieval]. The closest work to our document-level confidence is context-sufficiency estimation [@joren2025sufficient], which asks whether the provided context contains enough information to answer a question. In this work, our proposed DCCD combines document-level and token-level confidence estimates inside the contrastive decoding process.

3 DRQA: Deep Research Question Answering Benchmark↩︎

We derive DRQA from DRBench [@abaskohi2026drbench], a 100-task enterprise deep-research benchmark spanning three industries—retail, healthcare, and electric vehicles—and ten domains. DRBench contains internal and external supporting facts embedded in generated enterprise documents verified by human annotators. We use the 568 synthetically generated internal facts, which are designed not to appear on the public web and therefore are not expected to be recoverable from the model’s internal memory. Additional construction details and prompts are provided in 7.

3.1 Dataset Construction↩︎

Each DRBench task contains internal insights generated from the organization’s industry, priorities, customer segments, and business goals. Since we focus on retrieval-augmented question answering rather than full deep-research report generation, we convert each internal insight into a QA pair. For example, the insight “Lee’s Market reduced food waste by 8% in Q2 2024, saving $1.2M” becomes the question “What food-waste reduction rate did Lee’s Market achieve in Q2 2024?”

For each QA pair, we construct an evidence set with four document types: correct, misinformation, temporal, and noise. Correct documents contain the ground-truth answer, misinformation documents contain a plausible but incorrect answer, temporal documents contain an answer that was valid at an earlier document date, and noise documents are topically plausible but do not answer the question. Each document is assigned a document date in ISO 8601 format and an original source type such as email, chat, PDF, DOCX, or slide deck.

DRBench generates insight files through a three-step needle-in-a-haystack procedure [@abaskohi2026drbench] by first producing a document outline in a randomly assigned source type, and then injecting the ground-truth insight into an appropriate section, before finally filling the remaining sections with contextually plausible but topically irrelevant text. We reuse the DRBench insight and distractor files as correct and noise documents. Since we evaluate model reasoning rather than file extraction, we serialize documents as Markdown and JSONL formats for model input.

3.1.0.1 Misinformation

To construct misinformation documents, we prompt the LLM to generate a plausible perturbation of the ground-truth insight and then use the same three-step DRBench document-generation procedure. To make these documents realistic but not explicitly labeled, we add subtle format-specific uncertainty markers, such as conversational hedges in chat or tentative phrasing in reports. These markers reflect naturally occurring epistemic language in enterprise communication, but do not identify the document as incorrect. Thus, models must rely on weak cues, source metadata, and context rather than explicit correctness labels.

3.1.0.2 Temporal

Temporal documents are generated to read as authoritative at the time of writing, rather than as retrospective historical summaries. We first sample the document_date uniformly between 6 months and 3 years before the ground-truth document date, then prompt the LLM to generate a value that would have been accurate and current as of that date. A second call produces the full document in present tense using the same DRBench document-generation procedure.

We ensure that temporal documents contain no explicit “archived” or “outdated” labels, where document_date is the sole staleness signal. The direction of change is unconstrained, since the conflict should be resolved by comparing dates rather than by judging whether a trend is intuitive. When serialized for model input, each document is preceded by its document date and source format.

3.2 Quality Verification↩︎

Complementing DRBench’s per-file human spot-check protocol, we apply an automated LLM-as-a-judge quality filter to all generated conflict documents. Each document type is checked against a fixed rubric that returns a structured JSON binary verdict. For misinformation documents, we verify that the claimed answer is factually and specifically different from the ground truth while remaining plausible to an uninformed reader. For temporal documents, we verify that the historical answer is distinct from the current ground truth and that the document contains no explicit staleness labels. We manually inspect and fix failures, then rerun the filter until all criteria are met.

Unlike standard open-domain QA benchmarks, where retrieval is usually expected to help, DRQA creates a setting in which retrieval is necessary but not sufficient. The answers are synthetic private facts, and each retrieved bundle contains both the correct answer and plausible contradictions. The evaluation question is therefore not only whether the model benefits from retrieval, but which value it commits to when confronted with conflicting and noisy evidence. DRQA directly evaluates retrieval-grounded conflict resolution, a capability that remains underexplored in prior work.

4 Methodology↩︎

In this section, we describe our proposed Dual-Confidence Contrastive Decoding (DCCD) with an overview diagram illustrated in 1. Additional derivations and implementation details of DCCD are presented in 8.

4.1 Problem Setup↩︎

Let \(x\) denote the question and let \(\mathcal{D}=\{d_1,\ldots,d_n\}\) be the list of documents returned by the retriever. The goal is to generate an accurate and precise answer to \(x\) using \(\mathcal{D}\). At each decoding step \(t\), the model produces next-token logits \(\mathbf{z}_t\) conditioned on the current prefix \(y_{<t}\), and the decoding strategy selects the next token \(y_t\).

4.2 Document-Level Confidence↩︎

In multi-document RAG, the retrieved set \(\mathcal{D}\) can contain documents with different levels of relevance and evidential value. While retrieval similarity scores provide a useful filter, they are optimized for query-document relevance rather than for whether a document contains enough information to answer the question. Motivated by prior work studying whether the model knows the answer [@kadavath2022language] and the sufficiency of a retrieved document for answering the query \(x\) [@wan-etal-2024-evidence; @joren2025sufficient], we first estimate a document-level confidence score for each question-document pair \((x, d_i)\).

Specifically, we construct a support-probe prompt \(\pi_{\mathrm{sup}}(x,d_i)\) that asks the model whether \(d_i\) contains enough information to answer \(x\). The probe is formulated as a binary yes/no question. Let \(\mathcal{Y}_{+}\) be the set of yes verbalizer tokens and \(\mathcal{Y}_{-}\) be the set of no verbalizer tokens, including common case and leading-space variants. We use the model’s next-token distribution after the support probe, and define the document-level confidence score \(q_i\) as in 1 .

\[q_i = \frac{ \sum_{v\in\mathcal{Y}_{+}} p_\theta(v\mid \pi_{\mathrm{sup}}(x,d_i)) }{ \sum_{v\in\mathcal{Y}_{+}\cup\mathcal{Y}_{-}} p_\theta(v\mid \pi_{\mathrm{sup}}(x,d_i)) }. \label{eq:q-answerability}\tag{1}\]

Document-level confidence \(q_i\) is computed once before decoding and estimates whether a retrieved document \(d_i\) should be considered a candidate evidence source. Importantly, the support probe requires only a single forward pass per document, and adds a one-time preprocessing cost that is amortized over the answer-generation steps.

4.3 Token-Level Confidence↩︎

Document-level confidence estimates whether a retrieved document is sufficient for answering the question, but it does not indicate whether the model can use that document to make a confident prediction at a particular decoding step. We therefore compute a token-level confidence score for each document-conditioned generation stream.

At decoding step \(t\), let \(\mathbf{z}^i_t \in \mathbb{R}^{V}\) denote the next-token logits produced by the model conditioned on the question \(x\), document \(d_i\), and the current prefix \(y_{<t}\). Following the Dirichlet uncertainty framework of [@malinin-gales-2018-predictive], we approximate predictive uncertainty by treating the top-\(k\) logits of each document-conditioned stream as evidence for a Dirichlet distribution over next-token categorical distributions. This gives a lightweight, training-free confidence estimate based only on the most relevant next-token alternatives [@nguyen2026probabilities].

Let \(\mathcal{T}_{(i,t)}\) be the set of top-\(k\) token indices under \(\mathbf{z}^i_t\). For each \(v\in\mathcal{T}_{i,t}\), we define the Dirichlet concentration parameter: \[\alpha_{(i,t,v)} = \operatorname{softplus}\!\left(\mathbf{z}^i_t[v]\right)+1. \label{eq:token-alpha}\tag{2}\] Let: \[S_{(i,t)} = \sum_{v\in\mathcal{T}_{(i,t)}} \alpha_{(i,t,v)},~~~ \bar p_{(i,t,v)} = \frac{\alpha_{(i,t,v)}}{S_{(i,t)}} \label{eq:token-dirichlet-mean}\tag{3}\] We measure raw top-\(k\) uncertainty as the expected entropy of the categorical distribution induced by this Dirichlet posterior: \[\widetilde{H}_k(\mathbf{z}^i_t) = \sum_{v\in\mathcal{T}_{i,t}} \bar p_{i,t,v} \bigl[ \psi(S_{i,t}+1) - \psi(\alpha_{i,t,v}+1) \bigr] \label{eq:token-raw-entropy}\tag{4}\] where \(\psi(u)=\frac{d}{du}\log\Gamma(u)\) is the digamma function. We normalize this value by the maximum entropy over a top-\(k\) support, \[H_k(\mathbf{z}^i_t) = \frac{ \widetilde{H}_k(\mathbf{z}^i_t) }{ \log k }, \label{eq:token-normalized-entropy}\tag{5}\] and define token-level confidence as \[c_{(i,t)} = 1-H_k(\mathbf{z}^i_t). \label{eq:token-confidence}\tag{6}\]

Thus, \(c_{(i,t)}\) is high when document \(d_i\) yields a sharp next-token prediction under the current prefix, and low when its evidence is spread across many plausible next tokens. Unlike \(q_i\), which is computed once before decoding, \(c_{(i,t)}\) is recomputed at every decoding step and captures whether a candidate evidence source supports a clear local continuation.

4.4 Dual-Confidence Contrastive Decoding↩︎

At each decoding step, DCCD combines document-level and token-level confidence to choose the document-conditioned streams that define the contrastive direction. For each document \(d_i\), we define the dual-confidence score as the sum of the two terms: \[s_{(i,t)} = q_i + c_{(i,t)}. \label{eq:dual-confidence}\tag{7}\] Because both terms are normalized to \([0,1]\), this unweighted sum provides a simple hyperparameter-free aggregation of source sufficiency and local generation confidence, where weighted extensions are left as future work.

From 7 , we select the positive and negative document-conditioned streams based on the highest and lowest dual-confidence scores at each decoding step: \[i_t^+ = \arg\max_{i} s_{(i,t)}, \qquad i_t^- = \arg\min_i s_{(i,t)}. \label{eq:document-selection}\tag{8}\] We use the margin between the positive and negative scores to control the strength of the contrastive term: \[\epsilon_t = s_{(i_t^+,t)} - s_{(i_t^-,t)}. \label{eq:epsilon}\tag{9}\]

Finally, let \(\mathbf{z}^{\mathrm{full}}_t\) denote the next-token logits conditioned on the question with the full retrieved document set \(\mathcal{D}\) and the current output prefix \(y_{<t}\). Let \(\mathbf{z}^{i}_t\) denote the next-token logits conditioned on the question, document \(d_i\), and the same prefix. The decoding rule is then: \[\mathbf{z}_t = \mathbf{z}^{\mathrm{full}}_t + \epsilon_t \left( \mathbf{z}^{i_t^+}_t - \mathbf{z}^{i_t^-}_t \right). \label{eq:dccd-rule}\tag{10}\] This rule anchors generation in the full retrieved context by adding a document-level contrast that amplifies evidence from the most confident source and suppresses evidence from the least confident source at each decoding step.

5 Experiments↩︎

5.1 Settings↩︎

5.1.0.1 Datasets

In addition to DRQA, we evaluate on well-known multi-document question answering benchmarks, including Natural Questions (NQ) [@kwiatkowski-etal-2019-natural] and TriviaQA [@joshi-etal-2017-triviaqa], using the preprocessing of [@izacard-grave-2021-leveraging]. We also evaluate on the more recent PopQA [@mallen-etal-2023-trust] and RetrievalQA [@zhang-etal-2024-retrievalqa]. For DRQA, since the ground-truth answer is expressed as free-form text, we use LLM-as-a-judge to evaluate semantic correctness against the ground-truth insight. For all other datasets, we use normalized exact string match (str-em) following prior work.

5.1.0.2 Baselines

We compare DCCD against standard full-context decoding (full) and contrastive decoding baselines. In particular, we include the standard CAD [@shi-etal-2024-trusting], DVD [@jin-etal-2024-dvd], AdaCAD [@wang-etal-2025-adacad] and CoCoA [@khandelwal-etal-2025-cocoa]. We also include the zero-shot baseline without any addition evidence for reference. Additional details and hyperparameters setting of all decoding methods are described in 9.

5.1.0.3 Models

Motivated by recent trend on deploying Small Language Models (SLMs) for deep-research agents [@zheng-etal-2025-deepresearcher; @wan2025pokeeresearch], we evaluate all decoding strategies on instruction-tuned SLMs of small to medium sizes. Specifically, we use Qwen3.5-2B, Qwen3.5-9B [@qwen35blog], and Phi-3-medium (14B) [@abdin2024phi] provided by Hugging Face’s transformers library [@wolf-etal-2020-transformers]. For all models, we use the same retrieval inputs and the default chat template used for instruction-training.

5.2 Results↩︎

As shown in 1, DCCD achieves the best average performance across nearly all settings. The gains are consistent across model sizes and datasets, and are especially pronounced on DRQA. We attribute these improvements to the dual-confidence design, where document-level confidence helps identify documents that appear sufficient for answering the question, while token-level confidence identifies document-conditioned streams that support sharp local predictions. These two signals address complementary failure modes. A document may appear answerable but still yield an uncertain next-token distribution, while a document may yield a sharp prediction despite supporting an incorrect or misleading answer.

On DRQA, zero-shot accuracy is \(0\%\) for all evaluated models, which is consistent with the benchmark design. Results on DRQA also show substantially lower accuracies than the Wikipedia-style datasets, as the retrieved evidence contains the correct document alongside noise, misinformation, and temporally stale evidence. We further observe an accuracy drop from @5 to @10, since each example contains only one current ground-truth document, additional distractors increase the chance that decoding follows a misleading source. In this conflict-heavy setting, DCCD achieves larger gains over baselines than on NQ, PopQA, RetrievalQA, and TriviaQA. This trend is consistent with the dual-confidence design, which favors document-conditioned streams that are both answerable and locally confident while contrasting against unsupported, stale, or misleading alternatives.

5.3 Ablation Study↩︎

We ablate the main components of DCCD to understand where the performance gain come from. Specifically, we run Qwen3.5-2B with @5 and @10 on all 5 datasets, and consider the following four settings. (1) We set gate \(\epsilon_t=1\), removing the adaptive gate during decoding (Fixed Gate). (2) We remove the document-level confidence probe, and select contrastive pairs with only token-level confidence (Token-Only). (3) We remove the token-level confidence probe, and select contrastive pairs with only document-level confidence (Doc-Only). (4) We randomly select the positive and negative by uniformly sampling from the retrieved documents (Random).

The ablation results in ¿tbl:tab:ablation? show that DCCD benefits most from combining document-level and token-level confidence for source selection. Removing the document-level confidence probe and relying only on token-level confidence produces the largest degradation, especially on DRQA, with drops of \(-6.34\) at @5 and \(-4.58\) at @10. This suggests that token-level sharpness alone is insufficient in conflict-heavy settings, where a misleading document can still yield a confident local prediction. Randomly selecting the positive and negative documents also hurts performance in most settings, confirming that confidence-based source selection is important for identifying useful contrastive directions.

By contrast, using document-level confidence alone is comparatively stable, and even slightly improves over full DCCD in a few @5 settings. This indicates that document-level answerability is the dominant signal for resolving intra-context conflict, especially on DRQA, where the model must identify the document containing the current ground-truth answer. However, Doc-Only becomes consistently worse at @10, suggesting that token-level confidence provides a useful complementary signal as the retrieved bundle becomes more distractor-heavy. The fixed-gate ablation is mixed, setting \(\epsilon_t=1.0\) slightly improves TQA, but hurts most other settings, including larger drops on NQ and PopQA at @5 and on DRQA at @10. This suggests that always applying the same contrast strength can occasionally help, but the adaptive margin is more robust across datasets and retrieval depths.

6 Conclusion and Future Work↩︎

In this work, we propose Dual-Confidence Contrastive Decoding(DCCD), a training-free contrastive decoding method that combines document and token-level confidence to select positive and negative evidence sources during generation, and introduce DRQA, a factual-conflict QA benchmark derived from enterprise deep-research scenarios. Across DRQA and standard RAG benchmarks, DCCD consistently improves over prior decoding baselines, with the largest gains on DRQA. These results suggest that source-aware, confidence-guided contrast is especially useful when retrieved evidence is internally inconsistent.

For future work, improving the efficiency of DCCD is an important next step, since the method requires document-conditioned forward passes during decoding. Promising directions include more aggressive caching, document pruning, and lightweight confidence estimation. Second, DRQA currently focuses on factual-conflict QA in enterprise-style settings, extending the benchmark to longer-form answers, richer source metadata, and additional domains would further test retrieval-grounded conflict resolution. Finally, our confidence signals are simple and training-free. Learning or calibrating document-level and token-level confidence estimates may further improve robustness when retrieved evidence is ambiguous or insufficient.

Limitations↩︎

Similar to other document-aware contrastive decoding methods, DCCD introduces additional inference cost. At each decoding step, the method requires the full-context logits and document-conditioned logits for the retrieved documents, while the document-level confidence probe is computed once per question–document pair. This cost scales linearly with the number of retrieved documents, which has the same computational complexity as DVD [@jin-etal-2024-dvd]. In practice, our experiments use top-\(5\) and top-\(10\) retrieval settings, which are common in multi-document QA evaluation, but more efficient implementations such as batching document-conditioned forward passes, caching document states, or applying DCCD only at answer-critical decoding steps could reduce runtime overhead.

DRQA is designed to isolate retrieval-grounded conflict resolution in enterprise-style multi-document QA, where the answer is unavailable from internal memory and must be recovered from retrieved evidence. This controlled design is useful for evaluating the specific capability targeted by DCCD, but it is not intended to cover all forms of enterprise RAG. In particular, our current benchmark focuses on factoid question answering over text representations of enterprise documents, rather than full long-form report generation, interactive tool use, or multi-modal file understanding. These settings are closely related to deep-research workflows and could provide valuable extensions of DRQA.

Ethical considerations↩︎

DRQA is constructed from synthetic enterprise-style documents and is intended for research on retrieval-grounded conflict resolution, not for representing real companies, employees, or operational records. Because the benchmark includes deliberately generated misinformation and outdated evidence, the dataset should not be used as a factual knowledge source. Its intended use is to evaluate whether RAG systems can identify reliable evidence when retrieved documents conflict. The generated documents use synthetic personas, organizations, and facts. We have verified that the dataset does not contain real private enterprise records or intentionally identifying information. The benchmark is intended for research use only.

7 DRQA Construction Details↩︎

This section describes the procedure used to construct DRQA from DRBench. For each selected DRBench internal insight, we construct one DRQA example. Each example consists of a question, a ground-truth answer, and a retrieved document bundle. The question is derived from the internal insight and asks for a specific enterprise-contextualized value, such as a percentage, dollar amount, date, count, or named entity. The ground-truth answer is the original DRBench internal insight. For example, an insight such as “Lee’s Market reduced food waste by 8% in Q2 2024, saving $1.2M” can be converted into the question “What food-waste reduction rate did Lee’s Market achieve in Q2 2024?”.

7.1 DRBench Background↩︎

DRBench [@abaskohi2026drbench] is an enterprise deep-research benchmark designed to evaluate agents on complex, open-ended research tasks. Unlike standard question answering benchmarks [@kwiatkowski-etal-2019-natural; @izacard-grave-2021-leveraging; @mallen-etal-2023-trust; @zhang-etal-2024-retrievalqa], which usually ask for a short factual answer from a public corpus, DRBench tasks require an agent to synthesize information across public and private enterprise sources and produce a structured research report. Each task is grounded in an enterprise scenario that specifies a company context, a user persona, and a broad deep-research question. The retrieval environment includes heterogeneous sources such as public web pages, productivity documents, cloud files, emails, and chat conversations.

Each DRBench task contains supporting facts, referred to as insights, that are relevant to answering the deep-research question. These insights include both external facts that may be available on the public web and internal facts that are generated as private enterprise knowledge. The internal facts are embedded in generated enterprise documents using a “needle-in-a-haystack construction”. Specifically, a realistic document is first outlined in a target modality, the insight is inserted into an appropriate section, and the remaining content is filled with contextually plausible but topically irrelevant text. This construction creates realistic enterprise files in which the answer-bearing evidence is embedded among surrounding noise.

DRQA reuses DRBench as scenario scaffolding rather than as a long-form report-generation benchmark. In particular, we take the synthetically generated internal insights from DRBench, which are not expected to be recoverable from the model’s internal memory, and convert them into short-answer multi-document question answering tasks. This allows us to isolate the intermediate RAG capability we study in this paper: given a question and a bundle of retrieved documents, the model must identify which document contains the most reliable evidence and commit to the correct answer.

7.2 Document Generation Prompts↩︎

We list the prompt templates used during DRQA construction. Each template is shown as a Python f-string as passed to the LLM. Unless otherwise stated, prompts were run with anthropic/claude-opus-4-5 [@anthropic2025claudeopus45] through OpenRouter¹.

7.2.0.1 Wrong-answer generation for misinformation documents

This prompt produces the perturbed value inserted into each misinformation document. It is called once per misinformation document before the format-specific document generator. The document date is passed to the prompt so that generated dates or time periods remain coherent with the document timestamp.

7.2.0.2 Historical-answer generation for temporal documents

This prompt produces the historical value inserted into a temporal document. The document date is sampled first and passed to the LLM, ensuring that the generated value is coherent with the document’s timestamp.

7.2.0.3 Additional context for misinformation documents

For misinformation documents, we append a format-specific uncertainty context to the document-generation prompt. This context encourages the generated document to express the incorrect claim with subtle unreliability markers rather than obvious disclaimers. Two phrases are sampled uniformly without replacement from the pool corresponding to the document format, shown in 2.

7.2.0.4 Additional context for temporal documents

For temporal documents, we append a format-specific context block that makes the document read as authoritative at the time of writing. The document date is the only staleness signal. Two phrases are sampled uniformly from the format-specific pool in 3.

7.2.0.5 Format-specific document generators

After the wrong-answer or temporal-answer prompt produces a perturbed answer, we call a format-specific generator to produce the final document. The default word_limit values are 300 for email, 300 for chat, and 500 for Markdown. JSONL outputs are parsed line by line. Markdown outputs are stripped of surrounding Markdown code fences when present.

Topic Variable Prompt:

Email Generator Prompt:

Chat Generator Prompt

Markdown Document Generator Prompt:

7.3 LLM-as-Judge Verification Prompts↩︎

Each generated misinformation, temporal, and noise document is independently checked by an LLM-as-judge using a fixed type-specific rubric. The judge returns structured JSON containing a binary verdict, a one-sentence rationale, and a scalar confidence in \([0,1]\). The judge receives the question, the ground-truth answer, the claimed answer when applicable, the document date when applicable, and the first 2000 characters of the document text. Correct documents copied from DRBench source files are auto-passed because they were already validated in DRBench; applying a 2000-character truncation to long source documents can otherwise create false negatives when the answer appears later in the file.

7.3.0.1 Correct-Document Rubric Prompt:

7.3.0.2 Misinformation-Document Rubric Prompt:

7.3.0.3 Temporal-Document Rubric Prompt:

7.3.0.4 Noise-Document Rubric Prompt:

7.4 PPTX Markdown Reconstruction↩︎

Since DRBench PPTX files do not include an exported text representation, we avoid using an LLM for deterministic reformatting and render the structured file_dict.json metadata as Markdown.

This reconstruction preserves the text fields but does not attempt to preserve slide layout or visual formatting.

7.5 Final Dataset↩︎

Table 4 summarizes the two released DRQA bundle configurations. Both configurations are derived from the same source examples. The @10 configuration adds one additional misinformation document, one additional temporal document, and three additional noise documents relative to @5.

4pt

8 DCCD Implementation Details↩︎

This appendix provides implementation details for Dual-Confidence Contrastive Decoding (DCCD), complementing the method description in §4.

8.1 Support Probe for Document-Level Confidence↩︎

For each question–document pair \((x,d_i)\), we compute a document-level confidence score \(q_i\) before decoding. The goal of \(q_i\) is to estimate whether document \(d_i\) contains enough information to answer the question, independent of the current generation prefix.

8.1.0.1 Prompt Template

For each retrieved document \(d_i\), we use the following confidence probe:

8.1.0.2 Yes/No Verbalizers

Let \(\mathcal{Y}_{+}\) denote the set of yes verbalizer token ids and \(\mathcal{Y}_{-}\) denote the set of no verbalizer token ids. In practice, we construct these sets by tokenizing common case and leading-space variants: \[\{\texttt{yes}, \texttt{Yes}, \texttt{YES}, \texttt{ yes}, \texttt{ Yes}, \texttt{ YES}\}\] for yes, and analogously for no. We keep only variants that are represented as a single token by the tokenizer. Leading-space variants are included because the BPE [@sennrich-etal-2016-neural] for Qwen3.5 and the SentencePiece tokenizers [@kudo-richardson-2018-sentencepiece] encode the first content token of an assistant response with an initial whitespace marker.

8.1.0.3 Score Computation

We compute the document-level confidence score by re-normalizing the probability mass assigned to yes and no verbalizers. In practice, we compute the same quantity in logit space for numerical stability. Let \(\mathbf{z}^{\mathrm{sup}}_i\) be the final-position logits produced by the support probe for document \(d_i\). We compute \[\begin{align} \ell^+_i &= \operatorname{logsumexp}_{v\in \mathcal{Y}_{+}} \mathbf{z}^{\mathrm{sup}}_i[v], \\ \ell^-_i &= \operatorname{logsumexp}_{v\in \mathcal{Y}_{-}} \mathbf{z}^{\mathrm{sup}}_i[v], \\ q_i &= \sigma(\ell^+_i-\ell^-_i), \label{eq:app-q-logspace} \end{align}\tag{11}\] where \(\sigma\) is the logistic sigmoid. This is equivalent to the yes/no renormalization in Eq. 1 . All document probes for a query are batched into a single forward pass, so the support-probe stage is computed once before decoding.

8.2 Token-Level Confidence↩︎

At decoding step \(t\), let \(\mathbf{z}^i_t\in\mathbb{R}^{V}\) denote the next-token logits conditioned on the question \(x\), document \(d_i\), and the current output prefix \(y_{<t}\). Let \(\mathcal{T}_{i,t}\) be the set of top-\(k\) token indices under \(\mathbf{z}^i_t\). We compute token-level confidence by first mapping these top-\(k\) logits to positive evidence values and then measuring the expected entropy of the induced distribution.

8.2.0.1 Top-\(k\) Evidence

Following the Dirichlet uncertainty framework of [@malinin-gales-2018-predictive], we convert each selected logit into a positive concentration parameter for a lightweight Dirichlet approximation: \[\alpha_{(i,t,v)} = \operatorname{softplus}\!\left(\mathbf{z}^i_t[v]\right)+1, \qquad v\in\mathcal{T}_{(i,t)}. \label{eq:app-alpha}\tag{12}\] The softplus transformation keeps the evidence positive without exponentiating logits, and the \(+1\) shift ensures that each concentration parameter is at least one. We define the total concentration and the corresponding Dirichlet mean as \[S_{(i,t)} = \sum_{v\in\mathcal{T}_{i,t}} \alpha_{(i,t,v)}, \qquad \bar p_{(i,t,v)} = \frac{\alpha_{(i,t,v)}}{S_{(i,t)}}. \label{eq:app-dirichlet-mean}\tag{13}\]

8.2.0.2 Expected Entropy

We take the top-\(k\) next-token probabilities [@nguyen2026probabilities] and treat them as a categorical distribution sampled from a Dirichlet posterior: \[\boldsymbol{\pi}^i_t \sim \operatorname{Dir}\!\left(\boldsymbol{\alpha}^i_t\right).\] The raw token-level uncertainty is the expected entropy of this sampled categorical distribution: \[\widetilde{H}_k(\mathbf{z}^i_t) = \mathbb{E}_{\boldsymbol{\pi}^i_t\sim \operatorname{Dir}(\boldsymbol{\alpha}^i_t)} \left[ H(\boldsymbol{\pi}^i_t) \right]. \label{eq:app-raw-entropy-def}\tag{14}\] This expectation has the closed form \[\widetilde{H}_k(\mathbf{z}^i_t) = \sum_{v\in\mathcal{T}_{(i,t)}} \bar p_{(i,t,v)} \left[ \psi(S_{(i,t)}+1) - \psi(\alpha_{(i,t,v)}+1) \right], \label{eq:app-raw-entropy}\tag{15}\] where \(\psi\) is the digamma function: \[\psi(u) = \frac{d}{du}\log \Gamma(u) = \frac{\Gamma'(u)}{\Gamma(u)}. \label{eq:app-digamma}\tag{16}\] The uncertainty \(\widetilde{H}_k\) is small when the document-conditioned evidence is concentrated on a few next-token candidates, and larger when the evidence is spread across many candidates.

8.2.0.3 Normalized Confidence

To make the score comparable across documents and bounded in \([0,1]\), we normalize by the maximum entropy over a top-\(k\) support: \[H_k(\mathbf{z}^i_t) = \frac{ \widetilde{H}_k(\mathbf{z}^i_t) }{ \log k }. \label{eq:app-normalized-entropy}\tag{17}\] The token-level confidence used in the main decoding rule is then \[c_{(i,t)} = 1-H_k(\mathbf{z}^i_t). \label{eq:app-token-confidence}\tag{18}\] Thus, \(c_{(i,t)}\) is high when document \(d_i\) yields a sharp next-token prediction at step \(t\), and low when the model remains uncertain about how to continue from that document. Unlike the document-level confidence \(q_i\), which is computed once per question–document pair, \(c_{(i,t)}\) is recomputed at every decoding step.

8.3 Pseudocode↩︎

Algorithm 2 summarizes the complete procedure for one query.

8.4 Computational Cost↩︎

The document-level support probe is computed once per query. All \(n\) document probes can be batched into a single forward pass, producing the \(n\) values of \(q_i\). This cost is amortized over the entire generation.

At each decoding step, DCCD requires one full-context forward stream and \(n\) single-document forward streams. Thus, for \(n\) retrieved documents, the decoding stage uses \(n+1\) streams per token. The token-level confidence computation adds only \(O(nk)\) operations for top-\(k\) extraction and digamma evaluations, which is negligible compared with the model forward passes for the values of \(k\) used in our experiments.

The implementation can use the model’s standard KV cache for each stream. With caching, each decoding step only extends the existing streams by one token. The memory footprint scales linearly with the number of streams, and the runtime scales approximately linearly with \(n+1\) relative to full-context greedy decoding, plus the one-time support-probe pass.

9 Experimental Details↩︎

This appendix provides additional experimental details that describe the datasets, baselines, models, and headline evaluation protocol. Here, we record the implementation-level choices needed to reproduce the experiments, including dataset formatting, prompting, decoding configurations, and hyperparameters.

9.1 Dataset Details↩︎

Table 5 summarizes the datasets used in our experiments. Natural Questions (NQ) and TriviaQA (TQA) use the preprocessing of [@izacard-grave-2021-leveraging], which provides DPR-retrieved passages with title and text fields. PopQA uses the original release from [@mallen-etal-2023-trust]. RetrievalQA follows [@zhang-etal-2024-retrievalqa] and is converted into the same document-list format. DRQA is our factual-conflict benchmark derived from DRBench, which consists of 568 internal enterprise facts.

6pt

9.2 Prompt Templates↩︎

All methods use the same answer-generation prompt format. We adopt the same prompt template from prior work [@shi-etal-2024-trusting; @jin-etal-2024-dvd] with the zero-shot and contextualized prompts displayed below.

Zero-shot Prompt:

Contextualized Prompt:

All prompts are wrapped with the default chat template provided by the corresponding Hugging Face tokenizer. We use the chat template both for answer generation and for the document-level support probe in DCCD, so confidence estimation and final decoding are performed under the same instruction-following format. The per-document prompts follow the same template as the contextualized prompt following [@jin-etal-2024-dvd].

9.3 Baseline Decoding Configurations↩︎

We use greedy decoding with temperature \(0\) and a maximum generation length of \(60\) new tokens for all methods. To reduce artifacts from recall-oriented string-match metrics, we generate and evaluate a single-sentence answer. Since strem rewards the presence of a normalized gold string anywhere in the prediction, long generations that copy retrieved passages or list multiple candidate answers can be artificially rewarded even when the model does not clearly commit to the correct answer. Single-sentence generation makes the comparison across decoding methods more meaningful by forcing each method to produce a concise prediction. Unless otherwise stated, hyperparameters are fixed across all datasets and models. We describe the hyperparameter settings for all decoding strategies used in our experiments and summarize in 6.

9.3.0.1 Zero-shot Decoding

The zero-shot baseline generates from the question alone, without any retrieved documents. It is included to estimate how much of each dataset can be answered from parametric memory alone.

9.3.0.2 Full-Context Decoding

The full-context baseline decodes directly from the logits conditioned on the question and the full retrieved document set: \[\mathbf{z}_t = \mathbf{z}^{\mathrm{full}}_t.\]

9.3.0.3 CAD

Context-Aware Decoding (CAD) contrasts full-context logits against no-context logits: \[\mathbf{z}^{\mathrm{CAD}}_t = (1+\alpha)\mathbf{z}^{\mathrm{full}}_t - \alpha \mathbf{z}^{\mathrm{none}}_t.\] We use \(\alpha=0.2\) based on the results reported by [@shi-etal-2024-trusting].

9.3.0.4 AdaCAD

Adaptive Context Aware Decoding (AdaCAD) replaces the fixed CAD weight with a token-level adaptive weight based on the Jensen-Shannon divergence between the full-context and no-context distributions: \[\alpha_t = \mathrm{JSD} \left( p^{\mathrm{full}}_t \Vert p^{\mathrm{none}}_t \right),\] \[\mathbf{z}^{\mathrm{\mathrm{\small AdaCAD}}}_t = (1+\alpha_t)\mathbf{z}^{\mathrm{full}}_t - \alpha_t\mathbf{z}^{\mathrm{none}}_t.\] We set the minimum weight floor to \(0.0\) following [@wang-etal-2025-adacad].

9.3.0.5 DVD

Dynamic Contrastive Decoding (DVD) is a document-aware contrastive decoding baseline for multi-document QA. We use the static-weight configuration with \(\beta=0.25\), \(\gamma=0.2\), and document top-\(k=10\). In early experiments, we find that dynamic \(\beta\) performs poorly, so we followed the suggested configuration from [@jin-etal-2024-dvd]. The method selects document-conditioned streams according to its entropy-based criterion and applies a document-level contrast together with a context-prior contrast.

9.3.0.6 CoCoA

Confidence- and Context-Aware Adaptive Decoding (CoCoA) uses a confidence-aware interpolation between no-context and full-context predictions. We use the paper-faithful configuration with Rényi divergence order \(\alpha=0.5\), entropy-gap weight \(\gamma=1.0\), peakedness amplifier \(z=5.0\), and numerical stability constants \(\delta=10^{-8}\) and \(\varepsilon=10^{-12}\) following [@khandelwal-etal-2025-cocoa].

9.3.0.7 DCCD

Our proposed Dual-Confidence Contrastive Decoding(DCCD) has the advantage of being relatively hyperparameter-free, with the only hyperparameter being the top-\(k\) logits used for computing token-level confidence, which we set to \(k=10\) throughout all our experiments.

4pt

9.4 Models and Inference Backend↩︎

We evaluate the three instruction-tuned language models through Hugging Face transformers [@wolf-etal-2020-transformers], namely, Qwen3.5-2B² Qwen3.5-9B³, and Phi-3-medium⁴. All models are loaded with their default tokenizers and chat templates. We use bfloat16 inference when supported. Tokenizer padding is set to the left so the final-position logits are aligned across batched streams.

For contrastive decoding methods, each decoding step may require multiple forward streams. CAD, AdaCAD, and CoCoA use no-context and full-context streams. DVD and DCCD additionally use document-conditioned streams. For DCCD, the support probe is computed once before decoding, while the full-context and document-conditioned logits are computed during each decoding step. Streams are batched whenever possible.

9.5 Evaluation Metrics↩︎

9.5.0.1 Normalized string exact match.

For NQ, TriviaQA, PopQA, and RetrievalQA, we use normalized string exact match, denoted strem. We first extract the generated answer sentence and then normalize both the prediction and gold answers using the standard QA normalization procedure: lowercasing, removing punctuation, removing articles, and collapsing whitespace. A prediction is counted as correct if any normalized gold answer matches the normalized prediction by exact or substring match, following common retrieval-augmented QA evaluation practice.

9.5.0.2 LLM-as-a-judge for DRQA.

For DRQA, the reference answer is a free-form enterprise insight rather than a short canonical span. We therefore evaluate semantic correctness with an LLM judge. The judge receives the question, the reference answer, and the model prediction, and returns a binary correctness label. A prediction is marked correct if it conveys the same answer as the reference at the required level of specificity. Numerical and date answers must match the reference detail; refusals are marked incorrect unless the reference answer itself indicates that the information is unavailable.

We use deterministic judge decoding with temperature \(0\). Judge outputs are parsed as JSON with a boolean correctness field. If the judge response cannot be parsed, the example is conservatively marked incorrect. Below is the prompt template for our judge using anthropic/claude-opus-4-5 [@anthropic2025claudeopus45] through OpenRouter API⁵.

9.6 Hardware and Runtime↩︎

All experiments can be run on a single NVIDIA H100 80GB GPU. Runtime depends on model size, dataset size, retrieval depth, and decoding method. Full-context decoding is the cheapest setting because it uses a single generation stream. CAD, AdaCAD, and CoCoA require additional no-context streams. DVD and DCCD require document-conditioned streams. We report the average wall clock runtime in 7.

For \(n\)retrieved documents, the per-token cost of DCCD scales approximately linearly with \(n+1\): one stream for the full retrieved context and \(n\) streams for individual documents. The token-level confidence computation is inexpensive relative to the model forward pass because it only requires top-\(k\) extraction and digamma evaluations over a small number of tokens. The document-level support probe is computed once per question–document pair and is amortized over the generated answer.

10 Qualitative Examples↩︎

We present representative qualitative examples for Qwen3.5-2B with top-10 retrieved documents. Each example shows the first sentence generated by each decoding method. Predictions are truncated to 220 characters when necessary. Correct predictions are shaded green and incorrect predictions are shaded red.

For NQ, TriviaQA, RetrievalQA, and PopQA, correctness follows the first-sentence str-em metric described in 9.5. For DRQA, correctness is determined by the LLM-as-a-judge metric. To avoid str-em false positives from long generations that enumerate multiple candidate answers, we additionally filter qualitative examples to predictions of at most 240 characters that commit to a single value when the gold answer is numeric and contain no refusal phrase such as “no information” or “cannot determine”.

10.1 DRQA↩︎

Q (qid 194): How did Jason Wong respond to a simulated phishing incident in March 2025?
Gold: In March 2025, Jason Wong coordinated with the IT helpdesk and quickly contained a simulated phishing attack, reducing average internal response time to under 25 minutes.

10.2 Natural Questions↩︎

Q (qid 29): in which sea pearl is found in india
Gold: the Indian Ocean

Q (qid 1491): who wrote it’s gonna take a lot of love
Gold: Neil Young

Q (qid 2783): who is the all time leading scorer in ncaa tournament history
Gold: Pete Maravich

Q (qid 2792): where do they film young and the restless
Gold: CBS Television City

Q (qid 1600): where does stevia in the raw come from
Gold: the plant species Stevia rebaudiana

10.3 TriviaQA↩︎

Q (qid 7277): The name of a knot on a lasso is also the name of a Japanese motor manufacturer. What is the name?
Gold: HONDA

Q (qid 8593): In imperial measure, how many gills are there in a gallon?
Gold: 32

Q (qid 4705): How many US states border Lake Superior?
Gold: three

Q (qid 5136): In which “Star Wars” episode does Anakin Skywalker transform into the black-suited Darth Vader?
Gold: Star Wars: Episode III – Revenge of the Sith

Q (qid 8453): What most-rejected 1974 bestseller describes the 17-day journey of the author, who calls himself Phaedrus, and his son Chris from Minnesota to California?
Gold: Zen and the Art of Motorcycle Maintenance by Robert M. Pirsig

---

1. https://openrouter.ai↩︎

2. https://huggingface.co/Qwen/Qwen3.5-2B↩︎

3. https://huggingface.co/Qwen/Qwen3.5-9B↩︎

4. https://huggingface.co/microsoft/Phi-3-medium-128k-instruct↩︎

5. https://openrouter.ai↩︎