1 Introduction↩︎

Long-form Question Answering relies heavily on Retrieval-Augmented Generation (RAG) to generate comprehensive and factual responses [1], [2]. Although RAG mitigates “hallucinations”, it is prone to errors due to unreliable retrieval passages [3]. In practice, knowledge bases and web indices are noisy environments, containing not just irrelevant documents, but polluted evidence, which are plausible sounding texts that contain subtle fabrications, entity swaps, or incorrect temporal attributions [4], [5].

Current LLMs exhibit a “sycophancy” bias towards retrieved context, often propagating these errors into the final answer with high confidence [6]. Figure 1 demonstrates a typical failure mode: a single high scoring polluted document can steer the model away from established facts, resulting in generated answers that are relevant but factually incorrect. We argue that robustness in RAG requires moving beyond simple context concatenation toward active evidence adjudication.

To solve the aforementioned problem, we introduce MIRAGE, a methodology that defends the generation of RAG models against retrieval pollution by enforcing the verification of the corresponding claims. Our core insight is that correct claims are often repetitive and consistent across diverse sources, whereas statements containing misinformation are contradictory to each other. MIRAGE utilizes this observation via a two-stage process. First, it extracts sentence-level claims from retrieved passages and builds a support and contradiction graph using Natural Language Inference (NLI). By analyzing the graph’s internal inconsistency, MIRAGE decides whether to proceed with RAG or fallback to the model’s parametric knowledge. Second, for the generated response, we condition generation on a structured set of verified claims and instruct the model to ground its output in this evidence.

We rigorously evaluate MIRAGE on four long-form datasets using a new pollution protocol that injects controllable corruptions (e.g., Unambiguous, Conflicting, Misleading). Our results show that retrieval pollution severely degrades standard RAG. Under mixed pollution, MIRAGE improves factuality by retaining mutually supported clean claims while filtering inconsistent evidence. Under full pollution, MIRAGE detects unreliable retrieval and falls back to parametric generation, avoiding harmful context. Across polluted long-form QA settings, MIRAGE outperforms strong robust-RAG baselines. Our contributions are as follows:

• We introduce MIRAGE, a training-free and model-agnostic defense for long-form RAG that constructs cross-document NLI-based claim graphs to identify, prune, and gate misinformation-polluted evidence before generation (see Section 3).

• We develop a controlled minimal-edit retrieval pollution benchmark that systematically injects four realistic families of misinformation (Unambiguous, Conflicting, Misleading, and Fabricated), enabling reproducible evaluation under clean, mixed, and fully polluted retrieval settings (see Section 4).

• We conduct extensive experiments across four long-form QA benchmarks, multiple commercial and open-weight LLMs, and diverse robust-RAG baselines, showing that MIRAGE consistently restores factuality under polluted retrieval and substantially outperforms existing defenses while preserving the performance on clean retrieval (see Sections 5 and 6).

2 Related Work↩︎

Active RAG and Self-Reflective Generation. Retrieval-Augmented Generation (RAG) grounds LLM outputs in external evidence to reduce hallucinations. Beyond static “retrieve-then-generate,” active and self-reflective RAG methods adapt retrieval and revision decisions during generation. FLARE [7], Self-RAG [8], and DRAGIN [9] trigger iterative retrieval/refinement, while BRIDGE [10] tightly couples retrieval and generation via verification. However, these approaches largely assume a reasonably trustworthy corpus and mainly filter irrelevant context, leaving them vulnerable to semantically relevant but incorrect passages (i.e., polluted evidence). MIRAGE instead adds a plug-in verification layer that targets such semantic pollution before it influences generation.

Robustness Against Noisy, Conflicting, and Misleading Evidence. Recent work shows LLMs can over-trust retrieved context (“sycophancy”), propagating plausible misinformation [5]. Several defenses address conflicting/misleading retrieval: InstructRAG [11] studies strategies for misleading evidence; RetRobust [3] and AstuteRAG [12] improve robustness via re-ranking and dependence control. Other lines emphasize post-hoc verification, including RARG [13], TrustRAG [14], and multi-agent deliberation (MADAM-RAG; [15]); RobustRAG [16] aggregates generations across passages under perturbations. Unlike these approaches, MIRAGE is training-free and leverages global cross-document consistency: truthful evidence tends to be corroborated across sources, whereas polluted evidence induces structural inconsistency.

Concurrent work extends retrieval-pollution evaluation to multimodal evidence, introducing QIMG-7 and source-aware trust resolution over Parametric, Text-only, and Full-MM answer branches [17]. MIRAGE instead focuses on text-only long-form RAG, using a cross-source NLI claim graph and defended-claims gate to adjudicate polluted textual evidence before generation.

3 Methodology↩︎

3.1 Overview↩︎

MIRAGE operates as a wrapper around a standard retrieval–generation pipeline.¹ Based on top-\(k\) retrieved passages \(\mathcal{D}\), MIRAGE builds a defended-claims graph \(\mathcal{G}\) over extracted sentence-level claims. Then we prune inconsistent claims and use this graph as a global gate. If \(\mathcal{G}\) is sufficiently consistent and well-supported across independent domains, MIRAGE trusts retrieval and constructs a defended prompt for LLM containing only verified claims. Otherwise, it blocks retrieval and switches to a parametric (no-evidence) answer. Figure 2 and Algorithm 3 summarize the entire pipeline.

3.2 Claim Graph Construction↩︎

Given a user query \(q\), a retriever returns a set of top-\(k\) passages \(\mathcal{D} = \{d_1, \dots, d_k\}\). For each passage \(d \in \mathcal{D}\), MIRAGE performs lightweight sentence segmentation and treats each sentence as a claim \(c\), for which we track the following:

• a source identifier \(\text{src}(c) \in \{1,\dots,k\}\);

• a domain \(\text{dom}(c)\), derived from the source URL (e.g., wikipedia.org, nytimes.com);

• a claim-level retrieval score \(r(c)\), inherited from the source passage and computed as the dense-retriever similarity score, and its max-normalized counterpart \(\tilde{r}(c) \in [0,1]\), defined as \(\tilde{r}(c)=r(c)/\max_{p \in \mathcal{D}} r(p)\) over retrieved passages.

We discard empty sentences and take the highest-scoring sentences across passages according to \(\tilde{r}(c)\); see Appendix 17.1 for further details.

MIRAGE models relationships between claims with a directed, weighted graph. We first select pairs of claims that are likely to be semantically related, and then apply an NLI model to determine their relationship.

3.2.0.1 Pair selection.

We consider only cross-source pairs \((c_i, c_j)\) with \(\text{src}(c_i) \neq \text{src}(c_j)\), so that edges capture inter-document agreement or disagreement. Let \(T(c)\) denote the set of lowercased tokens in claim \(c\). We define a containment-style overlap: \[\text{overlap}(c_i, c_j) = \frac{|T(c_i) \cap T(c_j)|}{\min(|T(c_i)|, |T(c_j)|)}. \label{eq:jaccard}\tag{1}\] We retain pairs with \(\text{overlap}(c_i, c_j) \ge \tau_{\text{overlap}}\) and cap the total number of pairs per query.

The token-overlap filter in equation 1 is used only for efficient candidate generation before NLI adjudication. This containment-style overlap works well in practice because supporting evidence typically preserves key lexical anchors (e.g., named entities and dates). We use it as an efficiency-oriented approximation rather than a semantic matching strategy; Appendix 17.2 reports the resulting NLI savings and paraphrase-robustness trade-off.

3.2.0.2 NLI scoring.

For each selected pair \((c_i, c_j)\), we apply an NLI model to obtain probabilities \(p_\text{entail}, p_\text{contr}, p_\text{neut}\). We construct a weighted edge \(e_{ij} = (i, j, w^\text{supp}_{ij}, w^\text{contr}_{ij})\) where \[\begin{align} w^\text{supp}_{ij} &= \max\bigl(0,\; 2p_\text{entail}(c_i, c_j)-1 \bigr), \tag{2} \\ w^\text{contr}_{ij} &= p_\text{contr}(c_i, c_j). \tag{3} \end{align}\] Edges are classified as supporting if \(w^{\text{supp}}_{ij} > w^{\text{contr}}_{ij}\), and as contradictory if \(w^{\text{contr}}_{ij} > w^{\text{supp}}_{ij}\); neutral or low-confidence pairs are ignored.

3.2.0.3 Counter-evidence probing.

The initial graph captures only relations among retrieved claims. To detect missing counter-evidence, MIRAGE retrieves targeted counter-evidence for top-ranked claims and applies NLI between each claim and its retrieved passages. When contradiction dominates, a self-contradiction edge with weight \(w^{\text{contr}}\) is added, penalizing claims contradicted by external evidence; see Appendix 17.3 for details.

3.3 Inconsistent Claim Pruning↩︎

Given the graph, MIRAGE assigns each claim a score and selects a subset of mutually consistent, well-supported claims.

3.3.0.1 Claim scoring.

Each claim \(c\) is assigned a score \[\begin{align} s(c) &={} \log\bigl(1 + n(c)\bigr) \nonumber\\ &+ \alpha \cdot \mathrm{trust}(\mathrm{dom}(c)) + \beta \cdot \tilde{r}(c), \label{eq:claim95score} \end{align}\tag{4}\] where \(n(c)\) is the number of supporting claims, \(\mathrm{trust}(\cdot)\) is a weak domain-quality prior, and \(\tilde{r}(c)\) is the normalized retrieval score. The scoring function prioritizes cross-source agreement, using domain priors and retrieval rank only as secondary signals; thus, no single high-trust source can override broad cross-document disagreement. Details on the coefficient choices \((\alpha, \beta)\) are provided in Appendix 10.2, and details on the domain prior in Appendix 17.4 and Table 9.

3.3.0.2 Consistent subset selection.

Let \(V\) be the set of claims and \(E\) the set of edges with weights \(w^{\text{supp}}_{ij}\) and \(w^{\text{contr}}_{ij}\). We select a subset \(S \subseteq V\) by approximately maximizing \[F(S) = \sum_{c \in S} s(c) + \sum_{(i,j) \in E,\, i,j \in S} \bigl(w^{\text{supp}}_{ij} - w^{\text{contr}}_{ij}\bigr), \label{eq:defended95set95score}\tag{5}\] which favors individually reliable claims while promoting internal consistency.

Exact optimization is NP-hard, so we use a greedy hill-climbing procedure that iteratively removes claims whose exclusion improves \(F(S)\). The resulting subset \(S^\star\) forms the defended set, which is used for downstream gating.

3.4 Retrieval Trust Gating↩︎

MIRAGE decides whether to trust retrieval based on graph-level statistics of the defended set \(S^\star\). Let \(E^\text{supp}\) and \(E^\text{contr}\) denote the numbers of supporting and contradictory edges in \(S^\star\), classified by an NLI model. Let \(\mathcal{D}(S^\star)\) be the set of distinct domains among the claims in \(S^\star\). We define the contradiction ratio and source diversity as \[\tag{6} \begin{align} r_{\mathrm{contr}} &= \frac{E^\text{contr}}{E^\text{supp} + E^\text{contr} + \varepsilon}, \tag{7} \\ d_{\mathrm{src}} &= |\mathcal{D}(S^\star)|. \tag{8} \end{align}\] where \(\varepsilon\) is a small constant for numerical stability. Intuitively, contradiction ratio captures internal conflicts, while source diversity measures how broadly evidence is distributed across domains.

MIRAGE enables defended mode only when all active reliability criteria satisfy the gate thresholds \((r_{\max}, d_{\min})\): \[\tag{9} \begin{align} & r_{\mathrm{contr}} \le r_{\max}, \tag{10} \\ & d_{\mathrm{src}} \ge d_{\min}. \tag{11} \end{align}\]

If either condition in equation 9 is violated, MIRAGE deems retrieval unreliable and blocks RAG, instructing the LLM to answer from its parametric knowledge only. Otherwise, MIRAGE enters defended mode and conditions generation on the claims in \(S^\star\). In practice, the gate satisfies basic sanity checks: it passes most clean queries, degrades under mixed retrieval, and blocks nearly all fully polluted queries, see Table ¿tbl:tab:gate95outcomes?.

Thresholds are determined once offline from graph statistics on the combined held-out 20% splits of all datasets, without invoking the generator; these splits are excluded from downstream evaluation to avoid data leakage, and the resulting single global configuration is used across all datasets and models. Additional details are provided in Appendix 10, and a worked example is given in Appendix 15.

3.5 Generation from Verified Claims with No-Retrieval Fallback↩︎

If the gate accepts the defended graph (i.e., \(\mathcal{G}\) is considered reliable under equation 9 ), MIRAGE presents only the defended claims to the LLM through a structured verified-claims prompt. Each kept claim \(c \in S^\star\) is assigned a stable identifier \(C_i\) and reported as a bullet point, together with its score and source metadata. The prompt contains (i) the user question, (ii) the list of verified claims, and (iii) a short set of rules (see Appendix 8 for details). This design turns retrieval into an evidence-only input, constraining the model to generate answers grounded in verified claims and reducing hallucination.

4 Retrieval Pollution Protocol↩︎

To systematically stress-test long-form RAG under misinformation, we introduce minimally edited corrupted variants of retrieved passages, yielding three controlled retrieval regimes per query: Clean, Mixed, and Polluted, while keeping the evidence size fixed and preserving topical relevance.

4.0.0.1 Generation setup.

For each question \(q\), we start from cached clean retrieval passages \(D_{\mathrm{\small clean}}(q)=\{p_1,\dots,p_k\}\) and use GPT-4o-mini to generate corrupted variants \(\tilde{p}_i\) that remain fluent and on-topic while introducing at least one verifiable factual error. We enforce strict edit constraints, including high token overlap and neutral tone, to ensure realistic rather than overtly adversarial pollution.

The generator returns an explicit JSON object containing: type (pollution family), polluted (rewritten passage), and rationale (brief analysis-only note). The rationale and type are never shown to MIRAGE; they are used only for auditing and reporting.

4.0.0.2 Pollution families.

Inspired by prior evidence taxonomies for misleading/conflicting retrieval (e.g., [5]), we instantiate four corruption families under minimal-edit constraints:

• Unambiguous. Modify a single salient, checkable atomic fact (e.g., a year, count, location, or named entity) while leaving the rest of the passage unchanged.

• Conflicting. Paraphrase the passage while flipping the polarity of a verifiable claim (e.g., “was not convicted” \(\rightarrow\) “was convicted”, or a small but consequential numerical change), producing a direct contradiction.

• Misleading. Keep a subset of true statements but alter framing so a reader is nudged toward an incorrect conclusion (e.g., omission of key constraints, cherry-picking, or emphasizing rare exceptions), without inventing entirely new facts.

• Fabricated. Introduce plausible but non-existent particulars (e.g., invented committee names, fabricated study findings, or incorrect attributions), but keeping topic and entities consistent with \(q\).

For examples of each type of pollution, see Appendix 18.

4.0.0.3 Retrieval regimes.

Given the aligned clean and polluted pools, we define:

• Clean: provide only clean passages \(D_{\mathrm{\small clean}}(q)\).

• Polluted: provide only polluted passages \(D_{\mathrm{\small poll}}(q)=\{\tilde{p}_1,\dots,\tilde{p}_k\}\) (100% adversarial).

• Mixed: construct a fixed-size list of \(k\) passages where an \(\alpha\) fraction (50% in our experiments) are sampled from \(D_{\mathrm{\small poll}}(q)\) and the remainder from \(D_{\mathrm{\small clean}}(q)\), preserving per-query alignment.

This protocol produces matched clean, mixed, and polluted evidence sets for each benchmark. MIRAGE operates without access to ground‑truth passage labels; its decisions are based solely on the retrieved text and the structural properties of the defended‑claims graph.

5 Experimental Setup↩︎

5.1 Datasets and Models↩︎

We report results on four long-form benchmarks: LongFact [1], FAVA [18] , AlpacaFact [19] and Biography [20]. We evaluate MIRAGE on both commercial and open-weight LLMs to ensure our findings are model‑agnostic. We use GPT-4o-mini [21] and GPT-5-mini [21] as strong commercial baselines, and three diverse open-weight models, namely LLaMA-3.1-8B [22], Mistral-7B-Instruct-v0.2 [23], and Qwen3-8B [24].

All four benchmarks are prompt-only (questions without an accompanying gold evidence set). To enable controlled and reproducible retrieval/pollution evaluation, we construct a dataset-specific cached clean evidence pool from web search and derive Clean/Mixed/Fully-Polluted retrieval regimes from that fixed pool. Full details of evidence-pool construction, de-duplication, retrieval indexing, and caching are provided in Appendix 11.

0pt

5.2 Retrieval and Verification↩︎

We use a fixed dense retriever over a dataset-specific evidence pool and retrieve the top-\(k\) passages per query (\(k{=}20\)). For claim-graph construction (and the optional sentence-level checker), we use DeBERTa-large-MNLI [25].

We evaluate factuality with VeriScore [26] F1@\(k\) (median over prompts), which decomposes each generated answer into claims and uses a verifier to label each claim as supported, unsupported, or insufficient given evidence snippets [1]; see Appendix 12.

Unless otherwise stated, all experiments use a single fixed global configuration for both the core gate criteria (Table 7) and computational settings (Table 8). Core gate criteria configuration is selected offline via grid search on cached retrieval and NLI outputs without invoking the generator.

6 Experimental Results↩︎

6.1 Impact of Polluted Retrieval on RAG↩︎

Table 1 summarizes robustness to retrieval pollution averaged across datasets. Under clean retrieval, MIRAGE closely matches vanilla RAG, showing that the defended-claims gate rarely suppresses coherent evidence. Under mixed and especially full pollution, vanilla RAG degrades sharply and often falls below No-RAG, whereas MIRAGE restores factuality toward clean-RAG levels by filtering contradiction-heavy evidence and falling back to parametric answering when needed.

Under MixP, gains are large across backbones (e.g., GPT-4o-mini improves from \(53.88\%\) to \(83.43\%\) on average; for LLaMA-3.1-8B, it improves from \(41.71\%\) to \(71.91\%\)). Under FullP, vanilla RAG collapses (e.g., GPT-4o-mini: \(39.87\%\) vs.No-RAG \(75.88\%\)), while MIRAGE substantially recovers performance (GPT-4o-mini reaches \(78.00\%\) on average; LLaMA-3.1-8B reaches \(70.34\%\)). See Appendix 16 for per-dataset results and discussion.

6.2 Long-form QA Results↩︎

0pt

Table 2 compares MIRAGE to robust-RAG baselines under both Mixed Pollution (MixP) and Full Pollution (FullP). Across all datasets, MIRAGE ranks first for both backbones and both regimes, indicating that its defended-claims gating and pruning provide consistent robustness gains.

The largest gains appear under FullP, where baselines are vulnerable to contradiction-heavy retrieval while MIRAGE blocks unreliable evidence and falls back to parametric answering, e.g., on LongFact, MIRAGE improves over the strongest baseline for GPT-4o-mini (\(88.93\%\) vs.\(74.87\%\)) and for Qwen3-8B (\(85.11\%\) vs.\(66.41\%\)). Similar trends hold across Biography, FAVA, and AlpacaFact.

These gains primarily reflect avoiding harmful retrieval rather than exploiting polluted evidence. Under polluted retrieval, existing methods often fall below the No-RAG baseline, whereas MIRAGE surpasses No-RAG through trust-aware gating and fallback no-evidence generation. Under FullP, improvements mainly come from blocking polluted context, while under MixP, MIRAGE also benefits from retained clean, mutually supported claims.

6.3 Ablations↩︎

6.3.0.1 Gate behavior under clean, mixed, and polluted retrieval.

The gate relies on two interpretable consistency criteria \((r_{\max}, d_{\min})\), controlling contradiction tolerance, and source diversity. It trusts retrieval only when the defended-claims graph is globally consistent, well-supported, and backed by multiple independent domains.

Table ¿tbl:tab:gate95outcomes? shows that the gate passes 87.2% of clean queries on average, drops to 48.6% under mixed retrieval, and blocks nearly all fully polluted queries (0% pass). Full per-dataset results are provided in Appendix Table 10.

2.0pt (a) DeBERTa-large-MNLI gate outcomes

(b) Best-gate behavior across NLI backbones

6.3.0.2 Sensitivity to retrieval depth \(k\).

We conduct a sensitivity analysis on LongFact with GPT-4o-mini, varying the number of retrieved passages \(k\) while keeping all other hyperparameters fixed. Table 3 reports the median VeriScore F1@\(k\) over 250 prompts across different corruption settings.

Overall, performance varies only slightly across \(k=5\) to \(50\), suggesting that MIRAGE is robust to retrieval depth in this range. The optimal value depends mildly on the corruption level, with larger \(k\) benefiting heavily polluted settings.

6.3.0.3 Impact of MIRAGE components.

We ablate MIRAGE components for GPT-4o-mini across four datasets; Table 4 reports average F1 and relative drops from the full model. Each ablation hurts performance, showing that counter-evidence probing, contradiction-based trust, and diversity-based trust provide complementary robustness. Under mixed pollution, single-component removals cause similar drops, suggesting that no individual signal alone explains the gain. Under full pollution, diversity-based gating is most critical: removing it causes a 26.5% drop, while removing all trust-gating components causes the largest degradation (38.9%). See Appendix 13 for other baselines.

5pt

6.4 Automatic Label Validation↩︎

6.4.0.1 NLI labels validation.

We manually validate \(150\) claim pairs balanced across clean, mixed, and polluted retrieval (\(50\) each). Two annotators label each pair as supported or unsupported, reaching \(89.33\%\) raw agreement and Cohen’s \(\kappa=0.743\); per-subset results are reported in Table 5. This validates the graph edges used by MIRAGE before generation, rather than only the final answers. Comparing NLI labels against merged human annotations gives \(90.67\%\) agreement and \(\kappa=0.723\), with supported precision/recall of \(0.917/0.965\) and unsupported precision/recall of \(0.862/0.714\).

6.4.0.2 VeriScore labels validation.

We manually verify 150 VeriScore-labeled claims with retrieved evidence; two annotators achieve \(87.33\%\) agreement. This check complements the NLI-label audit above: while the NLI audit validates the claim-graph edges used by MIRAGE, the VeriScore audit validates the factuality labels used to evaluate generated answers. We further evaluate 600 claims balanced across clean, mixed, and polluted settings using GPT-5-mini, GPT-4.1-nano, GPT-4o, and GPT-5.5 [21], obtaining a mean agreement of \(92.33\%\) with VeriScore.

These results suggest that the observed robustness gains are unlikely to be an artifact of a single verifier or a single retrieval regime. The high agreement therefore supports using VeriScore consistently across clean, mixed, and fully polluted regimes. Prior work also supports VeriScore reliability, reporting strong agreement with human annotations, including substantial inter-annotator agreement (\(\kappa\) up to \(0.73\)), across multiple datasets and studies [2], [27], [28].

7 Conclusion and Future Work↩︎

To defend against retrieval pollution (on-topic misinformation), we introduce MIRAGE, a training-free, model-agnostic defense for long-form RAG. MIRAGE builds a cross-source claim graph and uses a defended-claims gate to either generate from globally consistent evidence or fall back to parametric answering.

Across four long-form QA benchmarks and multiple LLMs, polluted retrieval degrades vanilla RAG, while MIRAGE restores factuality under mixed and fully polluted evidence and outperforms prior robust-RAG methods. We also release a minimal-edit retrieval pollution protocol for controlled evaluation. Our results highlight cross-source consistency as a key signal for safeguarding factual generation under untrusted retrieval. More broadly, the findings suggest that robust RAG should verify retrieved evidence before generation, rather than treating topical relevance as sufficient evidence quality.

Future work includes improving robustness to domain shift, multilingual and coordinated misinformation, and scaling claim-graph construction with more efficient gating.

Limitations↩︎

MIRAGE inherits the coverage and diversity of retrieved evidence: low recall or limited source diversity can produce sparse graphs and more conservative blocking, yielding safer but less informative answers. It uses NLI-based consistency and lexical-overlap filtering as efficient, interpretable approximations for scalable cross-claim verification, though these can miss paraphrastic or omission-based misinformation. Coordinated misinformation repeated across sources may also appear trustworthy. MIRAGE is intentionally training-free and modular, improving robustness, interpretability, and auditability across settings, though this may limit adaptation to some domains or languages. Finally, while generation still requires only one LLM call per query, MIRAGE introduces additional local NLI computation for graph construction and counter-evidence checks; Appendix 14 reports the resulting overhead.

Ethical Considerations↩︎

MIRAGE aims to improve robustness of RAG under misinformation, reducing the risk of propagating incorrect retrieved content. Our pollution protocol, while designed for controlled evaluation, could be misused to generate plausible misinformation; we therefore recommend controlled release and standard safeguards.

It relies on NLI as a standard proxy for semantic consistency and on lightweight domain priors. While effective in practice, these components may exhibit bias or degrade under domain shift; we mitigate this through aggregation and conservative abstention, but monitoring and adaptation to deployment context remain important.

8 MIRAGE Prompts↩︎

8.1 Verified-Claims Generation (Gate Passed)↩︎

Task: Write a factual long-form answer using the verified claims below.

Rules:
(1) Do not add new unverified facts.
(2) Cite claims inline as [C#].
(3) Prefer higher-scoring claims.
(4) If claims are insufficient or conflicting, say so and stop.

Question: {q}

Verified claims:
C1: {text} (score={s}; src={domain});
...

8.2 Parametric Generation (Gate Blocked)↩︎

Write the best factual long-form answer to the question using only your internal knowledge.
Be explicit about uncertainty.
Avoid speculation.
If you are not confident, say what would be needed to verify the answer.

9 Pollution Protocol Details↩︎

9.1 Goal↩︎

To stress-test long-form RAG under realistic retrieval noise, we construct polluted evidence by minimally editing clean passages into on-topic, linguistically plausible text that is nevertheless factually incorrect. The objective is to preserve surface relevance and fluency while injecting concrete, verifiable errors that can mislead a generator that over-trusts retrieved context.

9.2 Input and output format↩︎

For each instance, the generator receives (i) a QA-style question (topic anchor), (ii) an evidence passage (optionally with a title), and (iii) a target pollution type (hint). It returns strict JSON with keys: type (pollution family), polluted (the rewritten passage), and rationale (brief researcher-facing explanation of what was changed and why it is wrong).

9.3 Controlled Corruption Constraints↩︎

To avoid trivial artifacts, we enforce: (1) topic preservation (the rewrite remains about the same entities and subject as the question/passage), (2) minimal editing (preserve 50–80% of tokens and cap token-change at \(\le 0.75\)), (3) non-absurd plausibility (neutral encyclopedic tone; no satire), and (4) anti-shortcut rules (reject vague comparatives and meta-commentary; no citations/URLs).

9.4 Generation Prompt↩︎

We use the following system and user templates, shown verbatim, to produce polluted passages:

You are an adversarial data generator for robustness testing.
You will receive:
- a QA-style question (topic anchor)
- an evidence passage (title optional)

Your task: rewrite the passage so it stays 100but is factually WRONG according to reality and the original passage.

Do NOT answer the question directly; produce an evidence-style paragraph.

Keep entities and topic consistent, but alter at least one concrete, verifiable atomic fact (e.g., year, number, location detail, role, or causal direction) to make the content incorrect yet superficially credible.

Never include citations, brackets, URLs, or meta-comments.
Do not say it is wrong or fabricated.
Mimic a neutral encyclopedic tone unless instructed otherwise.

You must pick at least one {pollution_types} strategy and follow its rules:

- Conflicting: explicitly contradict a key fact from the original passage
  (e.g., swap dates/names/causes).
- Misleading: keep some truths but cherry-pick, omit, or frame details so that
  a reader is led to a wrong conclusion.
- Fabricated: invent plausible but non-existent particulars
  (e.g., a committee name, a study, or figures).
- Unambiguous: minimally alter one salient, explicit, checkable atomic fact from the original passage while leaving the rest of the passage largely unchanged.

Hard constraints:
- Stay relevant to the given question and original passage.
- Avoid absurdity, satire, or obvious jokes.
- DO NOT mention which strategy you used.
- Output STRICT JSON ONLY with keys: type, polluted, rationale.
- type in {pollution_types}
- polluted: the rewritten wrong passage
- rationale: 1-2 sentences explaining, for researchers, how it is wrong.
  Do not include this rationale in the polluted text.

9.5 Dataset-level balancing↩︎

To reduce confounding from any single corruption style, we generate polluted passages using an approximately uniform mix over the four families. Table 6 reports dataset-level statistics, including the number of corrupted points for the full-pollution and mixed-pollution settings, as well as mixed clean and unknown counts. Figure 5 reports the resulting distribution per dataset.

9pt

10 Defended-Claims Gate↩︎

We distinguish between (i) core semantic gate criteria that define retrieval reliability (Table 7) and (ii) computational graph-construction parameters used for tractability (Table 8). Unless stated otherwise, we use a single fixed configuration across datasets and LLM backbones.

4pt

7pt

Complementing Table ¿tbl:tab:gate95outcomes?, which reports averaged offline gate pass/block rates across datasets, we provide the corresponding per-dataset breakdown in Table 10.

4pt

10.1 Domain Trust Mapping↩︎

For each evidence passage, we extract the source domain by parsing the URL host when available (or by regex-detecting a URL inside the passage text); if no URL is present, we set the domain to unknown. We then assign a domain-trust weight by applying the rules in Table 9 via substring or TLD matching. If multiple rules match, the implementation applies a fixed priority order to ensure deterministic assignment.

10.2 Claim Scoring Weights↩︎

The coefficients \((\alpha, \beta)\) in equation 4 are not tuned per dataset and are fixed across all experiments (\(\alpha = 0.5\) and \(\beta = 0.3\)). This choice encodes a simple priority ordering that we keep consistent throughout: (i) cross-source agreement should dominate, (ii) domain quality should provide a modest prior, and (iii) retrieval rank should only break ties.

We use \(\log(1+n(c))\) so that the first independent supporter yields a large gain (from \(0\) to \(\log 2 \approx 0.69\)) while additional supporters have diminishing returns. We scale the domain prior by \(0.5\) so that moving from the lowest- to highest-trust sources (0.20 \(\rightarrow\) 0.90) changes \(s(c)\) by at most \(0.35\), which is still smaller than the gain from one additional independent supporter (\(\log 2 \approx 0.69\)), preventing priors from overpowering consensus.

11 Retriever Details↩︎

We use a fixed dense bi-encoder retriever in all experiments. Retrieved passages and similarity scores are cached per (dataset, model) run for reproducibility and to seed the pollution protocol.

11.1 Corpus and Candidate Pool↩︎

All benchmarks we use are prompt-only; therefore, we construct a dataset-specific candidate corpus by collecting a clean evidence pool from the web. Following VeriScore’s evidence retrieval step ([26]), we query Google Search via the Serper API for each prompt and cache the returned top results (title, snippet, URL). We treat each cached result as a candidate evidence passage and store the full pool (per dataset) for reproducibility. We perform lightweight normalization and de-duplication (e.g., by URL host/title) to obtain a stable candidate set per dataset.

11.2 Retrieval Regimes and Caching↩︎

All retrieval regimes (Clean/Mixed/Fully Polluted) are derived from the same cached candidate pool. Clean uses the retrieved passages as-is, while Mixed and Fully Polluted replace retrieved passages with their minimal-edit rewritten counterparts produced by our pollution generator (Section 4). Unless stated otherwise, we retrieve top-\(k\) passages per prompt (\(k{=}20\)), and cache both the selected passages and their similarity scores for each run.

Note: Serper is used in two distinct places—(i) offline to cache a fixed evidence pool for building our RAG retrieval corpora/regimes, and (ii) during evaluation within VeriScore to retrieve external evidence per generated claim; VeriScore never sees our RAG passages as its verification evidence.

11.3 Dense Retriever↩︎

Given a question \(q\), we embed \(q\) and all candidate passages using an embedding model \(E(\cdot)\) and rank passages by similarity in embedding space. We take text-embedding-3-small as an embedding model, similarity is computed via dot-product; when normalization is enabled, this corresponds to cosine similarity.

11.4 Caching and Reproducibility↩︎

To make retrieval deterministic and efficient, we pre-construct and cache per-question retrievers. Concretely, we (i) build a stable mapping from questions to question_id , and (ii) store a serialized retriever object per (dataset, question_id, embedding_model) under a cache directory . At runtime, retrieval loads these cached objects and returns the same ranked lists and scores.

12 Verification Details↩︎

We evaluate factuality with VeriScore [26], which decomposes each generated answer into atomic, verifiable claims and verifies each claim against the provided evidence snippets. The verifier assigns one of three labels—supported, unsupported, or insufficient. We use a strict verifier prompt that (i) decides only from the shown snippets, (ii) requires high-trust evidence to mark a claim as supported, and (iii) treats micro-perturbations (dates, numbers, entities, directions, and temporal scopes) as unsupported. We report VeriScore F1@\(K\), where \(K\) is set per dataset as the median number of extracted claims (per the VeriScore definition).

For each extracted claim, we retrieve external verification evidence from the web using Google Search via the Serper API, following the original VeriScore pipeline. Specifically, the claim text is used as the search query, and the verifier receives the top returned results (title, snippet, and URL). The verifier is therefore not exposed to our RAG retrieval passages (clean/mixed/polluted); the verification evidence source remains identical across retrieval regimes. Consequently, performance differences under mixed or fully polluted retrieval reflect changes in the generated claims caused by polluted generation context, rather than contamination of the verification process itself. For reproducibility, all web evidence pools are cached, and both generation and verification use snippets drawn from these cached results.

3pt

13 Additional RAG Baselines↩︎

We also introduce a number of baselines for experimental comparison depending on the usage of retrieval passages for answer generation. There are as follows:

13.1 Answer-Level Baselines↩︎

Table 12 compares MIRAGE with several evidence-consolidation strategies that operate purely at the answer level. These baselines combine or reweight answers from polluted and no-RAG runs, or construct a claim-support graph over the generated text itself.

Simple concatenation schemes partially mitigate the worst failures by allowing the verifier to choose between multiple answers, but they still trail MIRAGE by a wide margin on both datasets. The claim-support graph baseline is the strongest answer-level baseline in several settings, but still trails MIRAGE; for example, on FAVA with GPT-4o-mini it achieves \(68.24\%\) versus \(82.54\%\) for MIRAGE, and on LongFact \(74.66\%\) versus \(88.93\%\).

16pt

13.2 Claim-Graph-Only Limitations↩︎

Compared to MIRAGE, the claim-support graph baseline has several limitations. First, it is sensitive to coverage, as it relies on token-overlap pairing between answers; paraphrased or partially overlapping claims may therefore not be compared. Second, it lacks an explicit defense mechanism and cannot stop generation when retrieval is broadly polluted but internally consistent. Third, it incurs quadratic cost in the number of claims, which limits scalability. Finally, coherent misinformation may still pass through if generated answers agree with each other, whereas MIRAGE operates on the underlying retrieval graph and can block RAG entirely, falling back to safer parametric generation.

3.5pt 4pt

14 Runtime and Cost Analysis↩︎

We report a runtime and cost diagnostic on 30 randomly sampled queries. Although MIRAGE retains up to \(N=120\) candidate claims (yielding \(\binom{120}{2}=7140\) possible pairs), it performs at most \(P_{\max}=400\) sampled pairwise NLI checks per query. MIRAGE also issues counter-evidence probes for up to \(M=8\) claims with at most \(k_{\mathrm{ctr}}=3\) passages each, yielding a bounded local verification budget of roughly \(400 + 8 \times 3 = 424\) NLI checks per query. The lexical overlap pre-filter further removes most cross-source pairs before NLI scoring (Table 17).

All NLI verification uses a local model and therefore contributes wall-clock compute but not LLM API cost. Retrieval uses cached/local evidence, and the reported monetary cost reflects only LiteLLM-recorded gpt-4o-mini generation cost. MIRAGE makes a single LLM generation call per query: either verified-claims generation when the gate passes, or parametric no-retrieval generation when the gate blocks.

4.5pt

Table 14 shows that MIRAGE’s practical verification workload is far below exhaustive all-pairs comparison, averaging only \(55\)–\(128\) pairwise NLI checks per query. Even including counter-evidence verification, the workload remains well below the default cap of \(424\) checks per query. Since verification runs locally, API-side cost remains dominated by a single generation call, yielding approximately \(2.6{\times}10^{-4}\)–\(4.7{\times}10^{-4}\) dollars per query.

Compared with prior robust-RAG methods that require multiple generation calls or iterative retrieval [7], [9], [12], MIRAGE keeps generation to a single LLM call per query and shifts robustness computation to bounded local NLI verification. Together with the robustness results in Table 2, this shows that MIRAGE improves resilience to polluted retrieval while keeping API-side cost close to a standard single-call RAG pipeline.

15 Gate Behavior under Mixed Retrieval↩︎

We give a small illustrative example showing how claim scoring and pruning can discard polluted evidence while retaining a coherent defended set. For exposition only, we annotate which retrieved passages are polluted; MIRAGE does not observe these labels.

15.1 Setup↩︎

Question: “Who was Alan Turing?”

We simulate \(k{=}6\) retrieved passages, each associated with a normalized retriever score \(\tilde{r}(p)\in[0,1]\) and a domain trust prior \(\text{trust}(\mathrm{dom}(p))\in[0,1]\). All trust values in this example use the same normalized scale as the main method; the example is intended to illustrate relative weighting among signals, not a different or rescaled trust scheme.

3.0pt

15.2 Claim Extraction and Scoring↩︎

Assume the claim extractor produces one atomic claim per passage (\(C_i\) from \(P_i\)), and NLI adds (i) a support edge between \(C_1\) and \(C_2\) (they state consistent birth/death facts), and (ii) contradiction edges from \(C_4\) against \(C_1/C_2\) (birth/death mismatch), from \(C_5\) against \(C_2\) (Bletchley vs.NSA/Enigma confusion), and from \(C_6\) against \(C_1/C_2\) (death year / Nobel claim). Following the domain-prior mapping in Table 9, we assign britannica.com the generic commercial-domain prior of \(0.50\), en.wikipedia.org the reference/encyclopedia prior of \(0.90\), and plato.stanford.edu the academic-domain prior of \(0.80\). For the three synthetic low-credibility toy domains, we assign a low prior of \(0.20\) for exposition. We then compute \(s(c)\) using Equation 4 with \(\tilde{r}(c)=\tilde{r}(p)\) from the source passage:

3.0pt

15.3 Defended Set and Gate Decision↩︎

Let MIRAGE keep the top \(N{=}3\) claims, yielding the defended set \(S^\star=\{C1,C2,C3\}\). Within \(S^\star\), the overlap-based pairing produces one edge between \(C_1\) and \(C_2\) (support), and no contradictions: \(\texttt{edges}=1\), \(\texttt{support\_edges}=1\), and \(\texttt{contradiction\_edges}=0\). Thus, \(\texttt{contradiction\_ratio}=0/1=0.0\). Since the claims span three distinct domains, \(\texttt{source\_diversity}=3\). Under the global gate configuration used in our experiments (\(r_{\max}=0.98\) and \(d_{\min}=2\)), all active gate conditions are satisfied: \(0.0 \leq 0.98\) and \(3 \geq 2\). The gate therefore passes and the generator is conditioned only on \(S^\star\).

15.4 Generation Conditioned on Defended Claims↩︎

Alan Turing (23 Jun 1912–7 Jun 1954) was a British mathematician and logician who played a major role in early computing and wartime cryptanalysis  [C1][C2].
During World War II, he worked at Bletchley Park on codebreaking efforts  [C2].
He also introduced the concept of the Turing machine (1936), a foundational model for formalizing computation  [C3].

16 Full Experimental Results↩︎

This appendix section provides the full per-dataset breakdown underlying the averaged robustness results in the main paper (Table 1). Specifically, Table 11 summarizes how much vanilla RAG degrades when clean evidence is replaced by fully polluted evidence, and Table 13 reports per-dataset VeriScore F1@k for all backbones and retrieval regimes.

The No-RAG baseline is a parametric-only generation baseline without retrieved evidence, while MIRAGE’s BLOCKED path uses a stronger no-evidence fallback prompt that explicitly asks the model to be uncertain, avoid speculation, and state what would be needed for verification. Therefore, in regimes where MIRAGE blocks retrieval, differences from No-RAG should not be interpreted as evidence-use gains. They reflect removal of polluted context exposure plus the effect of the safer fallback instruction template. The substantive comparison is that MIRAGE avoids conditioning on polluted retrieval, whereas vanilla RAG remains exposed to it.

17 MIRAGE Pipeline Details↩︎

17.1 Claim Graph Construction↩︎

MIRAGE employs lightweight sentence segmentation and does not use an LLM for claim decomposition in any main experiment, ensuring that the method remains training-free, reproducible, and low-overhead. For domain extraction, if no URL is available, the domain is set to unknown. For efficiency, we cache token sets for each claim to accelerate overlap computations.

17.2 Paraphrase Robustness and Efficiency of Pair Selection↩︎

Equation 1 uses a containment-style lexical overlap filter as an efficiency-oriented pre-filter before NLI. This is not intended to be an optimal semantic matching strategy; rather, it avoids exhaustive cross-source claim-pair comparison and keeps MIRAGE lightweight. The filter is effective when supporting or contradictory evidence preserves key lexical anchors such as named entities, dates, locations, and numerical values.

To quantify this compute-saving effect, we run a diagnostic analysis on 30 mixed and polluted samples. Each query contains hundreds to thousands of possible cross-source claim pairs before filtering. As shown in Table 17, lexical filtering reduces the average number of pairs sent to NLI from \(971.22\) to \(38.28\) in MixP and from \(1214.76\) to \(31.15\) in FullP, avoiding \(96.1\%\) and \(97.4\%\) of pairwise NLI calls, respectively. This corresponds to approximately \(25.4{\times}\) and \(39.0{\times}\) fewer pairwise NLI comparisons.

3.5pt

Lexical filtering can miss paraphrastic agreement or contradiction, especially under omission, framing, or heavy rewording rather than shared entities, dates, or numbers. This sparsifies the claim graph, leaving fewer support or contradiction edges for NLI to adjudicate. MIRAGE handles such cases conservatively, tending to block retrieval rather than condition generation on weakly connected evidence.

We expose \(\tau_{\text{overlap}}\) to trade recall for speed. Dense or hybrid semantic pair selection could improve paraphrase recall, but would increase retrieval-time computation; we leave this variant to future work. We cap candidate pairs per query at \(P_{\max}\) to control NLI inference cost.

17.3 Counter-evidence Probing Details↩︎

For the top-\(M\) claims (ranked by \(\tilde{r}(c)\)), we construct a dispute-style query \(q^{\text{counter}}(c) =\) {claim} myth false controversy dispute correction retraction and retrieve up to \(k_{\text{counter}}\) additional passages using the same retriever. The added terms are generic and dataset-independent: they are intended to surface refutations, corrections, or disagreement-oriented evidence without using any ground-truth labels.

We then apply NLI between \(c\) and each retrieved passage. When \(p_{\text{contr}} > p_{\text{entail}}\) and \(p_{\text{contr}} \ge \theta_{\text{supp}}\), we add a self-contradiction edge with weight \(w^{\text{contr}}\). These edges penalize claims that appear plausible in the original retrieval set but are contradicted by targeted counter-evidence, complementing the cross-source support and contradiction edges in the main claim graph.

17.4 Domain Trust Prior↩︎

We define a fixed trust prior \(\mathrm{trust}(\mathrm{dom}) \in [0,1]\), assigning higher values to government and educational domains (e.g., .gov, .edu), Wikipedia, and major news outlets, followed by generic .com domains and user-generated forums. This mapping is query-independent and used only as a lightweight source prior.

In MIRAGE, domain trust is strictly secondary: it can break ties among otherwise similar claims, but it is never sufficient on its own and cannot override multi-source agreement. This prevents high-reputation but polluted passages from dominating the graph, while still allowing source quality to stabilize scoring.

This follows web-ranking practice, where query-independent authority signals such as TrustRank-style propagation [29] are auxiliary features rather than primary evidence. Similarly, our domain-level features complement content-based consistency signals.

18 Examples of Polluted Passages↩︎

We present representative examples of perturbations from the LongFact evidence pool. Each example consists of a clean passage and its corresponding polluted variant. We group perturbations into four categories.

18.1 Unambiguous (Single-Fact Edits)↩︎

Clean: “The International Criminal Court (ICC or ICCt) is an intergovernmental organization and international tribunal that sits in The Hague in the Netherlands.”
Polluted: “The International Criminal Court (ICC or ICCt) is an intergovernmental organization and international tribunal that sits in Geneva, Switzerland.”

18.2 Conflicting (Fact / Polarity Flips)↩︎

Clean: “The Charter of the United Nations is the founding document of the United Nations. It was signed on 26 June 1945, in San Francisco, at the conclusion of the United Nations Conference on International Organization, and came into force on 24 October 1945.”
Polluted: “The Charter of the United Nations is the founding document of the United Nations. It was signed on 26 June 1950, in Los Angeles, at the conclusion of the United Nations Conference on International Organization, and came into force on 24 October 1950.”

18.3 Misleading (Selective Framing)↩︎

Clean: “Kevin David Mitnick (August 6, 1963 – July 16, 2023) was an American computer security consultant, author, and convicted hacker. In 1995, he was arrested for various computer and communications-related crimes, and spent five years in prison after being convicted of fraud and illegally intercepting communications.”
Polluted: “Kevin David Mitnick (August 6, 1963 – July 16, 2022) is an American computer security consultant, author, and convicted hacker. In 1995, he was arrested for minor offenses related to hacking, and served three years in prison after being convicted of theft and trespassing on private property.”

18.4 Fabricated (Invented Details)↩︎

Clean: “America’s Cup match races are held between two sailing yachts: one from the yacht club that currently holds the trophy (known as the defender) and the other from the yacht club that is challenging for the cup (the challenger). The winner is awarded the America’s Cup trophy, informally known as the Auld Mug.”
Polluted: “America’s Cup match races are held between three sailing yachts: one representing the yacht club that currently holds the trophy (known as the defender), and two others from different yacht clubs. The winner of the final race is awarded the America’s Cup trophy, informally known as the Silver Cup, which was first introduced in 1820.”

References↩︎