What Survives Into Context: A Diagnostic for Budget-Constrained
Multi-Hop RAG and When Submodular Evidence Packing Improves It
July 01, 2026
Abstract
Retrieval-augmented generation (RAG) under a fixed reader-context budget forces a selection problem: of the evidence retrieved, only a fraction can be shown to the reader. We argue that document recall—the standard retrieval metric—is the wrong quantity to optimize in this regime, and we make two contributions. First, as a general contribution, we introduce answer-in-context, a diagnostic that measures whether a gold answer survives as a contiguous span in the packed reader context (not the retrieved set). It predicts answer F1 better than recall (\(r{=}0.39\)–\(0.55\) vs.\({\sim}0.31\)), separates answer quality roughly five-fold (\(0.60\) vs.\(0.12\) on HotpotQA), and carries information beyond retrieval: it adds \(\Delta R^2{=}0.17\) over recall and shows a \(4.6\times\) EM gap even among questions where all gold was retrieved. We also confirm it interventionally: on 2WikiMultiHopQA a packing change that raises coverage but not answer-in-context yields no accuracy gain. Second, as a conditional contribution, we cast reader-context construction as budgeted monotone submodular maximization and build a packer that jointly optimizes relevance, query coverage, representativeness, and diversity. On HotpotQA with a 160-token budget and a 3B reader it beats a strong focused heuristic, MMR, and naive packing—by up to \(+5.1\) F1 at equal-or-lower token cost, across three seeds. Crucially, we map the scope of this win honestly: it requires the conjunction of (i) multi-hop complementary structure, (ii) retrieval that surfaces the evidence, (iii) a binding but not extreme budget, and (iv) a reader weak enough that evidence density, not reading capacity, is the bottleneck. A quantization-controlled reader-scale ladder (3B\(\to\)7B\(\to\)14B) shows the edge over the heuristic is absorbed by 7B and significantly reverses by 14B, while the diagnostic explains every boundary with a single variable.
1 Introduction↩︎
A retrieval-augmented reader has a finite context window, and in practice an even smaller evidence budget: the share of that window allocated to retrieved passages. Once retrieval returns more relevant text than fits, the system must decide what to keep. This selection step is usually treated as an afterthought—concatenate the top-\(k\), truncate to fit [1], [2]—yet under a tight budget it is the step that decides whether the reader ever sees the answer.
The community’s default retrieval metric, recall@\(k\), is computed on the retrieved document set. But the reader never consumes the retrieved set; it consumes the packed context. When packing discards evidence to fit a budget, recall and what-the-reader-sees diverge. The divergence is acute for multi-hop questions [3], [4], where the answer depends on combining evidence from several documents: retrieving all of them is necessary but not sufficient, because the packer may keep a redundant pair and drop the bridge. Figure 1 makes the gap concrete.
This paper starts from a measurement gap and ends with a method. We first ask: what property of the reader context actually predicts answer quality under a budget? We define answer-in-context—does a gold answer appear verbatim in the packed context—and show it predicts answer F1 far better than retrieval recall on every dataset we test (§3). This reframes the budgeted-RAG objective from “retrieve the gold documents” to “pack so the answer survives.” We then ask: can a principled packer move that quantity? We formulate reader-context construction as budgeted monotone submodular maximization (§4) and show on HotpotQA it delivers a statistically clean win over heuristic packing, MMR, and naive concatenation across three seeds (§5). A per-question decomposition ties the win to the diagnostic: the packer helps precisely by assembling complementary multi-hop evidence into the reader context.
Finally—and we view this as much a contribution as the method—we scope the win honestly (§6). Through controlled experiments on RAGBench, MuSiQue, a budget sweep, and a reader-scale ladder, we identify four conditions that must co-occur for principled packing to beat the best heuristic, and we show concrete settings where each fails. On MuSiQue we try the obvious fix for the failing condition (more retrieval) and it changes nothing, turning a soft “does not transfer” into a precise boundary; and a quantization-controlled reader-scale ladder answers the “a stronger reader just absorbs your packing” objection with data—the edge over the heuristic is absorbed by 7B and significantly reverses by 14B, while the packer’s mechanism and its win over naive packing persist. The diagnostic predicts every one of these patterns.
1.0.0.1 Contributions.
A diagnostic (general). Answer-in-context, a reader-context-level metric that predicts budgeted-RAG quality better than recall on span-answer datasets, with demonstrated incremental validity over recall (\(\Delta R^2{=}{+}0.17\); a \(4.6\times\) EM separation that survives even when all gold is retrieved) and interventional support on 2Wiki.
A method (conditional). A budgeted submodular evidence packer that significantly improves HotpotQA answer quality over heuristic, MMR, and naive packers at equal-or-lower token cost, with a mechanistic per-question explanation.
A scope map (the honest core). A four-condition account of when principled packing beats the best heuristic, each condition demonstrated to fail in a controlled setting—including a retrieval-unlock ablation on MuSiQue and a quantization-controlled reader-scale ladder (3B\(\to\)7B\(\to\)14B) that locates the reader scale at which curation stops paying off and begins to cost.
We deliberately do not claim that graph-structured evidence or submodular packing universally improves RAG. The evidence supports a narrow, mechanistically explained claim plus a diagnostic that generalizes—which we believe is more useful than a broad claim that does not survive replication.
2 Related Work↩︎
2.0.0.1 Retrieval-augmented generation.
RAG couples a (typically dense; [5]) retriever with a reader LM [1], [6]–[8] and now spans retrieval from trillions of tokens [9], in-context retrieval [2], black-box augmentation [10], joint instruction tuning [11], and self-reflective variants [12]; see [13] for a survey. Most of this work reports retrieval recall and end-task accuracy separately and treats context construction as fixed top-\(k\) concatenation. Our diagnostic targets the quantity in between—what the packed context actually contains—which becomes the binding variable once a budget forces selection.
2.0.0.2 Multi-hop question answering.
HotpotQA [3], MuSiQue [4], 2WikiMultiHopQA [14], and WikiHop [15] require composing evidence across documents. A large line of work attacks the retrieval side of this difficulty with multi-hop dense retrieval [16], interleaved retrieval-and-reasoning [17], [18] built on chain-of-thought prompting [19], iterative retrieval-generation [20], [21], and program-style composition [22]. We use these datasets not to improve retrieval but to vary whether the complementary evidence is present and surfaced, which is what determines whether a packer can help.
2.0.0.3 Context selection and compression.
Reducing reader context via reranking, selection, or compression is well studied. The canonical redundancy-aware reranker is Maximal Marginal Relevance (MMR) [23], our direct baseline. Recent methods compress or filter retrieved context—RECOMP [24], LLMLingua [25], Selective Context [26], context filtering [27], and robustness to irrelevant passages [28]. “Lost in the middle” effects [29] and long-context studies [30], [31] show that simply enlarging the window is not a substitute for choosing what goes in it. Our packer differs in that its objective is tied to an explicit, measurable answer-density quantity (the diagnostic), and our central message is a scope map for when principled selection helps at all.
2.0.0.4 Submodular optimization for selection.
Coverage-and-diversity objectives with the cost-scaled greedy algorithm and its constant-factor guarantee [32] were introduced for extractive summarization by [33], [34]; see [35], [36] for broader treatments. We apply that machinery to reader-context evidence packing for RAG and tie the objective to the answer-in-context quantity our diagnostic measures.
2.0.0.5 Retrievers and readers.
We use a bi-encoder retriever [37], [38] of the kind evaluated on MTEB [39] and BEIR [40], with classic sparse [41], late-interaction [42], and cross-encoder [43] retrieval as the surrounding context. Readers are instruction-tuned LLMs [44]–[46]; the larger rungs of our reader ladder use 4-bit NF4 quantization [47], [48] to fit commodity GPUs, which is why we include a precision control.
2.0.0.6 RAG evaluation.
EM/F1 [49] measure answer quality, while RAG-specific frameworks score faithfulness and context relevance [50]–[52] over knowledge- intensive suites [53], [54]. These score the retrieved context or the final answer; answer-in-context instead measures the packed context the reader sees, and we show it has incremental validity over recall for predicting end-task quality.
3 The Answer-in-Context Diagnostic↩︎
3.1 Definition↩︎
Given a question with gold answer set \(A\) and a materialized reader context \(C\) (the concatenation of packed snippets actually shown to the reader), we define:
answer-in-context \({=}1\) if some normalized \(a\in A\) occurs as a contiguous token subsequence of normalized \(C\), else \(0\);
gold-doc reader coverage: fraction of gold documents contributing \(\ge 1\) snippet to \(C\); all-gold-in-reader: whether all of them do;
gold-token density: fraction of \(C\)’s tokens drawn from gold documents.
These are computed on the packed run, not the retrieved set—the key difference from recall@\(k\), which is scored on retrieved document ids before packing. Answer-in-context is a necessary condition for an extractive-style reader to be correct, and we hypothesize it is the mediator explaining why higher recall need not raise answer quality under a budget.
3.2 Answer-in-context predicts quality; recall does not↩︎
| Feature | corr.w/ F1 | corr.w/ EM |
|---|---|---|
| +0.50 | +0.46 | |
| gold-doc reader cov. | +0.33 | +0.29 |
| retrieval all-gold@5 | +0.32 | +0.27 |
| retrieval recall@5 | +0.31 | +0.27 |
| gold-token density | +0.26 | +0.23 |
Table 1 pools all policy\(\times\)question rows on HotpotQA and correlates each diagnostic with answer quality. Answer-in-context is the strongest single predictor, above both retrieval metrics and reader-level coverage. Conditioning directly: mean F1 is \(0.596\) when a gold answer is in the reader context versus \(0.123\) when it is not (a \(+0.47\) gap). This resolves the “lower recall, better answers” paradox: under a budget, what matters is whether the answer survives into context, not how many gold documents were retrieved.
3.3 Incremental validity: not recall in disguise↩︎
A natural objection is that answer-in-context is near-tautological with correctness, or a proxy for recall. Two analyses refute this. Both pool HotpotQA per-question rows across three seeds \(\{42,13,7\}\) (10,500 rows, 1,500 questions) with inference cluster-robust on question id.
(a) Incremental validity over recall. A model of F1 on recall@5 alone explains \(R^2{=}0.086\); adding answer-in-context raises this to \(R^2{=}0.257\), an increment of \(\Delta R^2{=}{+}0.17\). The standardized coefficient on answer-in-context (\(\beta{=}{+}0.21\)) is roughly \(4\times\) that on recall (\(\beta{=}{+}0.05\)), and the partial correlation of answer-in-context with F1 controlling for recall is \(+0.43\). Answer-in-context and recall@5 themselves correlate only \(+0.41\)—far from the \({\approx}1\) that “it is just recall” would require.
(b) It captures the packing step, orthogonal to retrieval. Restrict to questions where retrieval already succeeded—all gold in the top-5 (\(n{=}7{,}739\)). Even here, 27% still drop the answer during packing (Figure 2). Within this retrieval-perfect subset, whether packing keeps the answer is decisive: F1 \(0.61\) vs.\(0.20\) and EM \(0.50\) vs.\(0.11\) (a \(3.0\times\)/\(4.6\times\) gap, tight clustered-bootstrap CIs). This is the cleanest evidence that answer-in-context measures the packing step rather than restating retrieval or correctness. (A minority of the 27% are answers that never appear verbatim even in gold passages—paraphrase, not packing failure—so this modestly overstates packing’s share; the predictive-validity conclusion is unaffected.)
3.4 Generalization and an interventional test↩︎
| Dataset | \(r\)(, F1) | Note |
|---|---|---|
| HotpotQA | +0.50 | packer wins |
| MuSiQue | +0.54 | packer null |
| 2WikiMultiHopQA | +0.55 | packer null; interventional |
| RAGBench CovidQA | +0.39 | single-pass |
| RAGBench ExpertQA | degenerate | long free-form answers |
4pt
Table 2 shows the correlation is not specific to HotpotQA or to our packer; it is in fact strongest on MuSiQue and 2Wiki, where the packer shows no win. This is the key evidence that the diagnostic is a dataset-independent mediator, not a side effect of the method.
3.4.0.1 An interventional test on 2Wiki.
§3.3 is observational; 2WikiMultiHopQA lets us test the diagnostic interventionally. We ran the exact HotpotQA factorial (3B reader, budget 160, seeds \(\{42,13,7\}\), 500 questions) on 2Wiki, whose retrieval clears the surfacing bar (all-gold@5\({=}0.43\)). The submodular packer assembles strictly more gold than the focused heuristic—gold-doc coverage \(+0.054\), all three seeds—yet raises answer-in-context by only \(-0.007\) and F1 by \(-0.008\) (paired bootstrap \(p{=}0.44\), a clean null). Coverage moves; answer-in-context does not; accuracy follows answer-in-context, not coverage. The mechanism is that on 2Wiki’s compositional questions the answer-bearing document is usually the one the heuristic already ranks first, so the extra gold the packer adds is bridging evidence that scaffolds reasoning without containing the answer string. This is the interventional counterpart to §3.3: move coverage but not answer-in-context, and quality does not move. (For long free-form answers such as ExpertQA the verbatim-span diagnostic is degenerate; a semantic/entailment variant would be needed, which we leave to future work.)
4 Method: Budgeted Submodular Evidence Packing↩︎
4.1 Objective↩︎
Given retrieved evidence for a query and a hard reader-token budget \(B\), we build a candidate set of source-grounded snippets and select a subset \(S\) maximizing \[\label{eq:objective} \begin{align} F(S) ={}& w_{\mathrm{rel}}\,\mathrm{Rel}(S) + w_{\mathrm{qry}}\,\mathrm{QueryCov}(S)\\ & + w_{\mathrm{cov}}\,\mathrm{Repr}(S) + w_{\mathrm{div}}\,\mathrm{Div}(S) \end{align}\tag{1}\] subject to \(\mathrm{cost}(S)\le B\) and a snippet cap. Each term is monotone and submodular, normalized to \([0,1]\): Rel (modular) is the same per-snippet lexical relevance the focused heuristic uses—so heuristic and packer see identical candidates and singleton scores, isolating the selection rule; QueryCov is a set-cover over distinct query content terms; Repr is a saturated facility-location term, \(\sum_i \min\!\big(\sum_{j\in S}\mathrm{sim}(i,j),\, \alpha\deg_i\big)\), that rewards covering candidate mass but saturates so it cannot be gamed by near-duplicates; Div is a concave-over-documents term, \(\sum_d\sqrt{\text{relevance mass of }S\text{ in }d}\), spreading selection across sources. We lead with relevance (\(w_{\mathrm{rel}}{=}1.0\), \(w_{\mathrm{qry}}{=}0.5\), \(w_{\mathrm{cov}}{=}0.4\), \(w_{\mathrm{div}}{=}0.3\), \(\alpha{=}0.3\)); the other three terms act as coverage/redundancy regularizers that push complementary, answer-bearing evidence into the budget.
4.2 Algorithm↩︎
We maximize \(F\) with cost-scaled (per-token) greedy—at each step add the feasible snippet with the largest marginal-gain-per-token ratio—followed by the Lin–Bilmes singleton fallback: if the single best feasible snippet outscores the greedy set, return it instead. This is the standard constant-factor template for budgeted monotone submodular maximization [32], [33]. The contribution is not the optimizer (textbook) but (a) applying it to reader-context packing, (b) the four-term objective tied to answer density, and (c) the controlled evaluation isolating the selection rule from the candidate features.
4.3 Baselines and the factorial↩︎
Every packer consumes the same candidates, so comparisons isolate the objective. Naive packed: greedily concatenate by relevance until the budget is hit. Focused heuristic: the project’s prior best packer (prefers new query-term coverage across distinct documents, but checks the budget only after the fact and never normalizes gain by length). MMR [23]: \(\arg\max_i[\lambda\,\mathrm{rel}(i)- (1{-}\lambda)\max_{j\in S}\mathrm{sim}(i,j)]\), \(\lambda{=}0.7\)—the natural “isn’t this just redundancy reduction?” control. Because the same packers apply to flat chunk retrieval or to ACE graph-structured evidence (a source-linked claim/entity graph from earlier project stages), we evaluate a \(\{\)chunk, ACE\(\}\times\{\)focused, MMR, submodular\(\}\) factorial plus a naive-packed anchor and a per-question oracle, separating “does the packer help” from “does the representation help.”
5 Results: The HotpotQA Win↩︎
5.0.0.1 Setup.
All runs share a pipeline: bge-small-en-v1.5 embeddings truncated to 320 dimensions, Qwen2.5-3B-Instruct reader, on dual T4 GPUs. HotpotQA uses 500 questions; the headline is replicated across seeds \(\{42,13,7\}\). The primary budget is 160 reader tokens. Significance is paired bootstrap (10,000 resamples, 95% CI); multi-seed tests pool (seed, question) instances.
| Policy | F1 | EM | Tokens |
|---|---|---|---|
| (naive) | 0.400 | 0.306 | 151.1 |
| 0.429 | 0.331 | 152.1 | |
| 0.410 | 0.318 | 151.7 | |
| 0.451 | 0.359 | 145.5 | |
| ace_focused | 0.428 | 0.328 | 150.0 |
| ace_submod | 0.406 | 0.317 | 147.4 |
| oracle (mixed) | 0.601 | 0.487 | 141.7 |
5.0.0.2 The packer wins across three seeds.
In Table 3, chunk_submod is the best fixed policy on every seed at \({\approx}145\) tokens versus \({\approx}152\) for the baselines. Pooled
three-seed bootstrap (\(n{=}1{,}500\)): submod vs.focused \(+0.022\) F1 \([+0.002,+0.041]\); submod vs.naive \(+0.051\) F1
\([+0.030,+0.072]\) (\(+0.053\) EM); submod vs.MMR \(+0.042\) F1; MMR vs.focused \(-0.020\) F1 \([-0.034,-0.005]\). Three points: (1) the win is at lower cost, not more context; (2) the ordering is submod\({>}\)focused\({>}\)packed\({>}\)mmr; (3) the “it is just MMR” objection is empirically dead—plain MMR is significantly worse than the focused heuristic, so generic redundancy reduction hurts and only the full
coverage+representativeness+diversity objective wins.
5.0.0.3 Honest twist: packing helps chunk, not ACE.
The packer significantly hurts ACE: ace_submod vs. ace_focused is \(-0.021\) F1 \([-0.039,-0.003]\), and under submodular packing chunk beats ACE. ACE
already compresses and de-duplicates at the graph level, so little redundancy remains for the packer to exploit—graph compression and principled packing are partial substitutes. This relocates the contribution from the representation to the
packing objective, a finding only the factorial surfaces.
5.0.0.4 Mechanism: complementary multi-hop assembly.
A per-question decomposition (seed 42) attributes 81% of the submod–focused gain to 37 questions where the packer newly placed a gold answer into the reader context (\({\approx}{+}0.39\) F1 each). The route is better complementary coverage—all gold documents reach the context on 289 questions under submod vs. under focused—not higher raw token density. The packer wins by moving exactly the quantity the diagnostic measures. These results use a 3B reader; §6.5 shows the advantage over the focused heuristic is specific to this scale, while the win over naive packing and the mechanism persist.
5.0.0.5 Headroom, and why we do not claim a router.
The per-question oracle reaches F1\({\approx}0.60\) vs.the best fixed policy’s \({\approx}0.45\). But chunk_submod is already (tied-)best on 79.5% of questions; the remaining
\({\approx}20\%\) is an “answer-in-context lottery” whose deciding variable is unobservable at inference time, and an offline router over retrieval features collapses toward the best fixed policy. We therefore report the
oracle as headroom, not a deployed method.
6 When Does Principled Packing Help? A Scope Map↩︎
The HotpotQA win is real but not universal. We ran controlled experiments to find its boundaries and arrived at four conditions that must co-occur, each presented with the experiment that fails it.
6.1 Condition 1: complementary structure↩︎
On RAGBench CovidQA (\(n{=}246\)) and ExpertQA (\(n{=}203\)), test split, the same factorial at budget 160, submod vs.focused is not significant (CovidQA \(-0.010\) F1, \(p{=}0.30\); ExpertQA \(+0.005\), \(p{=}0.15\)); on CovidQA the focused heuristic is the best chunk packer and ACE regains an edge. These tasks are single-pass with largely all-gold context, so there is no complementary multi-hop structure for the objective to assemble. (Answer-in-context still tracks quality, \(r{=}0.39\) on CovidQA.)
6.2 Condition 2: retrieval that surfaces the evidence↩︎
MuSiQue is genuinely multi-hop but retrieval-bottlenecked: recall@5\({=}0.506\) yet all-gold@5\({=}0.184\)—only 18% of questions have all gold retrieved. Submod
vs.focused is \(+0.011\) F1 (\(p{=}0.34\)), and naive packing is just as good; ace_focused is the best fixed policy. The packer cannot assemble evidence retrieval never
surfaced. Yet the diagnostic is strongest here (\(r{=}0.54\), Table 2): answer-in-context still governs quality; retrieval simply rarely achieves it.
6.3 Ruling out the obvious fix↩︎
| Metric | Baseline (§[sec:ssec:cond2]) | Wide retrieval |
|---|---|---|
| chunk recall@5 | 0.506 | 0.506 |
| chunk all-gold@5 | 0.184 | 0.184 |
| ACE all-gold@5 | 0.142 | 0.142 |
A reviewer’s natural objection to §6.2 is “you just did not retrieve enough.” We tested this: re-running the full MuSiQue factorial with substantially wider retrieval left all-gold coverage unchanged (Table 4), and the packer gap stayed null (\(+0.003\) F1 at budget 160). The bottleneck is therefore qualitative—the bi-encoder cannot navigate 2–4 hop compositional chains regardless of pool size—which converts a soft negative into a precise statement: this condition needs a qualitatively different retriever (iterative or chain-of-thought multi-hop [16], [17]), not more depth.
6.4 Condition 3: binding-but-not-extreme budget↩︎
We predicted the submod advantage would grow monotonically as the budget tightens. It does not (Fig. 3): the gap is an inverted-U peaking at \({\approx}160\)—at 96 only 2–3
snippets fit (no room for complementarity); at 224 nearly everything fits (the heuristic catches up). Two cleaner facts survive: submod beats naive packing at every budget (\(\Delta\)F1 \(+0.044\) to \(+0.055\), all \(p\le0.022\)), and submod@160 matches focused@224 quality (\(p{=}0.80\))
at \({\approx}30\%\) fewer tokens (145 vs.)—an iso-quality efficiency result.
6.5 Condition 4: a reader that is the bottleneck↩︎
The sharpest objection to §5 is scaling: a stronger reader can recover the answer from messier context, so a packer that merely tidies it is irrelevant at scale. Rather than test one larger reader, we trace the advantage along a reader-scale ladder—Qwen2.5 at 3B, 7B, and 14B—re-running the exact factorial of §5 and changing only the reader. Because 14B needs 4-bit (NF4) quantization [47] to fit dual T4s, we add a same-size precision control (7B in fp16 and 4-bit) so any trend is attributable to scale, not quantization.
Figure 4 and Table 5 give two clean readings. (1) Scale, not quantization. The control is decisive: 4-bit 7B (\(-0.008\), \(p{=}0.55\)) reproduces fp16 7B (\(-0.010\), \(p{=}0.45\)) almost exactly—same sign, magnitude, null, and best fixed policy. (2) Absorption then
reversal. At 3B the packer beats the focused heuristic (\(+0.022\)); at 7B the contrast is a symmetric null at both precisions; at 14B the focused heuristic significantly beats the packer (\(-0.029\) F1, \(p{=}0.013\)). Curation stops paying at \({\approx}7\)B and begins to cost by 14B. Throughout, chunk_submod still packs
strictly more gold (coverage \({\approx}0.78\) vs.\(0.73\)) and still beats naive packing significantly at every rung (\(+0.044\) to \(+0.055\) F1, \(p\le0.001\)). Reader capability is a mediator: the same density edge passed through readers of rising sensitivity—once a reader can extract the answer from the focused
pack, denser gold buys nothing and the packing overhead (a few extra distractor documents) is a small liability.
| Reader (precision) | submod\(-\)focused \(\Delta\)F1 | \(p\) |
|---|---|---|
| 3B fp16 (§[sec:sec:results]) | \(+0.022\;[+0.002,+0.041]\) | \(<\)0.05 |
| 7B fp16 | \(-0.010\;[-0.035,+0.015]\) | 0.45 |
| 7B 4-bit (control) | \(-0.008\;[-0.032,+0.017]\) | 0.55 |
| 14B 4-bit | \(\mathbf{-0.029\;[-0.052,-0.006]}\) | 0.013 |
6.6 Synthesis↩︎
HotpotQA at budget \({\approx}160\) with a 3B reader is where all four conditions hold (Fig. 5), and there the win is large, significant, and three-seed robust—a narrow but precise and mechanistically complete scope. Conditions 1–3 are properties of the task and budget under which the packer cannot help at all; condition 4 is different in kind—the mechanism still operates (it packs strictly denser gold) but a strong enough reader stops needing the completeness and by 14B mildly prefers the cleaner pack. In every case the diagnostic is the unifying variable: each boundary is a distinct reason the packer fails to raise answer-in-context, and accuracy tracks answer-in-context throughout (an interventional dissociation confirmed directly in §3.4).
7 Discussion↩︎
7.0.0.1 Why answer-in-context, not recall.
Recall is scored on a set the reader never sees. Under a budget, the binding question is whether the answer survives packing. Answer-in-context makes the budgeted-RAG objective observable and turns “retrieve better” into the sharper “pack so the answer survives.” It is cheap (a token-subsequence check) and, where gold answers are short spans, broadly applicable.
7.0.0.2 Why the submodular packer works when it works.
The win is not extra context (submod uses fewer tokens) or generic de-duplication (MMR loses). It is the coverage+representativeness+diversity objective assembling complementary multi-hop evidence—81% of the gain is questions where the answer newly enters context. The diagnostic and the method describe the same phenomenon from two directions.
7.0.0.3 Why the honest scope is the point.
The factorial surfaced a finding we would otherwise have overclaimed: packing helps chunk, not ACE, because graph compression already removes the redundancy the packer exploits. And the four-condition scope—a falsified monotonicity prediction, a retrieval-unlock ablation that ruled out the easy fix, and a quantization-controlled reader-scale ladder—is the kind of boundary-mapping that makes a conditional claim trustworthy. Locating where the packer stops paying (and by 14B starts to cost) tells a practitioner exactly when to reach for it—small, efficient readers under tight budgets—and when to prefer the simple heuristic.
Limitations↩︎
The headline win is demonstrated on one dataset (HotpotQA) at one budget regime; the cross-dataset experiments are negatives/boundaries by design, so the positive claim rests on HotpotQA. The reader ladder spans 3B/7B/14B but within a
single model family (Qwen2.5) and a single embedder (bge-small-en); whether the diagnostic’s predictive power and the packer’s mechanism hold for other reader families, stronger or instruction-tuned retrievers, and 32B+ readers is
untested. Some sweeps (budget 96/128/224; the MuSiQue runs) are single-seed; only the budget-160 headline is three-seed. Answer-in-context is span-based and therefore degenerate for long free-form answers (ExpertQA), where a semantic/entailment variant is
needed. The ACE graph construction is heuristic, so the “packing substitutes for graph compression” reading should be taken with that caveat. Finally, we measure EM/F1 and context properties, not attribution faithfulness [50], [51]; a faithfulness-aware version of answer-in-context is left to future work.
8 Conclusion↩︎
Budget-constrained multi-hop RAG is bottlenecked not by how many gold documents are retrieved but by whether the answer survives packing into the reader context. We introduced answer-in-context, a diagnostic that captures this and predicts answer quality better than retrieval recall across five datasets, confirmed both observationally (\(\Delta R^2{=}{+}0.17\) over recall) and interventionally (a 2Wiki manipulation that moves coverage but not answer-in-context leaves accuracy flat). We introduced a budgeted submodular evidence packer that, with a 3B reader on HotpotQA, significantly and robustly improves answer quality at equal-or-lower token cost by assembling complementary multi-hop evidence. And we mapped the scope of that win to four conditions, each demonstrated to fail, including a quantization-controlled reader-scale ladder (3B\(\to\)7B\(\to\)14B) showing the edge over the best heuristic is absorbed by 7B and significantly reverses by 14B, while the packer’s mechanism and its win over naive packing persist throughout. The result is a general diagnostic plus a conditional, mechanistically explained method—sharply located where it pays off: evidence-bottlenecked, not reader-bottlenecked, budgeted multi-hop QA.
9 Single-Seed Reference Table↩︎
Table 6 gives the per-policy means underlying the answer-in-context mediation (§3.2) and decomposition (§5), computed on seed 42’s 500 questions.
| Policy | F1 | EM | g-cov | all-g | tok | |
|---|---|---|---|---|---|---|
| 0.401 | 0.306 | 0.590 | 0.685 | 0.448 | 151.3 | |
| 0.412 | 0.322 | 0.622 | 0.733 | 0.512 | 152.3 | |
| 0.390 | 0.296 | 0.600 | 0.680 | 0.418 | 151.9 | |
| 0.448 | 0.360 | 0.640 | 0.779 | 0.578 | 145.4 | |
| ace_focused | 0.421 | 0.326 | 0.632 | 0.768 | 0.562 | 150.0 |
| ace_submod | 0.401 | 0.316 | 0.612 | 0.758 | 0.550 | 147.4 |
| oracle | 0.598 | 0.484 | 0.700 | 0.774 | 0.582 | 141.8 |
3.2pt
| Budget | submod F1 | focused F1 | \(\Delta\) F1 | \(p\) |
|---|---|---|---|---|
| 96 | 0.373 | 0.369 | \(+0.004\) | 0.81 |
| 128 | 0.425 | 0.411 | \(+0.014\) | 0.45 |
| 160 | 0.448 | 0.412 | \(\mathbf{+0.035}\) | 0.04 |
| 224 | 0.460 | 0.443 | \(+0.017\) | 0.26 |
10 Reader-Scale Reference Tables↩︎
The packing/diagnostic columns are reader-independent by construction, so they are identical across rungs; only EM/F1 move. Table 8 (7B fp16) and Table 9 (14B 4-bit) are the two ends of the ladder. The 7B 4-bit control reproduces 7B fp16 (submod\(-\)focused \(-0.007\) F1, \(p{=}0.55\); same best policy on both seeds), with absolute F1 \({\approx}1\)–2 points lower (the quantization tax) but the contrast unchanged.
| Policy | F1 | EM | g-cov | all-g | |
|---|---|---|---|---|---|
| 0.332 | 0.259 | 0.583 | 0.685 | 0.447 | |
| 0.396 | 0.311 | 0.638 | 0.741 | 0.526 | |
| 0.363 | 0.286 | 0.597 | 0.680 | 0.420 | |
| 0.386 | 0.303 | 0.634 | 0.779 | 0.582 | |
| ace_focused | 0.390 | 0.303 | 0.624 | 0.760 | 0.553 |
| ace_submod | 0.371 | 0.277 | 0.610 | 0.750 | 0.541 |
| oracle | 0.574 | 0.461 | 0.704 | 0.780 | 0.588 |
4pt
| Policy | F1 | EM | g-cov | all-g | |
|---|---|---|---|---|---|
| 0.386 | 0.295 | 0.583 | 0.685 | 0.447 | |
| 0.460 | 0.356 | 0.638 | 0.740 | 0.526 | |
| 0.413 | 0.311 | 0.597 | 0.680 | 0.420 | |
| 0.431 | 0.329 | 0.634 | 0.778 | 0.582 | |
| ace_focused | 0.457 | 0.353 | 0.624 | 0.760 | 0.553 |
| ace_submod | 0.416 | 0.323 | 0.610 | 0.750 | 0.541 |
| oracle | 0.599 | 0.480 | 0.702 | 0.784 | 0.596 |
4pt
11 2WikiMultiHopQA Interventional Check↩︎
3B reader, budget 160, seeds \(\{42,13,7\}\), 500 questions. Retrieval gate: recall@5\({=}0.718\), all-gold@5\({=}0.43\). Key contrast, pooled 3-seed
bootstrap (\(n{=}1{,}500\)): chunk_submod\(-\)chunk_focused \({=}-0.008\) F1 \([-0.027,+0.012]\),
\(p{=}0.44\), with coverage \(+0.054\) but answer-in-context \(-0.007\)—coverage and answer-in-context move in opposite directions, and F1 follows
answer-in-context. Conditional F1 is \(0.56\) when the answer is in context versus \(0.08\) when not.