1 Introduction↩︎

Personal AI agents are assistants that use a user’s history, preferences, plans, and records to answer and act across sessions. Personal-agent systems and agent-memory architectures already point to a future setting in which this memory is layered: long-term profiles, recent logs, plans, self-reports, and device records [1]–[7]. As these systems mature, they will collect more memory from more places rather than maintain one clean history. Preferences can drift, plans can be optimistic, self-reports can carry topic-dependent bias, and sensors can be missing or noisy [8]–[12]. The exercise example in Figure 1 shows the resulting conflict at small scale: plans and self-reports can overstate activity, objective and device traces can be incomplete, and the resolver must decide whether to answer conservatively or abstain. This creates a deployment-relevant evaluation problem as well as an accuracy problem. If one black-box LLM both extracts memory facts and resolves conflicts, changing the model can also change the conflict policy. We use a resolver for the decision layer that maps the available memory representation, structured atoms (closed-class observations extracted from each source) or natural-language memory, to an answer or SKIP. Our benchmark separates extraction from resolution: an LLM can extract source atoms, while a structured resolver can learn source-to-ground-truth mappings from labels.

To our knowledge, conflict resolution across heterogeneous personal-memory sources has not yet been formalized as an evaluation target. Among coherent chat-history memory benchmarks, LongMemEval [13] studies memory over a single user–assistant interaction history. Concurrent multi-source memory work such as LifeBench [14] studies long-horizon retrieval and integration from diverse digital traces. Our setting targets a different failure mode: multiple sources can report conflicting evidence about the same latent behavior, so the system must learn source reliability and abstain when support is insufficient. Prior conflict-QA work studies cases where contexts, passages, or evidence sources disagree [15]–[18]. Context-use studies ask whether QA systems rely on the provided context appropriately [19]. Our benchmark targets a different setting: named personal-memory sources, such as profiles, plans, logs, and self-reports, can be biased in different ways for different questions. Reliability-aware RAG is related, but it usually learns a source-level reliability weight rather than how each source type tends to distort particular facts [20]. Cross-modal conflict benchmarks probe perception rather than textual evidence fusion [21], [22]. In multi-source personal memory, source reliability is part of the task. This lets us test whether a method can learn how each source maps to ground truth, instead of folding all conflict handling into a single prompted LLM. Existing QA benchmarks often force an answer, but personal-memory agents also need a calibrated option to abstain when sources contain irreconcilable conflict. Appendix 7 compares related benchmarks on multi-source evidence, directed bias, selective QA, resolver-versus-input decomposition, and synthetic data-generation support.

The testbed follows this separation explicitly. A synthetic data-generating process (DGP) creates latent daily events, projects them into five biased memory streams, renders those streams as natural-language memory, and asks resolvers to answer closed-class questions or SKIP. Structured methods first use an LLM extractor to turn each stream into these atoms; LLM baselines may instead read the natural-language memory directly. This design lets us vary the resolver and the input representation separately.

We make three contributions.

1. Task formalization. We define bias-aware selective QA over multi-source personal memory, where sources have directional distortions and resolvers may abstain under irreconcilable conflict (§ 2; Appendix 7).

2. Testbed. We release a synthetic diagnostic testbed of 18 question templates \(\times\) 480 personas \(\times\) 4 seeds (34,560 instances), with a documented DGP, label rules, code, and runnable extension examples for new questions and evidence streams (§ 2).

3. Diagnostic evaluation. We compare a suite of structured and LLM resolvers plus a direct-readout reference, showing that learned source models, calibrated abstention, and type-specific failure signatures reveal where conflict-resolution systems break (§ 4).

2 Benchmark Design↩︎

We define selective QA over conflicting multi-source personal memory as follows. A persona \(p\) is associated with five evidence streams \(\mathcal{S} = \{s_1, \ldots, s_5\}\): long-term profile, planner, daily self-report, objective log, and device log. The streams describe the same 30-day window through different views: profile summaries can be stale, plans can be optimistic, self-reports can carry topic-dependent bias, objective logs can be noisy or incomplete, and device logs can have day-level and field-level missingness (full distortion model in Appendix 8.1). Given a question \(q\) with discrete answer space \(\mathcal{V}_q\) (\(|\mathcal{V}_q| \in \{3,4\}\)), a method must produce either \(v \in \mathcal{V}_q\) or SKIP. Answers are scored against a deterministic ground truth (GT) label defined by the question-specific rule. SKIP is distinct from factual edge labels in \(\mathcal{V}_q\), such as no_plans. We report both answer-only and selective variants in § 4.

The benchmark spans 18 question templates across eight reasoning types (Table 1). Table 1 groups templates by the evidence pattern a method must resolve. Seven types target different source-conflict patterns, while the control type checks calibration under minimal conflict. Topic coverage spans work, diet, social, sleep, and exercise; 15/18 questions use ordered answer scales and 3 use unordered categorical labels. Representative questions and the survey-design principles applied to all 18 are in Appendix 8.2.

2.1 Data-Generating Process and Scale↩︎

Real personal-memory data with verifiable behavioral ground truth is hard to obtain, so we use a synthetic data-generating process (DGP) following the precedent of CLEVR [23] and bAbI [24]. The DGP, combined with paired extracted-atom and direct-readout conditions, supports controlled comparisons between resolver and input quality. The DGP runs in four stages (persona traits (\(L_1\)) \(\to\) latent event table (\(L_2\)) \(\to\) biased source projection (\(L_3\)) \(\to\) natural-language (NL) memory render), then a shared LLM extraction step produces per-source structured atoms; per-stage details and per-stream bias parameters are in Appendix 8.1 and Appendix 8.3. Personas are balanced across three difficulty classes (stable / temporal_shift / stated_vs_revealed, 160 each), and instances are formed by crossing each persona with the 18 question templates. The released full testbed contains 18 questions \(\times\) 480 personas \(\times\) 4 seeds = 34,560 instances. Figure 2 shows how the DGP separates latent behavior, visible source projections, extracted atoms, GT computation, and the direct-readout reference. This lets the evaluation vary atom source and resolver separately while preserving a DAG with no extraction-to-GT path.

Throughout the paper, GT means the deterministic label \(f_q(L_2,L_3,q)\). An atom is the closed-class value, or null, that one evidence stream provides for one question. We use \(\hat{\mu}^m\) for atoms extracted from NL memory by extractor model \(m\), and \(\mu^\ast\) for atoms read directly from the structured \(L_3\) source streams, with no NL rendering or LLM extraction. Unless stated otherwise, \(\hat{\mu}\) means \(\hat{\mu}^{\text{GPT}}\), the shared GPT-5.4 extraction used in the main tables; Appendix 12.6 repeats extraction with Gemini 3.1 Pro. Results on \(\hat{\mu}\) and \(\mu^\ast\) keep the resolver fixed and change only the atom source. When later tables say GPT-5.4 on \(\mu^\ast\), the LLM is reading those direct-readout atoms as structured prompt input. Source Reachability is a GT-aided reference: it asks whether at least one direct-readout source atom exactly equals GT. We report it only as a diagnostic reference, not as a resolver or skip policy.

Figure 2 separates latent behavior from the memory evidence available to methods. The top path shows what methods observe: the \(L_2\) event table is transformed into five source-specific \(L_3\) projections: an early-window or stated profile summary, aspirational plans, biased self-reports, sparse objective records, and device traces with day- and field-level missingness. These structured projections are rendered as NL memory and extracted into atoms \(\hat{\mu}\). The bottom path shows GT and scoring: all GT labels depend on \(q\) and \(L_2\), and 9/18 templates also read \(L_3\) annotations such as self-report labels or planner commitments; non-skipped answers are scored against GT. The dashed green path is the direct-readout reference path: it replaces NL extraction with \(\mu^\ast\) from structured \(L_3\). This removes NL extraction noise but does not read \(L_2\) directly, so \(L_2 \to L_3\) projection loss remains.

2.2 Ground Truth, Splits, and Protocol↩︎

Ground-truth labels are deterministic. The label rules use window selection, counting, ratios, thresholds, discretization, and categorical checks, with no ML or probabilistic component. Full per-template label functions are listed in Appendix 8.4. The pipeline forms a DAG: \(L_2\) generates \(L_3\), \((q,L_2,L_3)\) feeds GT, and extraction never feeds GT. An internal independent audit implementation recovers all 34,560 labels exactly (Appendix 8.5).

Per seed, 480 personas are split into 216 train / 48 dev / 96 calibration / 120 test, stratified by difficulty class. Train fits learnable parameters when a method has them. Calibration tunes SKIP thresholds for threshold-based selective variants via \(F_{0.5}\) maximization; \(F_{0.5}\) serves only as the threshold-selection criterion. We use \(\beta = 0.5\) because wrong answers are costlier than abstentions. Dev supports development-time checks and tuning, and test is held out for reporting. All fitted parameters and SKIP thresholds are learned within each seed. Main test results pool the four 120-persona test splits, giving 480 seed-persona clusters and 8,640 question instances, and report 95% bootstrap percentile CIs (\(B = 2000\) resamples over seed-persona clusters).

3 Evaluated Methods↩︎

We evaluate resolvers in four tiers and include the Source Reachability reference (Table [tab:3]); each tier isolates a specific modeling assumption and exposes a diagnostic contrast. We use \(\hat{\mu}(s, q)\) as shorthand for the default GPT-5.4-extracted atom from source \(s\) for question \(q\) (a value from \(\mathcal{V}_q\), or null), and \(\mu^\ast(s, q)\) for the same slot read directly from the structured \(L_3\) source streams. Methods marked \(\pm\) in the Selective column have both answer-only and selective variants with SKIP; we report both in § 4.

T0 provides source-free baselines, and T1 gives a single-source selection baseline. T2 fusion methods use structured multi-source signals: BCF applies a hand-specified ordinal source-bias prior with learned deflation weights, ABF uses the same prior inside a soft-mixture abductive kernel, and NBF/DSNBF learn empirical per-source confusion matrices from training labels. Four derived questions use per-question bias-prior overrides where the topic-level prior would invert the source semantics; Appendix Table 7 gives the fixed prior and override resolution rule. Although DSNBF has difficulty-stratified parameters, it is not given the held-out test-set difficulty mixture or any test persona’s true difficulty label; at test time it infers a per-persona difficulty posterior from that persona’s observed atoms only, and true difficulty labels are used only after prediction for reporting breakdowns. T3 LLM methods bypass the extraction pipeline entirely and read raw NL memory. Schema-Aware extends LLM-Direct by injecting qualitative source-bias descriptions into the prompt (e.g., "the planner tends to overstate intentions; the self-report overreports sleep and exercise but underreports work hours"), supplying the LLM with the same kind of domain knowledge that fusion methods encode through learned parameters but without access to test data. Source Reachability is a GT-aided reference: it reads \(\mu^\ast\) directly from structured \(L_3\), preserves all \(L_2 \to L_3\) projection artefacts, and asks whether GT is directly present in the visible source projections. Its 93.2% / 6.8% interpretation is in § 4.1. Selective variants abstain via posterior margin (NBF/DSNBF), two-criterion thresholds on absolute and relative evidence (ABF), null fallback (SSB), or self-reported would_skip (LLMs); SKIP thresholds for threshold-based variants are calibrated on a held-out split via \(F_{0.5}\) maximization (Appendix 9.2).

All T1/T2 methods share an LLM extraction stage that converts each NL memory document into structured atoms \(\hat{\mu}(s, q)\); we use GPT-5.4 [25] at maximum reasoning effort, with Gemini 3.1 Pro Preview as a cross-extractor robustness check (Appendix 12.6). Frozen no-API T1/T2 rows fit/calibrate on \(\mu^\ast\) train/cal atoms and test on frozen GPT-5.4 \(\hat{\mu}\) atoms; Appendix 9.4 gives cache details. T3 evaluates four LLM families (GPT-5.4, Gemini 3.1 Pro Preview, DeepSeek V3.2, Qwen3-235B-A22B-Instruct-2507, reported as Qwen3 235B in tables) under identical prompt templates with no per-family tuning. The 2\(\times\)​2 crossing of resolver (DSNBF vs. GPT-5.4) \(\times\) input quality (\(\hat{\mu}\) vs.\(\mu^\ast\)) is performed on GPT-5.4 (§ 4.2); full prompt templates, schema-aware bias descriptions, and per-LLM configurations are in Appendix 9.4.

Appendix 8.6 traces one persona end-to-end through five-source extraction, conflict structure, and a cross-tier subset of the method predictions discussed above.

4 Experiments↩︎

4.1 Main Results and Selective QA↩︎

Table 2 and Figure 3 report macro accuracy (per-question accuracy averaged over 18 questions) under each method’s canonical information condition. We pool four held-out test splits (480 seed-persona clusters; 8,640 question instances) and report 95% bootstrap percentile CIs (\(B = 2{,}000\)).

The direct-readout column needs a careful reading because it combines three granularities. At the answer level, DSNBF scores 80.3% on extracted atoms \(\hat{\mu}\) and 82.3% on direct-readout atoms \(\mu^\ast\), so the aggregate extraction effect is about 2.0 pp. At the cell level, the GPT-5.4 extractor matches direct readout in 93.08% of (persona, seed, question, source) cells. At the instance level, Source Reachability is a GT-aided probe: 93.2% of instances have some direct-readout source atom equal to GT. The complementary 6.8% marks cases where GT is absent verbatim from every direct-readout source; methods may still infer GT from distorted projections. Appendix 8.4 separates this reachability effect from extraction-path effects.

The main results suggest three things. First, naive aggregation can be worse than choosing a single reliable source: SSB (77.3%) beats Majority Vote and BCF (both near 69%) because, without bias correction, a biased majority can outvote an accurate minority. Second, methods that learn per-source confusion matrices pull clearly ahead. NBF and DSNBF reach about 80%, and DSNBF leads the strongest prompt-only LLM baseline by 10.3 pp with non-overlapping CIs; the same lead holds against every LLM family, with a CI lower bound of at least 8.4 pp. Third, qualitative bias hints are a much weaker substitute for labeled training: Schema-Aware prompting buys an LLM 0.8–5.9 pp over Direct, while DSNBF leads Direct prompting by 11.7–38.2 pp and remains 10.3–32.3 pp above Schema-Aware prompting. A prompt that names each source’s bias direction therefore carries much less of the source-to-GT mapping than a confusion matrix estimated from labels.

Selective QA asks how accurate a method is on the questions it chooses to answer; coverage is that answered fraction. DSNBF answers 78.3% of the testbed and reaches 85.3% selective accuracy, climbing to 88.8% on direct-readout input. Calibration transfers cleanly to the held-out test split, with test \(F_{0.5}\) between 80.4 and 85.3 across the four threshold-based fusion methods. LLM self-reported abstention behaves differently: selective-vs-answer-only gains stay under 2 pp, GPT-5.4 abstains on roughly 5–8% of items, Gemini and Qwen3 stay below 2%, and DeepSeek never abstains. Figure 4 plots deployable SKIP variants. The split is partly an interface property: fusion exposes calibrated posterior margins, while a prompt-only LLM returns a binary flag with no continuous score for post-hoc calibration. Full selective and few-shot results are in Appendix 10.1 and 10.3.

4.2 Factorial Decomposition (Resolver \(\times\) Input)↩︎

The testbed’s factorial design enables a controlled decomposition that requires \(\mu^\ast\) atoms, which are not normally available in naturalistic data without manual annotation or instrumentation. We cross a trained resolver (DSNBF-NoSkip vs. GPT-5.4) with input quality (\(\hat{\mu}\) vs.\(\mu^\ast\)); to isolate the resolver gap from the extraction gap, we provide the LLM with the same structured atoms that fusion methods receive, removing NL comprehension as a confound.

The four cells are DSNBF on direct-readout atoms \(\mu^\ast\) (82.3%), GPT-5.4 on the same \(\mu^\ast\) atoms (73.5%), DSNBF on extracted atoms \(\hat{\mu}\) (80.3%), and GPT-5.4 on extracted atoms \(\hat{\mu}\) (71.4%). The diagonal contrast from DSNBF/\(\mu^\ast\) to GPT/\(\hat{\mu}\) is 10.9 pp, and a difference-of-differences attribution assigns about 81% to the resolver and 19% to the input. The interaction term is essentially zero, so the two effects are close to additive in this comparison. We read the 81/19 split as an evaluation-design result: DSNBF was trained on labeled personas, while GPT-5.4 was prompted without that training distribution. Two checks outside the 2\(\times\)​2 point in the same direction: qualitative schema descriptions buy GPT only +1.1 pp, with overlapping CIs in Figure 3, and a seed-1 few-shot variant lifts GPT Direct by +6.5 pp but still trails DSNBF (Appendix 10.3). Enriching the LLM’s input narrows the gap to fusion but does not close it.

4.3 Diagnostic Analysis by Reasoning Type↩︎

The testbed’s eight reasoning types serve as diagnostic probes (§ 2). Table 3 reports accuracy by type under matched \(\mu^\ast\) input (\(\Delta_{\text{matched}}\) = DSNBF on \(\mu^\ast\) \(-\) GPT-\(\mu^\ast\), isolating the matched-input DSNBF-versus-GPT gap), alongside the strongest LLM and the Source Reachability reference by type.

3pt

At the type level, the results suggest three things. First, DSNBF leads on 7/8 types, with the largest matched-input gaps on Arbitration (\(+\)​19.0 pp [17.0, 21.2]) and Factor attribution (\(+\)​12.5 pp [9.6, 15.5]); Identity is the exception (\(-\)​0.2 pp), with both methods near the Source Reachability reference (\(\geq\)​96.8%). This is directionally consistent with § 4.2. Second, Annotation (G) is the hardest type because the latent attribute lies behind the source projections and must be inferred indirectly. DSNBF reaches 63.5%, the best LLM 49.9%, and the reference 83.7%, suggesting that learned source-GT mappings help more than qualitative bias hints. Third, model families fail in different places, with stable type-specific signatures. GPT-5.4 and Gemini split by type: under Schema-Aware prompting, GPT-5.4 exceeds Gemini by 13.1 pp on Identity, while Gemini exceeds GPT-5.4 by 15.5 pp on Annotation (Appendix 11.1). Full failure distributions and seed-stable signatures are in Appendix 11.3; the bias-modeling gradient by difficulty class is in Appendix 11.2.

4.4 Robustness Summary↩︎

Robustness checks reduce the concern that fusion only fits one fixed synthetic setting. NBF and DSNBF estimate source-to-GT confusion matrices from labeled train personas and predict from the observed atom table (whether \(\hat{\mu}\) or \(\mu^\ast\); Appendix 9.1). Across seeds, fusion \(\sigma \leq\)​0.8 pp; training-size curves saturate with data; rankings stay stable across nine DGP bias\(\times\)dropout variants (mean Kendall \(\tau = 0.77\)); swapping GPT for Gemini extraction costs DSNBF only 0.7 pp downstream; and DSNBF trained only on the default projection setting stays within 1.6 pp of DSNBF refit on each shifted variant on average (Appendix 12.1, 12.2, 12.4, 12.6, 12.7). Taken together, these checks support the internal robustness claim: the main result holds across seeds, parameter settings, and extractor models. Real-user bias patterns remain an external-validity question.

5 Discussion↩︎

The main result is that trained fusion methods are better at deciding which source to trust when sources disagree. In the matched-input comparison, GPT-5.4 and DSNBF receive the same structured atoms, but DSNBF still performs better, suggesting that the improvement is driven by the resolver rather than by the quality of data extraction from natural language. In particular, prompt-only LLMs make a substantial part of their errors at the conflict-resolution step in this benchmark.

The benchmark is useful because it makes these failure points observable. Source Reachability checks whether the answer was visible in any source. The extraction audit checks whether the LLM recovered the intended structured atoms from the natural-language memories. The resolver comparison then tests what happens after those atoms have been fixed. Selective QA adds a final check: whether a method can skip cases where the evidence is too weak. These three measurements are combined together into a single end-to-end accuracy metric. The released testbed also supports extensions with new questions or evidence streams.

The benchmark has three main limitations. First, it is synthetic by design. Its source distortions are monotone and topic-dependent, while real personal data may include changing habits, changing sensor coverage, adversarial reports, or user-specific source biases. Second, the benchmark has 18 question templates, which limits within-type power even though the data-generating process can support more label functions. Third, the strongest fusion methods require labeled training personas, so their results should not be read as claims about cheaper deployment. Their value here is diagnostic: an LLM can extract source atoms, while a separate fusion layer resolves conflicts.

A diagnostic benchmark for personal-memory agents can improve reliability by making conflict handling and abstention explicit before deployment. The same line of work also raises privacy risks if used to infer sensitive habits from personal records or to normalize surveillance style aggregation. We therefore release only synthetic data and cached artifacts. Real deployment would require user consent, data minimization, privacy controls, and validation on the target population.

6 Conclusion↩︎

We introduced selective QA over conflicting multi-source personal memory: systems must reconcile biased evidence streams or abstain when evidence is insufficient. The core contribution is a diagnostic testbed that separates source coverage, extraction, resolver behavior, and abstention, so failures can be localized rather than collapsed into one leaderboard score.

The results show why this separation matters. Under the primary test condition, trained fusion reaches 80.3% macro accuracy versus 70.0% for the strongest prompt-only LLM baseline, and selective fusion reaches 85.3% accuracy at 78.3% coverage. We release data, code, cached artifacts, and extension examples for new questions, sources, and real-user validation; the reproduction repository is available at https://github.com/TianchengY/multisource-membench, and the dataset release is hosted at https://huggingface.co/datasets/ytc1997/multisource-membench.

Appendix Contents↩︎

7 Related Work↩︎

7.1 Comparison Table with Existing Conflict-Related Benchmarks↩︎

Table 4 summarises how our testbed compares to representative conflict-related and personal-memory benchmarks along five structural dimensions. The remaining subsections (§ 7.2–§ 7.8) discuss each line of literature in more depth, beginning with long-term memory benchmarks most likely to be confused with ours.

2pt

7.2 Long-Term Memory Benchmarks and Agent Memory↩︎

The benchmark most likely to be confused with ours is concurrent LifeBench [14], which studies long-horizon memory over diverse digital traces such as chats, app records, and health records. Its central question is whether a memory system can retrieve and integrate information scattered across heterogeneous artifacts over time. Our benchmark targets source conflict: several streams can make incompatible claims about the same latent behavior. The system must learn source reliability and abstain when support is insufficient. LifeBench emphasizes source diversity; our benchmark isolates source disagreement.

This distinction changes the evaluation object. LifeBench primarily stresses memory retrieval, temporal integration, and non-declarative memory over a long horizon. We instead fix five typed personal-memory streams and make disagreement systematic: profile, planner, self-report, objective log, and device log can be stale, optimistic, distorted, incomplete, or unavailable in source-specific ways. This lets us measure conflict resolution directly through closed-class GT, calibrated selective QA, and a resolver-versus-input decomposition.

LongMemEval [13] is complementary in another direction. It evaluates chat-assistant memory over coherent user–assistant histories, with questions testing information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention. Its own construction notes that sampled surrounding sessions “avoid providing conflicting information that would invalidate the question” [13]. LongMemEval therefore asks whether a system can recall from a coherent log; our testbed studies arbitration when source streams are intentionally inconsistent.

MemoryAgentBench [28] evaluates memory agents through incremental multi-turn interactions, including selective forgetting when later evidence contradicts earlier facts. This is complementary to our setting: it tests whether an agent updates memory over time, whereas our benchmark fixes several typed personal-memory streams at once and evaluates source-specific conflict resolution, resolver-versus-input decomposition, and calibrated abstention.

Broader agent-memory systems make a similar distinction clear. MemGPT [6] introduces layered memory management, Generative Agents [29] simulate daily-life memory with retrieval and reflection, and Reflexion [30] uses memory for verbal self-improvement. These systems study how agents store, retrieve, and reuse memory. Our evaluation target is the separate problem of resolving conflicts across systematically biased personal-memory streams.

7.3 Knowledge Conflicts↩︎

[31] survey knowledge conflicts in LLMs, categorizing them as context-memory, inter-context, or intra-memory discrepancies. [15] construct a controlled QA setting for parametric-memory versus counter-memory conflicts, while ECon [17] constructs a benchmark for evidence-conflict detection and resolution. [32] study confidence calibration under knowledge conflicts, finding that QA models are poorly calibrated when retrieved passages disagree. NatConfQA [16] introduces fine-grained evaluation of conflict detection across retrieved documents, and QACC [18] studies open-domain QA with conflicting contexts. Very recent ConflictQA [26] studies faithful reasoning when textual evidence and knowledge graph evidence conflict. [33] study QA with source citations in ambiguous settings where multiple valid answers may be supported by different sources.

ConflictQA is close to our setting because it also uses heterogeneous evidence sources, but its sources are external RAG artifacts rather than personal-memory streams. It does not model source-specific personal bias, calibrated selective QA, or a resolver-versus-input decomposition. More broadly, these settings use retrieved passages or contexts. Our testbed models named personal-memory streams with learnable, question-specific directional distortions and controlled source-disagreement profiles.

7.4 RAG Conflict Resolution↩︎

[34] survey the Naive / Advanced / Modular RAG paradigms and their retrieval, generation, and augmentation components. Most standard RAG formulations treat retrieved passages as exchangeable evidence. Reliability-aware variants such as RA-RAG [20] estimate scalar source reliability, which differs from learning question-specific mappings from typed personal-memory streams to GT labels.

ArgRAG [35] applies argumentation graphs to resolve retrieved document contradictions. Under our closed-class, equal-weight adaptation, ArgRAG reduces to plurality voting; richer ArgRAG variants with document-specific strengths are outside this comparison. FactReasoner [36] builds probabilistic factuality models for long-form generation, atomizing claims and scoring them against retrieved evidence. These methods target retrieved-evidence conflict or long-form factuality; our setting fixes typed, directionally biased streams and evaluates closed-class source-conflict resolution.

7.5 Cross-Modal Conflict Benchmarks↩︎

CrossCheck-Bench [21] evaluates vision-language models on compositional image-text contradictions and diagnoses failure modes across controlled difficulty tiers. CLASH [22] pairs COCO images with captions containing controlled object-level and attribute-level contradictions, evaluated via multiple-choice and open-ended QA. These benchmarks address perceptual conflict detection between vision and language modalities; our work addresses conflicts among multiple textual evidence streams with controlled systematic biases.

7.6 Selective Prediction and Abstention↩︎

[37] formalize selective classification with coverage-accuracy trade-offs. [38] apply it to QA. [39] study selective answering under question ambiguity using sampling-based confidence. [40] survey ~100 abstention methods across the LLM lifecycle. Our testbed extends selective prediction to multi-source settings where evidence insufficiency is an explicit driver of abstention, alongside model uncertainty.

7.7 Synthetic Benchmarks↩︎

Diagnostic benchmark design often uses synthetic or controlled tasks: CLUTRR [41] for systematic generalization, bAbI [24] for reasoning primitives, and BigBench-Hard [42] for controlled difficulty. Our DGP follows this tradition, providing a fully documented generation process that supports documented extensions to new question types and sandboxed source configurations.

7.8 Agent Evaluation Benchmarks↩︎

AgentBoard [43] and TaskBench [44] evaluate agent capabilities in tool-use and planning domains. InfiBench [45] evaluates code-LLM question answering, and AgentIF [46] benchmarks instruction following in agentic scenarios (tool use, system prompts, multi-step plans). These benchmarks evaluate planning, tool use, and instruction following; our benchmark focuses on multi-source conflict resolution and selective abstention over personal memory.

8 Benchmark Details and Ground-Truth Verification↩︎

8.1 Evidence Streams and Distortion Model↩︎

Table 5 lists the five evidence streams and their distortion characteristics. We encode directional bias as \(b\): \(b = +1\) indicates systematic overreporting (the source reports more favorably than reality), \(b = -1\) indicates systematic underreporting, and \(b = 0\) indicates no directional bias.

The self-report’s directional bias pattern reflects a well-documented phenomenon in survey methodology: respondents tend to overreport socially desirable behaviors (sleep quality, healthy eating, exercise frequency) and underreport undesirable ones (overtime work, social isolation) [10], [11]. The planner’s positive bias mirrors the planning fallacy [9], where intended behavior is systematically more favorable than realized behavior. The long-term profile’s early-window anchoring reflects a design choice: the profile is generated from the persona’s initial behavioral parameters and may not be updated promptly as behavior shifts. Device/activity-log missingness combines difficulty-dependent day-level dropout with field-level missingness, including an approximately 50% dropout rate for the work-session field, to simulate non-wear, connectivity loss, battery limits, and partial sensor coverage [12].

8.1.0.1 Bias-parameter grounding.

The literature above motivates the direction of each distortion, not an exact empirical effect-size calibration. We choose coarse bias signs (\(-1,0,+1\)), schema gaps, and missingness levels to create a controlled stress test in which source conflict is common enough to diagnose methods. The DGP perturbation grid in Appendix 12.4 then checks whether the method ordering is tied to one fixed bias magnitude.

8.2 Question Specifications↩︎

8.2.1 Full 18-Question Specification↩︎

Table [tab:B-2] lists all 18 questions with topic, time window, ordered/categorical answer-space classification, and answer space. Table [tab:B-3] gives the full question text and targeted failure mode. The complete reasoning-type taxonomy and per-question design rationale are released in configs/questions.yaml in the code artifact.

Composition: 15 ordinal (with monotone bias direction along the answer scale) and 3 unordered categorical templates (B3, C3, E1; bidirectional or categorical without natural ordering). Topic distribution: Work (3), Diet (3), Social (4), Sleep (4), Exercise (4). Edge-case options (no_frequency_described, no_plans, no_baseline, no_late_nights, no_social_activities, no_activity, no_meetings, no_targets, no_approach_described) represent factual edge cases ("the prerequisite condition does not exist for this persona"), not uncertainty; they are deterministic GT labels and do not participate in ordinal encoding. The table prints human-readable answer spaces. For BCF/ABF, the code also defines a per-question bias-shift order: the ordered rank axis along which a \(+1\) or \(-1\) source prior moves a candidate label. This order is part of the released YAML specification and can differ from the display order for derived-count questions; Table 7 documents the fixed prior, override rule, and rationale.

8.2.2 Full Question List↩︎

The question templates use survey-style safeguards to reduce ambiguity: fixed time windows, closed answer spaces, explicit numeric boundaries, and edge labels for missing prerequisites. All wording uses neutral language without value-laden terms.

8.3 Synthetic Data-Generating Process↩︎

The DGP runs in four stages.

• Stage 1: Personas. Generates 480 personas across three difficulty classes (stable, temporal_shift, stated_vs_revealed; 160 each), and samples behavioral parameters from domain-specific distributions over five topics.

• Stage 2: Latent event table. Produces a 30-day latent event table for each persona recording actual daily behavior.

• Stage 3: Source projection. Projects \(L_2\) into five visible source files through source-specific schemas, bias/noise, quantization, and missingness, using seeded random number generators.

• Stage 4: NL render. Renders each structured source into a natural-language (NL) memory document via deterministic templates, ensuring exact reproducibility and removing NL variance as a confounding variable.

A shared LLM extraction step then recovers per-source structured atoms from the NL memory. By design, the LLM performs only factual extraction; all cross-source reasoning is handled by downstream resolvers (§ 3). The DGP is method-agnostic: any new method can be evaluated on the released data without re-running generation or LLM calls. We release the DGP code, generated data across four seeds, pre-rendered NL memories, and pre-extracted structured atoms in the code and dataset artifacts.

8.4 GT Construction Notes↩︎

For the 9 questions that measure source-reality discrepancies (B2-B3, C2-C3, D2, E2, F1-F3), \(f_q\) additionally reads the generated source outputs as immutable inputs. A direct-readout gap arises when a visible source schema drops, coarsens, or biases the relevant \(L_2\) field enough to move it across an answer boundary. This is intentional: systematic reporting biases can make recorded behavior differ from actual behavior, and some latent attributes, such as whether a social event was voluntary, are only partially visible in the source projections. We quantify this recoverability gap with the Source Reachability reference in § 4.1 and by type in Appendix 11.1. The statistic measures exact visibility of GT in direct source readouts; inference methods can still recover some remaining cases by learning how biased source values map back to GT.

Table [tab:B4-label-functions] gives the label function for each template. Answer labels and edge labels are listed in Table [tab:B-2]; the table below specifies which fields are read and how one label is selected. For G1/G2, GT is a DGP-latent event attribute, such as intentional exercise or voluntary social activity, while visible sources expose biased projections of that attribute.

Table 6 asks a narrow diagnostic question: after splitting test instances by whether GT appears as an exact match in any direct-readout source atom, how often does each method still predict GT? The three accuracy columns are instance accuracies on the full test set, the GT-present slice, and the no-exact-source-match slice.

Table 6 gives two checks on the 6.8% complement. The no-exact-source-match slice is hard for methods that mostly trust visible labels, confirming that it is a real source-reachability gap. At the same time, DSNBF with extracted atoms still answers 53.5% of these instances correctly, so exact visibility of GT is not required for recovery. The \(\mu^\ast\) row is lower (36.0%) because the slice is defined by the absence of an exact \(\mu^\ast\) match; the extracted-atom row therefore mixes learned source-to-GT inference with extraction-path changes. Since the slice is label-distribution shifted, we use it as a diagnostic companion and keep Table 2 as the main aggregate comparison.

8.5 Independent Reproduction of Ground Truth↩︎

As an internal audit, a separate implementation, written independently and without reference to the released GT computation code, reimplements the full label generation pipeline: latent event table construction, source projection, and the 18 question-specific label functions \(f_q(L_2,L_3,q)\). The audit re-reads the raw event tables and structured source JSON files, recomputes each closed-class label with separate question-specific functions, and compares only final answer labels. Running this independent implementation on all 4 seeds \(\times\) 480 personas \(\times\) 18 questions (34,560 labels) produces 100% exact agreement with the released GT labels. This establishes that the released labels are a stable, deterministic consequence of the formal task definition and DGP specification, not an artifact of one specific code path or implementation choice. This is an internal reproducibility check for the formal GT definition, not a reproduction command; external validity of the latent target remains a design question for real-user follow-up.

8.6 Walked-through Example: One Persona, One Question↩︎

We applied a deterministic selection protocol (§8.7) to all 18 question templates on the test split (120 personas \(\times\) 4 seeds \(=\) 480 (persona, seed) pairs per question) and chose A3 (Home-Cooked Share, Diet topic), whose qualifying pool contained \(N=43\) pairs satisfying all five hard filters; the displayed instance was selected by the deterministic protocol in Appendix 8.7. The numbered paragraphs below correspond to the boxes of Figure 2, walking one instance through every stage of the DAG.

8.6.0.1 Stage 0 (Figure 2, Question \(q\)). A3.

“In the past 30 days, on what share of meal occasions did this person eat home-cooked food?” Answer space (3 ordinal bins): less_than_40 \(<\) 40_to_69 \(<\) 70_or_more. Ground truth: 40_to_69 (52 of 92 meals home-cooked, \(\rho = 0.565\)), computed deterministically from the latent event table.

8.6.0.2 Stage 1 (Figure 2, Persona L\(_1\)).

bench_shift_123_taylor_ellis, seed 1, difficulty class temporal_shift: a 34-year-old vegetarian accountant whose stated profile claims roughly 68% of meals home-cooked, but whose realised 30-day behavior shifts toward more delivery.

8.6.0.3 Stage 2 (Figure 2, Latent Events L\(_2\)).

The 30-day meal log contains 92 events; 52 are home-cooked. Eight representative days:

The drop from \({\sim}70\%\) home-cooked in days 1–13 to \({\sim}40\%\) in days 14–30 instantiates this persona’s temporal_shift difficulty class.

8.6.0.4 Stage 3 (Figure 2, 5 Source Streams L\(_3\)).

Each source projects the same \(L_2\) table through its DGP-defined schema. Diet-relevant fields:

8.6.0.5 Stage 4 (Figure 2, NL Memory Render). Excerpts (single day, 2026-01-06).

The deterministic renderer turns each \(L_3\) source into prose. One day per source:

• profile_ltm: “Diet: averages 3.1 meals per day, about 2.1 of those are home-cooked.”

• planner: “2026-01-06 (Tue). Home-cooked food prioritized. Work capped at 8.0 hours, finish by 18:00.”

• daily_self_report: “2026-01-06. Diet: 3 meals total, 3 home-cooked. Work: 9.8 hours, overtime.”

• objective_log: “2026-01-06. Payments: 2 coffee purchases, 1 food-delivery purchase. Calendar: work block 09:00–18:24.”

• device_log: “2026-01-06. Sleep: bed 23:39, wake 07:58, 7.8 hours. Activity: 19 active minutes.” (no diet content)

8.6.0.6 Stage 5 (Figure 2, LLM Extraction). NL \(\to \hat{\mu}\).

The extractor (GPT-5.4, max reasoning) reads each source’s NL render in isolation and emits structured atoms per question; no cross-source reasoning and no GT access. For A3 it returns a JSON of the form {"A3": "\(\langle\)bin\(\rangle\)"} per source, where \(\langle\)bin\(\rangle \in \{\texttt{less\_than\_40}, \texttt{40\_to\_69}, \texttt{70\_or\_more}\}\). For daily_self_report, the 30-day NL text implies a self-reported share \(\rho_{\text{self}} = 0.720\), and the emitted atom is consistent with binning that aggregate as 70_or_more. The emitted atoms across the five sources are tabulated next.

8.6.0.7 Stage 6 (Figure 2, Atoms \(\hat{\mu}\)). Source extracted atoms.

Note \(\dagger\): For A3, objective_log records delivery purchases but not grocery or cooking events, so it is an incomplete proxy for home-cooked share. The resulting 70_or_more atom reflects information missing from this source rather than LLM extraction error; aggregate extraction faithfulness is audited in Appendix 8.8.

8.6.0.8 Stage 6\('\) (Figure 2, direct source readout). \(\mu^\ast\) atoms and extraction faithfulness.

The Source Reachability reference reads the structured \(L_3\) fields directly via a hand-coded Python readout and emits atoms \(\mu^\ast\) with zero LLM involvement. Walking through the same day for daily_self_report: the readout pulls the JSON field {"meals": 3, "home_cooked": 3} for 2026-01-06, repeats over the 30 dated entries, sums to \((93, 67)\), computes \(\rho_{\text{self}} = 67/93 = 0.720\), and bins to 70_or_more. The two extraction paths are reading the same underlying \(L_3\) projection in different surface forms; for this instance \(\mu^\ast = \hat{\mu}\) for all five sources: (40_to_69, 70_or_more, 70_or_more, 70_or_more, null). NL extraction introduces no atom-level noise here. This is the dominant pattern globally: across the test split, \(\hat{\mu} = \mu^\ast\) in \(93.08\%\) of (persona, seed, question, source) cells (Appendix 8.8).

8.6.0.9 Stage 7 (Figure 2, GT \(f_q(L_2,L_3,q)\)). Ground truth derivation.

The label function \(f_{A3}\) is deterministic: count home-cooked meal events in the 30-day latent event table, divide by total meals, assign to bin. \[\rho \;=\; \frac{\#\{\text{home\_cooked}\}}{\#\{\text{meals}\}} \;=\; \frac{52}{92} \;=\; 0.5652 \;\in\; [0.40,\,0.70) \quad\Longrightarrow\quad \texttt{40\_to\_69}.\] The independent audit implementation (§8.5) computes the same value bit-for-bit; both populations write to ground_truth.json as {"A3": {"answer": "40_to_69", "derivation_detail": "home=52/total=92 ratio=0.565"}}.

8.6.0.10 Per-source distortion taxonomy on this instance.

To separate which mechanism produced each source’s answer, we decompose the five sources below:

The five rows instantiate five distinct distortion mechanisms: same-bin coincidence, boolean quantization, boundary-crossing bias, schema projection gap, and schema absence. MV and SSB do not model these mechanisms jointly; a learned per-source confusion matrix (NBF, DSNBF) absorbs all five jointly, which is why it is the first fusion family to recover GT on this instance.

8.6.0.11 Stage 8 (Figure 2, resolvers + reference). Method predictions.

Variants marked +SKIP are the selective (abstaining) counterparts; NBF/DSNBF+SKIP abstain when posterior margin \(p(\text{top1}) - p(\text{top2}) < \theta_E\) (per-method calibrated threshold, Appendix 10.2). LLM+SKIP abstains when the answering LLM reports would_skip=true. “n/a” marks methods without a skip mechanism.

8.6.0.12 Stage 9 (Figure 2, Score vs.GT). How this instance enters the aggregate metrics.

For each non-skipping resolver, the prediction is compared against GT (40_to_69) and contributes one binary correctness sample to the per-question accuracy in Table 2: NBF, DSNBF, Majority-Class, and GPT Direct each contribute a \(1\), while SSB, MV, BCF, GPT Schema, GPT-5.4 on \(\mu^\ast\), Gemini Direct, and Gemini Schema each contribute a \(0\). Source Reachability is reported separately as a GT-aided direct-readout reference; this instance is reachable because GT appears in the \(\mu^\ast\) readout. For methods with SKIP, the same instance feeds the selective-QA pair \((\text{coverage}, \text{selective accuracy})\) in Table 10. NBF+SKIP, DSNBF+SKIP, and GPT-LLM+SKIP skip this item, lowering coverage while leaving the selective-accuracy numerator and denominator unchanged. \(F_{0.5}\) is used only on the calibration split to choose SKIP thresholds; once fixed, the thresholds determine the reported test coverage and selective-accuracy cells. The instance therefore enters Tables 2, 3, 10, and Tables 12–15 once at the (method, A3, temporal_shift) coordinate.

8.6.0.13 Takeaway.

This single instance illustrates five mechanisms. First, the planner and daily_self_report atoms both shift one bin upward relative to GT, matching their positive Diet bias. Second, objective_log also lands on 70_or_more even though its spec-level bias is \(b_{\text{obj}} = 0\). The source records only delivery transactions here, and \(\hat{\mu}_{\text{obj}} = \mu^\ast_{\text{obj}}\), so the error comes from structured projection loss at \(L_2 \to L_3\) rather than from LLM extraction or a mean source-bias parameter.

DSNBF’s per-\((s,q)\) confusion matrix is estimated from the empirical joint distribution of \(\hat{\mu}\) and GT, so it absorbs projection loss, quantization loss, and bias jointly. This accounts for its margin over methods that model only one axis of distortion, such as SSB and Majority Vote.

Third, Schema-Aware prompting flips GPT from a correct 40_to_69 to an incorrect 70_or_more, matching the aggregate overcorrection pattern in § 4.3. The matched-input GPT row (GPT-5.4 on \(\mu^\ast\), § 4.2) gives the same incorrect answer, placing the error after extraction for this example.

Fourth, selective abstention pays a coverage cost on this item. NBF+SKIP, DSNBF+SKIP, and GPT-5.4 LLM+SKIP all abstain, while their answer-only counterparts choose the GT bin. The aggregate coverage/accuracy trade-off is reported in Appendix 10.2. The skip mechanisms also differ: NBF+SKIP and DSNBF+SKIP use calibrated posterior thresholds, GPT-5.4 LLM+SKIP uses a self-reported flag, and SSB+SKIP fires only when the chosen source returns no atom.

Fifth, the two majority baselines diverge. Source-level Majority Vote follows the three biased sources to 70_or_more, while population-level Majority Class happens to match GT through the training-set modal label. Among fusion methods, NBF and DSNBF recover GT through learned per-source reliability. LLM Direct answers split across model families, with GPT correct and Gemini wrong; Table 2 gives the aggregate per-method result.

8.7 Selection Protocol for the Walked-through Example↩︎

The instance shown in § 8.6 was selected by the following deterministic protocol applied to all 18 question templates on the test split (120 personas \(\times\) 4 seeds \(=\) 480 (persona, seed) pairs per question). Hard filters: (i) at least 4 of 5 sources produce a non-empty extracted atom, and the present sources include planner, daily_self_report, and objective_log; (ii) DSNBF \(\arg\max == \mathrm{GT}\); (iii) at least one frontier T3-LLM (GPT-5.4 Direct or Gemini 3.1 Pro Direct) \(\arg\max \neq \mathrm{GT}\); (iv) the source extracted atoms span at least two distinct values; (v) DSNBF posterior margin \(p(\text{top1}) - p(\text{top2}) \in [0.08, 0.35]\), excluding both trivial and marginal cases. Soft preferences (used to rank within the qualifying pool): planner positive bias visible in the extracted atom; daily_self_report directional bias visible in the source text; cross-family LLM divergence; at least one other fusion method also correct. Final tiebreak: lexicographic sort on (persona_id, seed, q_id). The qualifying pool size for A3 was \(N = 43\).

8.8 Atom Extraction Faithfulness Audit↩︎

To quantify how often LLM extraction recovers exactly what the direct structured readout produces, we compare \(\hat{\mu}\) against \(\mu^\ast\) at the (persona, seed, question, source) cell level on the full test split (120 personas \(\times\) 4 seeds \(\times\) 18 questions \(\times\) 5 sources \(= 43{,}200\) cells). Both arms see the same \(L_3\) structured projections; only the atom-level extraction path differs (\(\hat{\mu}\) via GPT-5.4 reading NL, \(\mu^\ast\) via deterministic Python reading the structured JSON). \(\hat{\mu} = \mu^\ast\) in \(\mathbf{93.08\%}\) of cells. The remaining \(6.92\%\) are concentrated in three questions (A2, G2, F2; per-question equality \(81\)–\(84\%\)) and one source (objective_log; equality \(86.5\%\)).

Cells where both \(\hat{\mu}\) and \(\mu^\ast\) are null count as equal; this primarily affects device_log (no diet schema by design) and objective_log on questions outside its observation domain. Equality rate is uniform across seeds (per-seed range \(92.66\)–\(93.94\%\)) and across difficulty classes (stable \(93.02\%\), temporal_shift \(93.81\%\), stated_vs_revealed \(92.40\%\)).

The audit supports two claims used in the main text. First, LLM extraction is faithful on average: for \(93\%\) of cells the NL\(\to\hat{\mu}\) step produces the same atom that a hand-coded JSON readout would. Second, for GT/source mismatches, the audit shows that most are unlikely to be caused by NL extraction alone, because \(\hat{\mu}\) usually matches \(\mu^\ast\). The Source Reachability reference (Table 2) bypasses extraction noise but inherits every upstream projection artefact. See the number legend in § 4.1 for the distinction among 80.3, 82.3, 93.08, 93.2, and 6.8. The pure extraction contribution is captured by the \(\sim\)​2 pp factorial input effect in § 4.2, which holds the resolver fixed and swaps \(\hat{\mu}\) for \(\mu^\ast\). Because \(\hat{\mu}\) in the main tables denotes the frozen GPT-5.4 extraction cache, this audit uses GPT-5.4; Appendix 12.6 reports the corresponding Gemini 3.1 Pro cross-extractor comparison, including per-question extraction accuracy.

9 Method Details↩︎

9.1 Fusion Method Formulas↩︎

Bias Corrected Fusion (BCF). BCF treats each candidate answer as a hypothesis. For each hypothesis \(v\), a forward bias model predicts what each source would report, \(g_s(v)\), by shifting \(v\) along the source’s bias direction and clamping to the ordinal range. The candidate with the most source prediction matches wins:

\[\hat{y} = \arg\max_v \sum_{s:\mu_s \neq \text{null}} (1 - \delta_s)\cdot \mathbb{1}[\mu_s = g_s(v)]\]

where \(\delta_s \in [0, 0.5]\) are per-source deflation weights (4 learnable, \(\delta_{\text{obj}} = 0\) fixed), grid-searched on the training split. Because the shift mechanism assumes ordinal structure, BCF degenerates to Majority Vote on the 3 unordered categorical questions. Table 7 gives the fixed directional prior used by the forward bias model. A shift of \(+1\) or \(-1\) means one step along the per-question bias-shift order in the released YAML specification; the signs are semantic priors fixed before evaluation, not empirical fits. Table [tab:B-2] prints human-readable answer spaces, which need not be identical to this bias-shift order for every derived-count question.

Override rationale. The defaults are topic-level priors, but four derived questions reverse the usual topic semantics. A2 counts long-work days; because its bias-shift order is reversed, underreporting work requires a \(+1\) shift toward fewer overtime days, not toward the high-work label. C2 asks which planned social activities were actually realized; retrospective self-report tends to emphasize completed plans, shifting toward higher realization rather than the generic “less social” prior. F1 asks about unplanned social activity: planner records should shift toward fewer unplanned events, while self-report can reveal activities that occurred despite no prior plan. G2 asks whether social activities were voluntary or obligatory; self-report idealization shifts toward voluntary framing. These overrides therefore encode the intended source semantics for each answer scale, without using held-out labels.

Naive Bayes Fusion (NBF). NBF drops BCF’s assumption that bias acts as a fixed ordinal shift and instead learns the full empirical relationship between ground truth and each source’s report. For each source \(s\), a confusion matrix \(C_s \in \mathbb{R}^{|\mathcal{V}| \times |\mathcal{V}|}\) records \(C_s[v, v'] = P(\mu_s = v' | y = v)\), estimated with Laplace smoothing on the training split. Given a new instance, the posterior is:

\[\hat{y} = \arg\max_v \; P(v) \prod_{s:\mu_s \neq \text{null}} C_s[v,\; \mu_s]\]

The class prior \(P(v)\) is estimated as the empirical ground-truth marginal on the 216-persona training split with Laplace smoothing (\(\alpha=1\)), re-estimated per seed; calibration and test splits are not accessed during parameter estimation. SKIP thresholds are selected later on the calibration split. This yields \(5 \times |\mathcal{V}|^2\) free parameters per question.

Difficulty Stratified NBF (DSNBF). DSNBF observes that sources distort differently across persona difficulty classes (stable, temporal_shift, stated_vs_revealed) and learns a separate confusion matrix \(C_s^{(d)}\) for each class. To avoid overfitting when per-class counts are small, each \(C_s^{(d)}\) is regularized toward the global matrix via a Dirichlet prior with strength \(\eta\). At test time the persona’s difficulty class is unobserved; DSNBF infers it from the pattern of source observations across all 18 questions using Bayesian marginal likelihood, then blends global and difficulty-stratified posteriors with a mixing weight \(g_w\).

Information boundary for DSNBF. The difficulty stratification is learned from labeled training personas only. The method estimates a training prior \(\pi_{\mathrm{train}}(d)\) and per-difficulty source matrices from the 216-persona train split; \(\eta\), the emission temperature \(T\), the difficulty-estimation temperature \(T_{\mathrm{diff}}\), the global/difficulty mixing weight \(g_w\), and any SKIP margin threshold are selected on the calibration split. No statistic of the held-out test-set difficulty distribution is passed to DSNBF. For a test persona \(i\), the runtime call receives only its observed atom table \(\mu_i\) and computes \[\tilde{p}_i(d) \propto \pi_{\mathrm{train}}(d) \prod_q \sum_{y \in \mathcal{V}_q} P_d(y) \prod_{s:\mu_i(s,q)\neq\mathrm{null}} C_{s,q}^{(d)}[y,\mu_i(s,q)]^{T_{\mathrm{diff}}}.\] This posterior is persona-local: it may infer that an individual test persona looks stable, temporal-shifted, or stated-vs-revealed from its source pattern, but it never observes the aggregate counts of stable/temporal-shift/stated-vs-revealed personas in the test split, nor the true difficulty label used later for per-difficulty reporting. In the evaluation code, evaluate_method calls prepare_persona(inst["mu"]) before prediction; inst["difficulty"] is consumed only after the prediction to accumulate reporting cells.

Persona identifiers are bookkeeping keys only. They are used to join files, cache outputs, and group bootstrap resamples, but they are not passed as model features or included in LLM prompt text for the target persona. Evaluated prediction calls receive only the source atom table (T1/T2), the rendered NL memory (T3), or the structured atom grid used for the matched-input diagnostic; they do not receive persona_id, seed names, split paths, or difficulty-prefix strings such as bench_stable, bench_shift, or bench_stated. Difficulty labels and ID prefixes are used for generation, training/calibration labels, selection of few-shot exemplars, and reporting strata, but not as target-persona prediction inputs. External evaluators should preserve this boundary by stripping or ignoring identifiers before invoking prediction code; using ID prefixes as features would be leakage rather than a valid benchmark submission.

Abductive Bias Fusion (ABF). ABF scores each candidate answer \(v\) via a soft-mixture kernel that blends two alignment paths. The bias-aware path compares each source’s observation \(\mu_s\) against the bias-shifted prediction \(g_s(v)\) (as in BCF, using the fixed prior and override rule in Table 7). The bias-free path compares \(\mu_s\) directly against \(v\) without any bias correction. The combined score is:

\[\text{ExplScore}(v) = \sum_s w_s \big[\pi \cdot \kappa_\alpha(d_{\text{bias}}(v, \mu_s, \delta_s)) + (1-\pi) \cdot \kappa_\alpha(d_{\text{id}}(v, \mu_s))\big]\]

where \(\alpha\) controls kernel bandwidth and \(\pi\) controls the bias-identity mixture. ABF has 6 learnable parameters (4 bias offsets \(\delta_s\) + \(\alpha\) + \(\pi\)), fitted by MLE on the training split.

9.2 Selective (SKIP) Mechanisms↩︎

Methods marked \(\pm\) in Table [tab:3] support a selective mode where the system may abstain instead of committing an answer. The SKIP decision differs by method family.

• Bayesian fusion (NBF, DSNBF). SKIP is triggered when the posterior margin (the gap between the top two posterior probabilities) falls below a calibrated threshold \(\theta\); instances with fewer than two active sources always answer.

• ABF. A two-criterion rule: if the best ExplScore falls below \(\theta_E\) (insufficient absolute evidence) or if the gap to the second-best candidate falls below \(\theta_\Delta\) (ambiguous winner), the method abstains.

• SSB. Skips when its best source returns null or, under a stricter variant, when fewer than \(\theta_{\text{agree}}\) fraction of active sources agree with the chosen answer.

• LLM methods. The model itself decides: each prompt requests both an answer-only label and a binary would_skip indicator; no external threshold is applied.

Source Reachability is a GT-aided diagnostic row. It reports whether some direct-readout source observation matches GT, and we use it only to measure how much of the benchmark is exactly visible in the source projections.

SKIP thresholds for threshold-based variants are calibrated on a held-out calibration split by maximizing \(F_{0.5}\) (\(\beta = 0.5\), weighting precision above recall).

9.3 Evaluation Metrics↩︎

All answer-only accuracy numbers reported as “macro accuracy” use a macro-over-question reduction. Let \(\mathcal{Q}\) be the 18 question templates, \(\mathcal{I}_q\) the evaluated personas for question \(q\), \(y_{i,q}\) the deterministic GT label, and \(\tilde{a}_{i,q}\) the method’s answer in answer-only mode. For selective methods, \(\tilde{a}_{i,q}\) is the answer that would have been returned before abstention when the method records one; otherwise a SKIP without a raw answer counts as incorrect. The reported forced macro accuracy is \[\mathrm{Acc}_{\mathrm{macro}} = \frac{1}{|\mathcal{Q}|} \sum_{q\in\mathcal{Q}} \frac{1}{|\mathcal{I}_q|} \sum_{i\in\mathcal{I}_q} \mathbb{1}[\tilde{a}_{i,q}=y_{i,q}].\] Coverage is micro-averaged over evaluated instances: \(\mathrm{Cov}=\sum_{q,i}\mathbb{1}[\text{answered}_{i,q}]/\sum_q|\mathcal{I}_q|\). Selective accuracy is macro-over-question on the answered subset: \[\mathrm{SelAcc}_{\mathrm{macro}} = \frac{1}{|\mathcal{Q}_{\mathrm{ans}}|} \sum_{q\in\mathcal{Q}_{\mathrm{ans}}} \frac{\sum_{i\in\mathcal{I}_q}\mathbb{1}[\text{answered}_{i,q}\wedge a_{i,q}=y_{i,q}]}{\sum_{i\in\mathcal{I}_q}\mathbb{1}[\text{answered}_{i,q}]},\] where \(\mathcal{Q}_{\mathrm{ans}}\) excludes question templates for which a method abstains on every evaluated persona. Thresholds are selected on the calibration split by maximizing \(F_{0.5}\). In the released code, this uses \(P=\mathrm{SelAcc}_{\mathrm{macro}}\), \(R=(\mathrm{SelAcc}_{\mathrm{macro}}\cdot \mathrm{Cov})/\mathrm{Acc}_{\mathrm{macro}}\), and \[F_\beta = \frac{(1+\beta^2)PR}{\beta^2P+R},\qquad \beta=0.5.\] Main-table confidence intervals pool all four held-out test seeds and use percentile bootstrap resampling over seed-persona clusters; appendix tables marked “4-seed mean” instead average the stated cell metric across seeds.

9.4 LLM Configurations and Information Conditions↩︎

T1 and T2 methods share a common extraction stage in which an LLM converts each NL memory document into structured atoms \(\hat{\mu}^m(s,q)\) for all (source, question) pairs. We use GPT-5.4 [25] as the primary extraction model with reasoning effort set to maximum, so \(\hat{\mu}\) denotes \(\hat{\mu}^{\mathrm{GPT}}\) unless stated otherwise. The extraction prompt constrains the LLM to factual recovery only: it reports what each source says without cross-source reasoning or conflict resolution. As a cross-extractor robustness check, we also extract with Gemini 3.1 Pro Preview and feed the resulting atoms into fusion methods (Appendix 12.6). A direct \(L_3\) readout uses hand-coded Python functions that read the structured JSON fields directly, with zero LLM involvement.

9.4.0.1 Released-cache boundary.

The input column in Table [tab:3] denotes prediction-time input. In the API-free released reproduction, fitted T1/T2 rows labeled \(\hat{\mu}\) fit source-selection, fusion, and SKIP parameters on deterministic train/calibration direct-readout atoms \(\mu^\ast\), then evaluate on frozen GPT-5.4 held-out test extractions \(\hat{\mu}\). Rows labeled \(\mu^\ast\) use direct-readout atoms for train, calibration, and test. Thus the reported \(\hat{\mu}\) fusion rows measure a structured resolver trained from clean source-to-GT supervision and deployed on extracted test atoms; refitting from extracted train/calibration atoms would require regenerating non-test extractions with live APIs and is not part of the frozen no-API reproduction path.

We evaluate four LLM families under identical prompt templates with no per-family tuning:

• GPT-5.4 [25] with reasoning effort xhigh (maximum).

• Gemini 3.1 Pro Preview [47], API model gemini-3.1-pro-preview, with thinking_level/thinkingLevel=high; also feeds extraction into all fusion methods as a cross-extractor robustness check (Appendix 12.6).

• DeepSeek V3.2 [48], 2025-12 release, strong open-weight model.

• Qwen3-235B-A22B-Instruct-2507 [49], [50], 2025-07 release, largest instruction-tuned variant.

All four families run LLM-Direct and Schema-Aware across all four seeds. GPT-5.4 additionally serves as the LLM arm of the matched input 2\(\times\)​2 decomposition (§ 4.2) and as the base for a supplementary few-shot experiment on seed 1 (Appendix 10.3). The four families span two proprietary (GPT, Gemini) and two open-weight (DeepSeek, Qwen3), reducing the chance that the resolver-versus-LLM gap is an artifact of one model family. Each LLM call returns an answer label from the question’s answer space \(\mathcal{V}_q\) plus a binary would_skip indicator, allowing us to evaluate both answer-only and selective modes from a single API call.

We define four information conditions to support the factorial decomposition in § 4.2:

1. NL. The LLM reads raw NL memory (LLM-Direct).

2. Schema-Aware. The LLM receives NL memory augmented with source-bias descriptions and reliability guidance.

3. \(\hat{\mu}\) input. A method reads structured atoms \(\hat{\mu}\) extracted from NL memory.

4. \(\mu^\ast\) input. A method reads structured atoms \(\mu^\ast\) directly from the structured \(L_3\) source streams.

The 2\(\times\)​2 crossing of resolver (DSNBF vs. GPT-5.4) and input quality (\(\hat{\mu}\) vs.\(\mu^\ast\)) enables the gap decomposition reported in § 4.2. We interpret the split as a supervised-resolver versus prompted-LLM design result.

9.5 ArgRAG Adaptation↩︎

ArgRAG builds a Quantitative Bipolar Argumentation Framework (QBAF) from a claim and retrieved evidence passages. The original algorithm labels evidence as support, contradict, or irrelevant, drops irrelevant evidence, adds support/attack edges, and predicts the claim from its final strength under QE gradual semantics [35]. Our input is different: each source provides a closed-class atom, not a free-text passage. To keep this a deterministic T2 method, our label-level adaptation scores each candidate answer \(v\) as the claim “the answer is \(v\).” A non-null source atom equal to \(v\) supports that claim. A non-null atom with another label attacks it. Null atoms are irrelevant. We add no evidence-evidence edges and no source-specific weights.

Under this equal-weight, no evidence-edge adaptation, all evidence nodes have equal base strength. QE semantics is monotone increasing in supporters and decreasing in attackers. If \(N\) sources report non-null atoms, candidate \(v\) has \(n_v\) supporters and \(N-n_v\) attackers. A candidate with more votes therefore has weakly higher claim strength. Ties use the same seeded tie rule as Majority Vote. The adapted ArgRAG procedure therefore returns the plurality label for both ordinal and unordered categorical answer spaces.

9.6 Verbatim Prompts↩︎

All four LLM families (T3) consume the same instruction-style prompt. Each persona is processed by a single chat-completion call: the system message contains the task instructions and the per-question answer-space specifications, the user message contains the persona’s complete NL memory render (the same five-section document used by extraction, § 3 and Appendix 9.4), and the model returns one JSON object covering all 18 questions. We reproduce the exact prompt content below.

9.6.1 LLM-Direct System Prompt (Verbatim)↩︎

You will answer 18 survey questions about a single persona. The user message
contains the persona's complete natural-language memory document, organized
into five sections:

  ## Long-Term Background and Habits  (profile/background)
  ## Plans and Intentions             (planner)
  ## Daily Self-Reports               (daily journal)
  ## Objective Records                (timesheet/measurements)
  ## Device and Activity Records      (wearable/tracker)

For EACH of the 18 questions listed below:
  1. Read the question text carefully.
  2. Consider evidence from ALL relevant source sections.
  3. When sources disagree, use your judgment to determine the most likely
     true answer.
  4. Select exactly one answer label from the answer space (forced answer
     — you MUST pick one).
  5. Also decide: if you had the option to abstain (because evidence is too
     conflicting or insufficient for a confident judgment), would you?
     Record this as would_skip (true/false).

# Answering Principles
- Synthesize across sources. Different sources may tell different stories.
- No single source is presumed correct. Every source has potential biases.
- Forced answer required. Pick the best-supported answer even if ambiguous.
- The skip decision is separate from the forced answer.
- Use exact labels with underscores ("20_or_more", not "20 or more").
- Time windows matter (7, 14, or 30 days as specified per question).
- Cross-midnight times: treat 00:10 as 24:10.

# Output Format
Return a single JSON object with the schema given below. No prose outside
the JSON. No markdown fences.

# Survey Questions
Answer ALL 18 questions. For each question, select exactly one answer label
from the answer space.

## A1 — Sleep Duration Count (Sleep, 30 days)
In the past 30 days, how many nights did this person sleep 7 hours or more?
- fewer_than_10 (0–9 nights)
- 10_to_19 (10–19 nights)
- 20_or_more (20–30 nights)

## A2 — Long Work Days Count (Work, 30 days)
In the past 30 days, on how many days did this person work more than 9 hours?
- 0_to_3 (0–3 days)
- 4_to_7 (4–7 days)
- 8_or_more (8+ days)

## A3 — Home-Cooked Share (Diet, 30 days)
In the past 30 days, on what share of meal occasions did this person eat home-cooked food?
- less_than_40 (<40- 40_to_69 (40- 70_or_more (70
## B2 — Exercise Frequency vs Profile (Exercise, 30 days)
This person's profile describes exercising a certain number of days per week.
In the past 30 days, how did their actual exercise frequency compare?
- more_than_1_below (>1 day/week less than profile)
- within_1_day (within +/-1 day/week of profile)
- more_than_1_above (>1 day/week more than profile)
- no_frequency_described (profile has no exercise frequency)

## B3 — Weekend Work vs Profile (Work, 30 days)
This person's profile describes a specific approach to weekend work.
In the past 30 days, did the actual weekend work pattern match that description?
- matches
- does_not_match
- no_approach_described

## C2 — Social Plan Realization (Social, 14 days)
In the past 14 days, on days when a social activity was planned, what share
of those plans were actually realized?
- below_25_pct (<25- 25_to_50_pct (25- above_50_pct (>50- no_plans (no social plans in planner)

## C3 — Bedtime vs Target (Sleep, 14 days)
In the past 14 days, on days when a target bedtime was set, did the person
go to bed within 20 minutes of that target?
- within_20min_more_than_50pct
- later_more_than_50pct
- earlier_more_than_50pct
- no_targets

## D1 — Social Trend (Social, 30 days)
Comparing the first 14 days to the last 16 days of the 30-day window: did
the number of social activities per day increase, decrease, or stay the same?
- decreased
- stayed_same
- increased

## D2 — Diet vs Baseline (Diet, 30 days)
In the past 30 days, did the person's daily home-cooked meal count and total
meal count differ from the profile averages by more than 1 meal/day?
- within_1
- differs_more_than_1
- no_baseline

## E1 — Late Night Cause (Sleep, 30 days)
In the past 30 days, on nights when bedtime was after midnight, which
co-occurring factor appeared on more than 50- work_activity (overtime/unusually long work hours on >50- social_activity (social activity present on >50- no_single_factor (neither factor exceeds 50- no_late_nights

## E2 — Exercise Skip Cause (Exercise, 30 days)
In the past 30 days, on days when exercise was planned but not done, did the
person work more than 8.5 hours that same day?
- no_fewer_than_30 (<30- between_30_60 (30- yes_more_than_60 (>60
## F1 — Unplanned Social Count (Social, 30 days)
In the past 30 days, on how many days did the person attend social
activities when the planner showed no social intent?
- 0_to_3
- 4_to_6
- 7_or_more
- no_social_activities

## F2 — Tracker Missing vs Inactive (Exercise, 30 days)
In the past 30 days, on days when the fitness tracker recorded no workout,
did other sources indicate that the person exercised?
- inactive_confirmed
- both_occurred
- yes_tracker_missing

## F3 — Timesheet Missing vs Off (Work, 30 days)
In the past 30 days, on days when the work timesheet had no entry, did
other sources indicate that the person worked?
- truly_off
- both_occurred
- yes_worked_despite_no_entry

## G1 — Deliberate vs Incidental Exercise (Exercise, 30 days)
In the past 30 days, on days when the person was physically active, was the
activity deliberate exercise or incidental movement?
- incidental_movement_70plus
- mix
- deliberate_exercise_70plus
- no_activity

## G2 — Voluntary vs Obligatory Social (Social, 30 days)
In the past 30 days, when the person attended social activities, were those
activities voluntary or obligatory?
- obligatory_70plus
- mix
- voluntary_70plus
- no_meetings

## Ctrl1 — Outside Meals (Diet, 7 days)
In the past week, how many days did the person eat food prepared outside
the home?
- 0_to_1_days
- 2_to_3_days
- 4_or_more

## Ctrl2 — Short Sleep Nights (Sleep, 7 days)
In the past week, how many nights did the person sleep less than 6 hours?
- 0_nights
- 1_to_2
- 3_or_more

[Output schema: JSON object mapping each question ID to
{"answer": <one valid label>, "would_skip": <true_or_false>}.]

9.6.2 Schema-Aware System Prompt (Delta from Direct)↩︎

The Schema-Aware variant is identical to LLM-Direct except for two additions inserted into the system prompt: (i) a global Source Bias Characteristics table (Table 8), and (ii) per-question augmentations attached to each of the 18 question entries. Each per-question augmentation contains three fields: a Source Relevance row indicating which of the five sections are informative for the question, an optional Cross-Source Context Instruction (e.g., "compare against the planner’s target bedtimes"), and a Reasoning Hint that paraphrases how a knowledgeable analyst would compute or judge the answer. The user message (the NL memory render) and the output schema are identical to LLM-Direct. These additions are qualitative descriptions of bias direction and reasoning structure only; they do not expose DGP bias magnitudes, dropout rates, learned confusion matrices, GT labels, train/test split statistics, or any artifacts produced during DSNBF fitting.

Schema-Aware per-question augmentations. Summary of the per-question prompt block. Src columns: Pr=profile_ltm, Pl=planner, Sr=daily_self_report, Ob=objective_log, Dv=device_log (\(✔\) = relevant). Ctx = cross-source context instruction (blank if none). Hint = computation/judgment hint summarized from the prompt.

The table above summarizes the per-question Schema-Aware prompt block; the full prompt source files are released alongside the dataset.

9.6.3 Output Schema (Verbatim)↩︎

Both LLM-Direct and Schema-Aware return one JSON object per persona with the same schema:

{
  "answers": {
    "A1": {"answer": "<label>", "would_skip": <true|false>},
    ...
    "Ctrl2": {"answer": "<label>", "would_skip": <true|false>}
  }
}

Acceptance rules: (i) answers has exactly 18 keys; (ii) answer is always a valid label from that question’s answer space (never null, even when would_skip is true); (iii) labels use exact underscored strings; (iv) the JSON validates against the schema before any downstream evaluation. Records that fail validation are not coerced into labels; they are tracked as missing-output errors (§ 9.7).

9.6.4 Few-Shot Variant (Supplementary)↩︎

The few-shot experiment on seed 1 in § 4.1 augments the Direct prompt with \(k=3\) exemplars per question, drawn from the training split via a frozen, deterministic rule:

For each question qid, partition the 216 training personas by difficulty class
(stable / temporal_shift / stated_vs_revealed). Within each class, sort by
sha256("{qid}|{seed}|{class}|{persona_id}"). Pick the first persona whose GT
answer differs from already-selected exemplars; if all share the same label,
fall back to the first by hash order. This yields 3 exemplars per question
covering the 3 difficulty classes with label-diversity tie-breaking.

The class labels and persona identifiers in this rule are consumed only by the deterministic exemplar-selection script. The prompt shown to the model redacts exemplar IDs and difficulty labels, and the target persona’s ID is never included. Each exemplar consists of the full NL memory of a training persona followed by the question text and the final GT answer label. We do not provide a separate worked-solution trace, model-generated rationale, or skip demonstration; the model can still infer the reasoning pattern from the memory–label pairs. The selection rule was frozen before the pilot run and applied identically across all 18 questions on seed 1.

9.6.5 Extraction Task Template↩︎

The shared extraction stage produces one frozen atom bundle per persona. The task reads the persona’s complete five-section NL memory render together with the extraction specification: the source-question map, context map, answer spaces, edge-case rules, and per-question extraction hints. It then writes all source-question atoms in one JSON file. The operative task template is:

SYSTEM: <empty>

USER:
Read the extraction specification and the NL memory render for the current persona.
The memory render contains five source sections:
  profile_ltm, planner, daily_self_report, objective_log, device_log.

Extract mu(source, question) for ALL 18 questions x 5 sources.
Follow the source-question map exactly:
  - if a source is not mapped to a question, write null;
  - if a mapped source has no relevant evidence, write null;
  - otherwise write exactly one valid answer label from that question's
    answer space.

For questions with a context-map entry, read the specified context source
only as instructed. Do not otherwise merge evidence across sources.

Write a JSON object whose top-level keys are question IDs:
{
  "A1": {
    "profile_ltm": <label_or_null>,
    "planner": <label_or_null>,
    "daily_self_report": <label_or_null>,
    "objective_log": <label_or_null>,
    "device_log": <label_or_null>
  },
  ...
}

Each cell contains either a label from that question’s answer space or null. Null observations are passed to fusion methods as missing inputs and are never coerced into a guess. The extraction task is factual recovery only: it reports what each source says under the allowed context rules and does not resolve conflicts or access GT labels.

9.7 Inference Parameters↩︎

All LLM calls use temperature = 0.0; no maximum-output budget is tuned or reported as a method setting. Reasoning-capable models use the serving API’s native control when available: OpenAI calls set reasoning_effort=xhigh, Gemini calls set thinking_level/thinkingLevel=high, and DeepSeek V3.2 and Qwen3 calls omit reasoning-control fields. Other decoding parameters (top_p, top_k, stop, seed) are left at the API default; we do not specify them in our requests.

Calls are deterministically cached by SHA256(model | system_prompt | user_prompt) and replayed from disk across re-evaluations, so reported numbers are exactly reproducible from the released cache. Failed or empty responses are retried with exponential backoff up to 6 attempts; persistent failures are logged and carried as missing-output errors.

2pt

Vendor-specific notes: GPT-5.4 uses OpenAI’s reasoning_effort parameter, while Gemini 3.1 Pro Preview uses thinking_level in the released config and thinkingLevel in the Gemini API REST field. DeepSeek V3.2 and Qwen3-235B-A22B-Instruct-2507 omit reasoning-control fields. We did not tune these controls per family; the configuration is identical across the four LLMs apart from the model identifier and API-specific reasoning/thinking controls.

9.8 Compute Footprint↩︎

The original LLM stages used vendor APIs, but the released reproduction path replays frozen outputs and makes no live API calls. On an Apple M1 Pro laptop with 16 GB RAM, the full released reproduction takes about two hours, including structured fusion, parameter fitting, threshold calibration, and bootstrap confidence intervals.

Table [tab:api95inventory] reports the original LLM API workload (4 seeds \(\times\) 480 personas for extraction; 4 seeds \(\times\) 120 test personas for resolution, except where noted).

Measured token values are reported where exported usage logs are available. For the Gemini extraction row, the released logs cover 480 observed test-split calls; the table reports the measured per-call token averages from those calls, while the call inventory reflects the paper’s full 1,920-call workload. OpenAI token values are measured from the successful completed calls in the released 33-call batch sample; the matched-input row pools GPT calls over \(\hat{\mu}\) and \(\mu^\ast\) atom inputs. The table counts original API calls, not the number of JSON files in the released cache. The cache stores the outputs needed for API-free reproduction: test-split GPT atoms under extracted_atoms/, Gemini cross-extractor atoms under method_outputs/gemini_p2/*/extract/, frozen T3/few-shot/structured-LLM predictions under method_outputs/, and deterministic aggregate JSONs under benchmark/results/.

Each extraction call processes a single persona’s complete NL memory and returns structured atoms \(\hat{\mu}(\text{source}, \text{question})\) for all 18 questions \(\times\) 5 sources in one structured-output JSON. Each LLM-Direct, Schema-Aware, and matched input GPT-5.4 resolution call processes a single persona and returns all 18 answers and per-question would_skip flags in one JSON. Few-shot calls are issued per (persona, question) to allow per-question demonstration selection. All calls are deterministically cached by SHA256(model \(|\) system_prompt \(|\) user_prompt); re-running with unchanged inputs incurs no additional billed API usage.

10 Selective QA Details and Few-Shot Supplementary↩︎

10.1 Full Selective QA Table↩︎

Table 10 reports selective accuracy and coverage for all deployable methods with SKIP (4-seed pooled), plus the Source Reachability reference row. Sel. Acc and Cov report each method under its canonical input (GPT-5.4-extracted \(\hat{\mu}\) for T1/T2, NL for T3). Fusion methods are additionally evaluated under \(\mu^\ast\) input. Figure 4 visualizes the deployable SKIP methods; Source Reachability stays table-only as an interpretive reference (§ 4.1).

10.2 Selective QA Analysis↩︎

Fusion methods achieve high selective accuracy at moderate coverage. DSNBF reaches 85.3% selective accuracy while covering 78.3% of the testbed; under \(\mu^\ast\) input this climbs to 88.8%. All fusion SKIP thresholds were calibrated by maximizing \(F_{0.5}\) on the held-out calibration split (§ 2 and Appendix 9.2). Test set \(F_{0.5}\) (answer precision = selective accuracy, answer recall = fraction of answer-only correct instances that are answered) confirms effective calibration transfer: NBF 85.3, DSNBF 84.9, SSB 81.9, ABF 80.4. LLM self-reported abstention yields lower \(F_{0.5}\): GPT-5.4 Schema 75.0, GPT-5.4 Direct 74.1.

On the 78.3% of instances DSNBF chooses to answer, it achieves 85.3%, compared to 80.3% in answer-only mode. The ~22% of skipped instances are disproportionately difficult, consistent with the posterior identifying harder cases. ABF achieves comparable selective accuracy (83.0%) but at much lower coverage (62.0%); its calibration is more conservative. Moving from extracted input to \(\mu^\ast\) input improves selective accuracy for all fusion methods, with DSNBF gaining ~3.5 pp and NBF ~2.6 pp, while ABF gains only 0.4 pp and answers slightly fewer instances.

LLM methods abstain rarely and gain little from doing so. GPT-5.4 Direct covers 92.3% but achieves only 70.3% selective accuracy, barely above its answer-only macro accuracy of 68.6%. GPT-5.4 Schema shows the same pattern: 71.0% at 95.4% coverage. Gemini abstains even less: Gemini Direct covers 98.5% at 68.4% selective accuracy. The open-weight families push this pattern further: DeepSeek V3.2 never abstains (100.0% coverage on both Direct and Schema), while Qwen3 abstains on fewer than 1.2% of instances.

10.2.0.1 Protocol asymmetry.

Fusion methods use thresholds tuned on a held-out calibration split (§ 3); LLM methods return a self-reported binary would_skip flag. We therefore read Table 10 as a comparison of the interfaces evaluated here: fusion exposes posterior margins that can be calibrated after training, whereas the prompt-only interface gives a skip decision without a continuous confidence score. In deployment settings where \(\mathrm{Cost}(\mathrm{wrong}) \gg \mathrm{Cost}(\mathrm{skip})\), that tunable uncertainty interface is a practical advantage.

Source Reachability is the only reference row. Its 93.2% value is the GT-aided direct-readout reachability rate: the fraction of instances where some \(\mu^\ast\) source atom exactly matches GT. The complementary 6.8% has no exact visible GT match, but learned inference can still recover some of those cases. Table 10 keeps the row next to selective methods only as an interpretive reference; its structure by type is analyzed in § 4.3 and Appendix 11.1.

10.3 Few-Shot Supplementary Check↩︎

A few-shot variant run on seed 1 (GPT-5.4, 3 exemplars per question) achieves 75.4% answer-only accuracy, the highest among all NL-input LLM configurations in our experiments. The +6.5 pp gain over GPT-5.4 Direct (seed 1: 68.9%) is the largest prompt-level improvement among NL-input configurations. The result still trails the main DSNBF result (80.3%, 4-seed), with the seed-scope caveat, so we treat it as a supplementary consistency check; the primary finding remains the four-seed comparison.

This prompt-side check also clarifies the scope of stronger prompting. The locked scoring format requests only labels and skip flags; we did not add externalized chain-of-thought traces. GPT-5.4 and Gemini 3.1 Pro Preview already use vendor-maximum reasoning/thinking controls. Self-consistency would multiply LLM calls and falls outside the cost-matched single-call regime. The matched \(\mu^\ast\) input condition tests direct source readout; tools that compute new statistics would define a different information condition. In the structured-input setting evaluated here, GPT-5.4 trails fusion by 8.9 pp.

The few-shot variant contains no abstention demonstrations, so the model relies solely on written instructions for skip decisions. It produces a 2.6% skip rate, yielding 76.1% selective accuracy at 97.4% coverage, only 0.7 pp above its answer-only accuracy. Among the skipped instances, answer-only accuracy drops to 46.4%, suggesting that the model identifies some difficult cases but abstains too rarely to materially improve selective performance. The largest remaining gaps to DSNBF occur on F-Missing-Data (63.1% vs. 77.9%, \(-\)​14.8 pp) and G-Annotation (53.8% vs. 63.5%, \(-\)​9.7 pp), the two types where examples provide weaker support for bias-aware reasoning.

11 Extended Diagnostic Tables↩︎

11.1 Per-Type Accuracy↩︎

Table 11 expands Table 3 with the full set of 13 comparison columns: three T2 fusion methods (DSNBF, NBF, SSB), the matched input GPT reference (GPT-\(\mu^\ast\), GPT-5.4 reading the same \(\mu^\ast\) atoms as structured prompt input), eight T3 LLM settings (four families \(\times\) two prompting regimes), and the Source Reachability reference by type. Rows are ordered by DSNBF accuracy descending. Bold marks the best method entry per type.

2.5pt

Figure 5 gives a colored-table view of the same values, so type-specific failure signatures can be read without scanning the full numeric table.

11.2 Difficulty-Class Breakdown↩︎

5pt

3pt

Table 12 reveals a bias-modeling gradient on the stated_vs_revealed (svr) class: overall svr accuracy drops from stable by only \(-\)​5.1 pp for DSNBF/NBF and \(-\)​1.8 pp for SSB, by \(-\)​10.6 pp for ABF, and by \(-\)​16.9 pp (BCF) and \(-\)​16.4 pp (MV). The gradient tracks how finely each method models source-specific distortion: MV ignores it entirely, BCF captures only a scalar shift, ABF blends bias-aware and bias-free kernels, while NBF/DSNBF learn full per-source confusion matrices (and, for DSNBF, additionally condition on inferred persona class). The type-level view exposes where this collapse concentrates: on D-Temp, BCF and MV fall to 33.8% and 47.2% under svr (vs. DSNBF 76.2%); on A-Arbit, ABF/BCF/MV drop to 43-47%; on F-Miss, BCF and MV drop to 44-49%. In contrast, SSB’s robustness is uneven: on E-Factor svr it reaches 87.8% (the highest among non-reference methods), because it picks the objective_log source and discards the misleading self-report; on A-Arbit svr it ties the confusion-matrix methods (81.5%) for the same reason.

Table 13 shows that frontier LLM Schema methods degrade sharply under svr: GPT Schema \(-\)​13.4 pp, Gemini Schema \(-\)​13.4 pp. The near-identical degradation across GPT and Gemini indicates a structural limitation of schema prompting, not a family-specific artifact. Schema accuracy falls below Direct under svr (GPT: 59.8% vs. 61.8%; Gemini: 60.3% vs. 61.2%), indicating overcorrection when stated preferences and revealed behavior diverge systematically. The type-level breakdown localizes where this collapse concentrates: D-Temp svr is where Schema fails most dramatically (GPT-S 39.4%, Gem-S 40.0%, vs. their Direct counterparts at 59.7%); E-Factor svr is the second collapse zone for GPT/Gem Schema (51.6-52.2%, far below SSB’s 87.8%). B-Ident svr is where Schema actually outperforms Direct (GPT-S 91.2%, Gem-S 85.3%), confirming schema descriptions help on identity-style questions but harm trend/factor reasoning under stated-vs-revealed bias.

Open-weight models show a different pattern: their svr drops are near zero or positive (DS Direct +1.2, DS Schema +5.1, QW Direct +2.9, QW Schema +4.4), but only because their stable-class baselines are already low (DS 55-58%, QW 41-46%). DeepSeek Schema actually improves on svr (+5.1 pp vs. stable), suggesting that schema descriptions interact differently with weaker models on directionally biased data. Qwen3 collapses on E-Factor across all difficulty classes (3.4-26.9%), indicating it cannot perform multi-source factor attribution regardless of bias direction.

Across the no-skip tables (E2a, E2b), the relative method ordering on svr is type-stable: DSNBF/NBF lead on 7 of 8 types, BCF/MV trail on 7 of 8 (Annot is the only inversion, where bias modeling no longer helps). The cross-method spread on a given type also tracks how much the per-source signals diverge: D-Temp svr exhibits the largest spread (DSNBF 76.2 vs. BCF 33.8, gap about 42 pp), while B-Ident svr is the tightest among the stronger fusion methods (DSNBF, NBF, ABF, and SSB all fall within 2.5 pp). This monotone coupling between signal-divergence and method-ordering is a sanity check on the diagnostic design: harder difficulty classes do not flatten the comparison; they sharpen it.

Tables 14 and 15 report the SKIP variants on the same per-type \(\times\) per-difficulty grid using selective accuracy (correct over answered, excluding skipped cells from the denominator); the second number in each cell is coverage (share of cells answered). The footer convention is stable \(-\) SvR selective accuracy: positive values mean SvR is lower than stable, and negative values mean SvR is higher than stable.

The selective tables show a sharper split between methods that expose useful uncertainty and methods that do not. DSNBF+S and NBF+S keep their SvR deficits small (1.5 and 3.3 pp), while the no-skip comparators BCF and MV drop by more than 16 pp. ABF+S sits between them: it abstains aggressively where GT is available (D-Temp SvR coverage 39.4%, Annot SvR 43.8%) but still answers enough hard cells to leave an 8.5 pp deficit. T3 LLMs behave differently. GPT and Gemini have large positive footer drops (\(+\)​8.6 to \(+\)​13.5 pp), driven mainly by D-Temp overcorrection under Schema prompting; DeepSeek and Qwen3 show negative footer drops (\(-\)​0.7 to \(-\)​4.0 pp) because their stable-class baselines are weak and their skip rates are near zero. Overall, abstention helps most when the underlying scorer can identify low-confidence cases, which favors per-source confusion-matrix methods over threshold-only heuristics and end-to-end LLM judges.

Comparing answer-only tables (E2a, E2b) against selective tables (E2a-skip, E2b-skip) quantifies the marginal value of abstention per family. DSNBF and NBF show the clearest robustness recovery: answer-only accuracy is 5.1 pp lower on SvR than stable, whereas selective accuracy is lower by only 1.5 and 3.3 pp after thresholding. ABF abstains aggressively (SvR coverage 39.4% on D-Temp, 43.8% on Annot); its selective gap improves modestly, from a 10.6 pp answer-only drop to an 8.5 pp selective-accuracy drop. SSB+S shows a 1.1 pp selective drop. BCF and MV are no-skip comparators with 100% coverage, so their answer-only and selective-table entries are numerically identical. On T3, GPT and Gemini +S keep coverage near 95% and pay the schema-overcorrection penalty in selective accuracy (Schema drops of 13.5 and 13.1 pp); DS and Qwen3 +S report coverage of 100% in every cell, indicating that the +SKIP prompt template does not elicit abstention from these models regardless of difficulty class. Abstention is therefore a model-capability axis orthogonal to raw QA accuracy: a method’s SvR robustness depends both on whether it knows when to abstain and on whether its cost function makes abstention worthwhile.

3pt

2pt

11.3 Prediction Distributions on Failure Questions↩︎

Tables 16 and 17 report Schema-Aware prediction distributions on four representative high-failure templates, split by difficulty class: E1 and E2 (factor attribution) in Table 16, C2 (plan-reality) and F3 (missing-data) in Table 17. These diagnostic distributions use the locked artifacts available for this analysis: GPT seed 1 and 4-seed pooled DeepSeek/Qwen3 runs; Gemini is omitted to keep the table focused. The four diagnostic patterns identified after Table 17 characterise distinct failure modes across model families.

3pt

3pt

Four diagnostic patterns emerge from these distributions.

Qwen3 saliency bias. On E1, Qwen3’s modal prediction flips from no_single_factor on stable (54.4%) to work_activity on TS (62.5%) and SvR (80.0%), despite SvR’s GT being 92.5% no_single_factor. The TS/SvR fixation on work_activity drives E1 accuracy down to 33.1% (TS) and 22.5% (SvR). On E2, Qwen3 never predicts the GT modal no_fewer_than_30 in any difficulty and reaches only 5.0% on SvR. The errors point to a narratively salient factor dominating elimination and negation answers under bias.

DeepSeek plan-completion overestimation. On C2, DS predicts above_50_pct at 71.9% (St), 58.1% (TS), and 61.9% (SvR), although the GT share is only \(12\)–\(30\)%; the GT modal label 25_to_50_pct barely appears (\(<1.3\)% in every difficulty class). On F3, DS spreads mass across all three options and changes its modal answer with difficulty: stable favors truly_off (65.0%, GT only 12.5%), TS favors yes_worked (44.4%), and SvR favors both_occurred (47.5%). The answer distribution therefore shifts with persona difficulty rather than preserving one stable error mode.

Localized catastrophic profiles. Using 30% per-(question, difficulty) accuracy as the catastrophic threshold, Qwen3 fails on E1-St (15.6%), E1-SvR (22.5%), E2-St (26.2%), E2-TS (20.6%), E2-SvR (5.0%), and F3-SvR (13.8%). DS fails on E1-St (24.4%), C2-St (20.0%), C2-SvR (26.2%), F3-St (28.7%), and F3-TS (25.6%). GPT has one catastrophic cell, E1-SvR (17.5%). Cells in the 30–50% range remain weak but fall outside this threshold. SvR concentrates more catastrophic cells across all three models, reinforcing stated-vs-revealed bias as the hardest difficulty class for schema-prompted LLMs.

GPT’s localized E1-SvR collapse. GPT’s sharpest failure is concentrated in one (question, difficulty) cell: on E1 SvR, it predicts social_activity at 87.5% while GT is 92.5% no_single_factor. This mirrors the Qwen3 failure in form but with a different salient factor; when stated and revealed signals disagree, GPT commits to one plausible explanation instead of the elimination answer. The error stays localized, with F3-SvR accuracy at 87.5%.

12 Additional Robustness Analyses↩︎

12.1 Cross-Seed Stability↩︎

We evaluate all methods across four independent random seeds, each instantiating 480 fresh personas. Table 18 reports per-seed accuracy for key methods (\(\mu^\ast\) input in this diagnostic table).

Cross-seed standard deviations are \(\leq\)​1.1 pp for all methods. Fusion methods are the most stable (\(\sigma \leq 0.8\)). All four LLM families show \(\sigma \leq 1.1\) pp, confirming that the main findings are stable and not driven by idiosyncratic seed-specific persona configurations. In particular, the 10+ pp trained-resolver vs. zero-shot/prompted LLM gap for frontier models and the 30+ pp gap for open-weight models are reproduced across all four seeds.

12.2 Training Size Sensitivity↩︎

We vary the training set size from 50 to 216 personas (the full training split) and re-train all parametric fusion methods (4-seed mean \(\pm\) \(\sigma\), \(\mu^\ast\) input). Table 19 reports the results.

3pt

Performance largely saturates by \(N = 100\): the gain from doubling the training set (100 \(\to\) 216) is \(\leq\)​0.36 pp (4-seed mean). This suggests the source-to-GT mappings are regular enough to estimate from the train split. MV is invariant to training size by construction (no learnable parameters). ABF and BCF have learnable parameters but change by \(\leq\)​0.1 pp over this range, suggesting their low-dimensional parameterizations also saturate quickly.

12.3 GPT-5.4 vs. Gemini Cross-Condition Comparison↩︎

GPT-5.4 and Gemini remain within \(\pm\)​2.2 pp under all four information conditions (seed 1). The Direct and Schema rows are consistent with the 4-seed pooled T3 rows in Table 2. The \(\hat{\mu}\) input and \(\mu^\ast\) input rows overlap with the LLM column of Table 3 (§ 4.3). This aggregate similarity coexists with large per-question extraction differences in Table 24.

12.4 DGP Perturbation↩︎

We vary the DGP’s two core distributional parameters (bias scale and device dropout scale) across a 3\(\times\)​3 grid (\(\{0.5\times, 1.0\times, 2.0\times\}^2\)), yielding 9 distribution variants per seed. The default configuration is \(1.0\times / 1.0\times\). Table 21 reports macro accuracy (%) for key methods across all 9 variants (4-seed mean \(\pm\) \(\sigma\), \(\mu^\ast\) input).

3pt

The mean pairwise Kendall \(\tau\) across all \(\binom{9}{2} = 36\) variant-pair ranking comparisons (canonical 7-column set: DSNBF, NBF, ABF, BCF, MV, SSB-G, Source Reachability reference) is 0.77, with a bootstrap 95% CI of \([0.74, 0.80]\) over 144 pairs (4 seeds \(\times\) 36 pairs).¹ Every one of the 144 pairs has \(\tau > 0\), and 143 of 144 have \(\tau > 0.5\). No aggregate cell shows a full rank reversal, and only one falls below the strong-correlation threshold.

Each variant is retrained on its own training split, so this tests whether the model class is appropriate across bias scales, not whether specific parameters transfer. Under simultaneously doubled bias and dropout (b2.0, d2.0), bias-unaware methods (MV) degrade sharply (\(-\)​6.9 pp from the default setting, 4-seed mean), while DSNBF degrades only \(-\)​1.6 pp. DSNBF leads or ties (within cross-seed variability) every other fusion method in 32/36 \(\times\) (seed, variant) cells; in the remaining 4 cells, NBF leads DSNBF by \(< 0.7\) pp, well within \(\sigma_{\text{DSNBF}}\).

SSB-G’s accuracy range across the 9 DGP variants is 21.3 pp (4-seed mean), compared to 3.7 pp for DSNBF and 4.6 pp for NBF. Under low bias (b0.5), SSB-G benefits from device-log dominance and performs well (76–79%); under high bias (b2.0), device-log dropout compounds with amplified distortion in other sources, and SSB-G collapses to 57–60%. The global source-selection heuristic cannot adapt to distributional shifts where no single source remains uniformly reliable, whereas confusion-matrix methods redistribute weight across sources as bias conditions change.

12.5 Extraction Noise Tolerance (Full Analysis)↩︎

We simulate extraction errors by randomly flipping a fraction of \(\mu^\ast\) values to uniformly random labels. Methods are fit and calibrated on the unperturbed train and calibration splits; only test atoms are flipped. Table 22 reports macro accuracy under increasing noise rates (4-seed mean, \(\mu^\ast\) as baseline).

3pt

At 30% noise, all learned methods converge toward the Majority Class baseline (57.1%), as expected: random flips destroy the signal that confusion matrices rely on. Majority Class is noise-invariant because it ignores input entirely and always predicts the per-question modal label.

A reversal occurs at \(\epsilon \geq 0.1\): NBF overtakes DSNBF, and the gap widens from \(-\)​0.6 pp at \(\epsilon = 0.1\) to \(-\)​4.7 pp at \(\epsilon = 0.5\) (4-seed mean). DSNBF’s difficulty-stratified weights concentrate influence on hard personas, which are also the most noise-sensitive. DSNBF’s cross-seed variability increases under perturbation, while NBF remains more stable. This pattern indicates that difficulty conditioning helps under clean atoms but can amplify random extraction noise when the atom table is heavily corrupted.

12.6 Cross-Extractor Robustness↩︎

Main paper results use GPT-5.4 as the extraction backend (NL \(\to \hat{\mu}\)). To answer whether fusion accuracy is an artifact of a single extractor, we repeat the full extraction pipeline with Gemini 3.1 Pro (the other closed frontier family evaluated as an LLM-Direct answerer) on all four seeds and feed the resulting atoms \(\hat{\mu}^{\text{Gem}}\) to every fusion method. The split and fitted fusion parameters are held fixed; only the test-set extraction backend changes. Table 23 compares the two extraction backends; the direct readout column \(\mu^\ast\) is the no NL extraction reference.

The extractor swap preserves the main method ordering: under both extractors, DSNBF \(\approx\) NBF \(>\) SSB \(>\) ABF \(\approx\) BCF \(\approx\) MajVote, with no rank flip among the top four. DSNBF-NS drops only \(0.7\) pp under Gemini extraction, comparable to its own cross-seed variability under each extractor (\(\sigma_{\text{GPT}} = 0.2\), \(\sigma_{\text{Gem}} = 0.6\) pp; 4-seed pstdev). This supports the § 4.2 attribution of the trained-resolver vs. prompted-LLM gap across two comparable closed frontier extractors. Sensitivity is method dependent: SSB loses \(2.3\) pp because it commits to a single source and inherits that source’s extraction quality directly, while ABF is essentially unchanged (\(\Delta = +0.4\) pp; sign mixed across seeds). NBF and DSNBF are less sensitive because their learned source-to-GT maps tolerate small extractor changes. The table tests transfer under a fixed fitted resolver; per-extractor refitting would be a separate condition.

Selective variants show the same qualitative pattern: DSNBF selective accuracy changes from \(85.3\%\) at \(78.3\%\) coverage with GPT extraction to \(83.1\%\) at \(79.3\%\) coverage with Gemini extraction (all 4-seed means; full table in the data release). In this GPT-to-Gemini check, the same thresholding protocol remains effective.

12.6.0.1 Scope.

We test GPT-5.4 and Gemini 3.1 Pro. Their LLM-Direct accuracies are within \(\pm 1\) pp of each other (Table 20), so this check compares two broadly similar extractor tiers. The statement concerns answer-level Direct accuracy, while the per-question atom extraction differences are reported below. An extractor with substantially lower baseline quality would be expected to widen the downstream gap in proportion to its extraction faithfulness rate (Appendix 8.8).

12.6.0.2 Per-question extraction accuracy.

To diagnose where the two extractors diverge, Table 24 reports per-question atom-level extraction accuracy \(P(\hat{\mu} = \mu^\ast)\) against the direct readout, aggregated over all four seeds (480 test personas per question, non-null-cell denominator).

2.5pt

The per-question extraction differences are clustered. Most of the overall Gemini deficit (\(-2.0\) to \(-2.3\) pp across difficulties) comes from five question families that degrade on all three difficulty classes: Ctrl1 (\(-17\) to \(-21\)), E1 (\(-12\) to \(-15\)), F2 (\(-8\) to \(-11\)), F3 (\(-6\) to \(-9\) on St/TS), and B3 (especially on SvR at \(-9.5\)). Ctrl1 and E1 show Gemini underperformance that does not track persona difficulty, consistent with a structural extractor weakness. Some gaps are difficulty-sensitive: Gemini outperforms GPT on A3 SvR (\(+6.7\)) and A2 St/TS (\(+5.4\)/\(+6.4\)), but A2 reverses on SvR (\(-0.9\)), while B3 widens on SvR. The error balance is also asymmetric. Gemini produces more false-negative cells (\(\mu^\ast\) non-null, Gemini emits null), consistent with a more conservative extractor that drops atoms under uncertainty; we document the per-seed counts in the data release.

Despite per-(question, difficulty) extraction gaps reaching \(20.8\) pp, the fusion-level macro accuracy gap (Table 23) stays within \(|\Delta| \leq 1.2\) pp for confusion-matrix methods (ABF \(+0.4\), DSNBF \(-0.7\), NBF \(-0.9\), BCF \(-1.2\)). The same fitted confusion-matrix resolver absorbs these extractor differences at the aggregate level. SSB’s steeper \(-2.3\) pp drop reflects its single-source design: it reads one chosen source atom and has no learned cross-source absorption mechanism. The selective variants in Appendix 10.2 show the same pattern.

12.7 Cross-Parameter Transfer Without Refit↩︎

Table 6 tests a stricter transfer setting than Table 21. DSNBF is fit and calibrated only on the default projection setting (\(b{=}1.0,d{=}1.0\)), then evaluated without refit on shifted bias/dropout variants. The target-retrained column is recomputed on the same rebuilt variant, so both columns use the same test data. All rows use direct-readout atoms \(\mu^\ast\).

Across the eight shifted variants, DSNBF trained only on the default projection setting is 1.6 pp below DSNBF refit on each target setting on average, with a worst gap of 3.8 pp at \(b{=}2.0,d{=}0.5\). This supports moderate cross-parameter stability across projection settings. The negative gaps also show that target-distribution calibration can still help under larger shifts.

References↩︎

---

1. We report aggregate effect-size statistics rather than per-pair significance because each pairwise Kendall test compares only \(n = 7\) ranks, which yields a theoretical \(p\)-value floor of \(\approx 0.0014\) regardless of the underlying data. The aggregate signal across 144 pairs (mean \(\tau\) lies \(\approx 52\) standard errors above the random-ranking null of 0) is the appropriate evidence for ranking stability.↩︎