1 Introduction↩︎

The standard worry about long-context autoregressive generation is exponential. If every token has independent error probability \(e\), then after \(n\) tokens the chance of a fully correct output is \((1-e)^n\), which collapses to zero for any nontrivial \(e\) and large enough \(n\) [1], [2]. Earlier work in this series [3] argued that this worry is misplaced because errors are not distributed uniformly across tokens: only \(5\)–\(10\%\) of tokens are “key” (genuinely dependent on long-range context [4]), while the remaining majority become nearly deterministic once enough context accumulates. Replacing \((1-e)^n\) with the two-rate model \(P(\text{correct}) = (1-e_{\text{key}})^{k} (1-e_{\text{non}})^{n-k}\) with \(k \ll n\) scaling sublinearly in \(n\) recovers the observed long-context coherence and converts the reliability question from “how does \(n\) grow?” to “how does \(k\) grow?”.

1.0.0.1 From sparse tokens to recurring patterns.

This paper takes the next step. Sparsity tells us where errors live; the natural follow-up is what they are. Recent failure-mode atlases supply a striking empirical answer: errors are not only sparse, they are also repetitive. ErrorAtlas [5], covering 83 models across 35 datasets, organises observed failures into 17 named categories sorted by prevalence, with a long-tailed distribution heavily concentrated in the head. In code, two error types (AssertionError and NameError) cover 86.35% of failures on HumanEval across 14 LLMs, replicated across 23 models on HumanEval Pro and MBPP Pro [6]. In math, MWPES-300K [7] categorises 304,865 errors from 15 LLMs across four math word-problem datasets and reports that dataset characteristics shape error patterns systematically. The same picture appears in multi-hop QA [8], agentic tool use [9], and retrieval-augmented generation [10]. These observations suggest a third layer in the reliability architecture, complementing key-token sparsity (Layer 1) and within-key stratified-manifold structure (Layer 2 in [3]). Within the small set of key tokens, only a fraction \(\beta\) produce hard failures, and inside bounded deployment patches those hard failures appear to cluster into a finite or effectively capped catalogue of recurring modes whose size grows much more slowly than the number of observed events.

1.0.0.2 Contributions.

The central contribution is a shift in the reliability object. Universal LLM reliability is not a finite-library problem, but patch-local reliability can be treated as catalogue discovery and intervention coverage. We formalise this transition with two propositions and one corollary.

Proposition 1 is the worst-case-mode-wise negative result: if an unbounded domain keeps producing intervention-distinguishable failures, no finite intervention dictionary can guarantee bounded residual error for every mode of the domain. Corollary 1 is the inverse-discovery implication of the logarithmic upper bound: the bound cannot accommodate linearly more distinct tail modes without exponentially more observed hard-failure events, so open-domain tail discovery has rapidly diminishing returns. Proposition 2 is the matching positive result inside a fixed deployment patch. Under log active-mode exposure and head-heavy coverage, a sufficient per-hard-decision intervention budget satisfies \(m \geq \lceil |C_{\text{eff}}|^{1 - \varepsilon / e_{\text{hard}}} \rceil\), with a doubly-logarithmic sequence-length rate as the optimistic pre-cap special case and a domain-constant budget once the patch catalogue saturates. The sequence-level analogue is strictly tighter and approaches full-catalogue coverage as \(k\) grows; we surface that asymmetry rather than bury it.

Around this transition the paper does four supporting things. A three-layer framework, comprising sparsity (\(\alpha\)), hard-token stratification (\(\beta\)), and patch-local mode catalogue (\(|C_D|\)), separates where errors occur, what recurring forms they take, and which capability interventions address them (§3). Logarithmic mode discovery is stated as an empirical postulate, not a theorem, calibrated against ErrorAtlas, HumanEval, and MWPES with \(\sigma \in [0.87, 1.85]\) across anchors and \(\sigma \approx 1.85\) carried as a conservative planning value. Section 4 synthesises evidence for failure clustering, cluster-selective interventions, and sublinear length scaling, drawing on \(\sim\)​60 prior published results including the six-axis capability-elimination harvest of 28 quantitatively-anchored citations stratified into Patterns A/B/C (Appendix 9). And the most-cited steep-decay counter-evidence (Appendix 10) is re-audited and shown to decay primarily over task-structure variables, including compositional graph size, fact count, log-time horizon, capacity threshold, and evidence scope, rather than raw token length.

2 Related Work↩︎

2.0.0.1 Failure-mode taxonomies.

ErrorAtlas [5], MWPES-300K [7], the HumanEval categorisation of [6], and the RFMDataset [11] together suggest that, at any given corpus size, the number of distinct named failure modes is small (typically 8–20) and the cumulative coverage of the top modes is high. Domain-specific taxonomies for multi-hop QA [8], multi-agent systems [9], and tool-augmented agents [12] report the same pattern.

2.0.0.2 Targeted interventions.

Each named cluster has been the subject of focused intervention research. Python execution closes most of the arithmetic-error cluster [13]–[15]; constrained decoding eliminates format violations [16]–[18]; execution feedback resolves most code-logic errors [19], [20]; process reward models reduce step-level reasoning errors [21], [22]; RAG strongly reduces hallucinations [10]; preference optimisation curbs over-refusal [23]; structured uncertainty fixes most tool-call failures [24]. Two structural observations recur: each intervention is cluster-selective (residuals belong to a different named cluster, not to the targeted one), and additivity is approximate but not perfect [25], [26].

2.0.0.3 Length-decay benchmarks.

A parallel literature measures the shape of the reliability decay curve. Mild-decay results (Loong [27], GSM-\(\infty\) [28], RULER [29], anchor-based LLMs [30]) report log-linear, sigmoidal, or threshold-like decay inconsistent with smooth \((1-\varepsilon)^n\). Steep-decay results [2], [31]–[33] are sometimes cited as exponential-compounding evidence. The apparent tension is reduced on careful reading: each of the steep-decay results we re-audit decays over a variable distinct from raw token length, which Appendix 10 documents.

2.0.0.4 Gap.

What is missing is a quantitative bridge between the small, repeating taxonomy catalogue and a polylog intervention budget. We provide that bridge in §3.

3 Theoretical Framework↩︎

3.0.0.1 Four levels of \(|C|\).

Before any proposition, a definitional setup. LLM error analysis routinely conflates four objects that the rest of this paper needs to keep apart:

• L1: failure events. Concrete LLM errors observed in a benchmark; the raw data behind any taxonomy.

• L2: empirical taxonomy categories. Researcher-chosen labels grouping L1 events. ErrorAtlas’s 17 categories, MWPES’s top-4 classes, and HumanEval’s AssertionError/NameError partition all live here. L2 is observable but depends on the taxonomer’s resolution choices.

• L3: latent failure modes. The unobserved underlying clusters that L2 approximates. L3 is the quantity Postulate 1 is morally about, but it cannot be measured directly; we treat L2 as a noisy proxy for L3.

• L4: capability axes / interventions. The engineering unit: a Python interpreter, a constrained decoder, a retrieval-augmented generator. L4 is coarser than L2 in practice; one capability axis typically targets several L2 categories at once (§4, Claim B).

Throughout this section, \(|C|\) refers to the L2 count: that is what published taxonomies report. Postulate 1 is therefore engineering-relevant because L4 is coarser than L2 (so a small intervention library can sweep across many categories at once), and epistemically conditional because L2 is a noisy proxy for the actual L3 mode catalogue whose faithfulness has not been independently measured. Tagging the formal claims at the right level keeps the framework honest where casual error-talk slips.

3.0.0.2 Roadmap.

The framework separates three questions that are often conflated. First, where do errors occur? Only a sparse subset of key decisions. Second, what kinds of errors recur? A local catalogue of failure modes. Third, what fixes them? A smaller library of capability interventions. The two propositions of §3.4 formalise the transition from universal impossibility to patch-local tractability: there is no finite intervention dictionary that covers all possible deployments, but inside a fixed deployment patch a sufficient library is small and slowly growing.

3.1 \(\beta\)-stratification of key tokens↩︎

The two-rate model of [3] distinguishes \(k\) key tokens (error rate \(e_{\text{key}}\)) from \(n - k\) non-key tokens (\(e_{\text{non}} \ll e_{\text{key}}\)). Empirical atlases suggest that even within the key-token class errors are not uniformly distributed: most key tokens are “decisions” for which the model has stable representations; only a fraction concentrate the actual failures.

Under Definition 1, the composed sequence-level reliability becomes \[\label{eq:three-rate} P(\text{correct}) \;=\; (1 - e_{\text{hard}})^{\beta k}\,(1 - e_{\text{easy}})^{(1 - \beta) k}\,(1 - e_{\text{non}})^{n - k}.\tag{1}\] We do not assume iid token errors: the three rates summarise conditional per-decision failure hazards after conditioning on prior decisions being correct, and grouping by stratum yields the multiplicative survival expression. Because \(e_{\text{easy}} \approxe_{\text{non}}\) once context accumulates, the failure rate is dominated by the \(\beta k\) hard tokens, and we treat \(e_{\text{hard}}\) as the load-bearing quantity. The parameter \(\beta\) is latent: failure-mode atlases report \(\Pr(\text{category} \mid \text{error occurred})\), which does not identify \(\Pr(\text{hard} \mid \text{key-token decision})\). We carry \(\beta\) symbolically.

3.2 Empirical postulate: logarithmic mode discovery↩︎

Stratification names a target, the \(\beta k\) hard-token decisions, but does not yet tell us how many distinct ways those hard decisions can fail. The empirical answer from Section 1 is that failures repeat: a small number of recurring patterns covers most of the mass. The next question is how the catalogue’s size grows with the number of observed failures, because that is the quantity an intervention library has to keep up with.

Two candidates exist in the type–token literature: Heaps’ law (power-law, \(|C| \approx K \cdot k_{\text{hard}}^{b}\) with \(b \in [0.4, 0.6]\) [34]), and logarithmic discovery. Zipfian rank-frequency does not imply logarithmic growth. Heaps’ law is the correct type–token consequence of Zipf-distributed events, and it is power-law, not logarithmic. We therefore state logarithmic mode discovery as an empirical postulate, defensible by direct measurement of error taxonomies, not as a theorem.

Two sample-size variables matter, and they are easy to conflate. Let \(h(n) = \beta k(n)\) denote the number of hard-token decisions in a single sequence of length \(n\); let \(T\) denote the number of observed hard-failure events in a corpus used to discover the domain catalogue. These are not the same object: \(h(n)\) controls per-sequence exposure, \(T\) controls empirical discovery. We write \(C_D\) for the full reachable catalogue inside domain patch \(D\), \(C_{\text{seen},D}(T)\) for the subset discovered after \(T\) sampled failures, and \(C_{\text{active},D}(n)\) for the subset a single sequence of length \(n\) can activate. We write \(\ln T\) and \(\ln h(n)\) for clarity; small-sample bounds may be read with \(\ln(1+T)\) and \(\ln(1+h(n))\), and discrete counts such as \(\beta k\) are interpreted either as expected values or as \(\lfloor \beta k \rfloor\) asymptotically. These reformulations do not affect any rate claim below.

The two bounds answer different questions, and the rest of the paper picks the right one at each point. Proposition 2’s sequence-length scaling rate is governed by \(C_{\text{active},D}(n)\) via Assumption 1: how much of the catalogue can a single sequence touch? Full domain-library budgeting, the practical engineering target for a deployed system, uses \(C_D\): how large does the library need to be to cover the domain’s reachable failure modes? The cap \(|C_D|\) enters both: once enough has been sampled (or activated) to saturate the domain ceiling, further growth stops and the budget becomes a domain-constant function of \(|C_D|\) alone.

3.2.0.1 Empirical calibration.

Existing LLM error taxonomies provide endpoint anchors for small named catalogues at observed corpus scales. ErrorAtlas, HumanEval-style code taxonomies [6], and MWPES [7] place \(|C|\) roughly in the \(8\)–\(20\) range across corpora ranging from approximately \(10^4\) to \(3 \times 10^5\) observed failures, yielding \(\sigma \in [0.87, 1.85]\) under the simple \(A=0\) logarithmic calibration. We use \(\sigma \approx1.85\) as a conservative planning value, not as a fitted law. These endpoint counts are not discovery curves: the missing test is a subsample-vs-discovered-modes measurement within a fixed deployment patch. Full calibration details are in Appendix 12.

3.2.0.2 Capabilities are coarser than error categories.

The category count \(|C|\) is conservative for engineering because deployed interventions operate at the capability level, not the label level. A Python interpreter can remove execution-error components across arithmetic, unit conversion, counting, list manipulation, and date arithmetic; a constrained decoder can eliminate several structural output failures at once. The formal coverage problem is therefore closer to weighted set cover over failure mass than one-intervention-per-category counting. The ranked-category model used in Proposition 2 is a tractable proxy; the full capability harvest appears in Appendix 9.

3.2.0.3 L2 \(\to\) L4 as weighted set cover.

Formally, let each L4 intervention \(I_j\) cover a subset \(S_j \subseteq C_D\) of the L2 mode catalogue, with per-mode residual-reduction weight \(r_{ij} \in [0, 1]\) giving the fraction of mode \(i\)’s hard-error mass that \(I_j\) removes. The deployment problem chooses a library \(\mathcal{I}\) of size at most \(m\) to maximise covered hard-error mass: \[\max_{\mathcal{I} : |\mathcal{I}| \leq m}\; \sum_{i \in C_D} p_i \, \max_{j : I_j \in \mathcal{I}} r_{ij}.\] Proposition 2 below uses the analytically tractable binary special case \(r_{ij} \in \{0, 1\}\) in which the \(m\)-th-ranked intervention covers the \(m\)-th-ranked mode (one mode per intervention, ordered by marginal covered mass). In real deployments one L4 capability typically covers several L2 categories simultaneously, so the ranked-category budget of Proposition 2 is a conservative proxy for the true set-cover optimum: the empirically achieved library size for a given residual target may be a constant factor smaller than the formula predicts. Overlap, additivity, and prompt-channel interference are treated empirically in the capability-harvest of Appendix 9 (“Additivity and its limits”), not as a formal set-cover optimisation.

3.2.0.4 Operational patch.

A deployment patch \(D\) is more than a topic label. It is an operational tuple fixing, over a chosen time window, the components that determine which failure modes are reachable at all: \[D = (\mathcal{X},\, \mathcal{S},\, \mathcal{U},\, \mathcal{R},\, \mathcal{E},\, \mathcal{P},\, H,\, \tau),\] where \(\mathcal{X}\) is the task input distribution, \(\mathcal{S}\) the input/output schema family, \(\mathcal{U}\) the user or client class, \(\mathcal{R}\) the retrieval corpus or knowledge source, \(\mathcal{E}\) the evaluator or acceptance criterion, \(\mathcal{P}\) the policy and safety constraints, \(H\) the workflow horizon, and \(\tau\) the time window over which these components are fixed. A patch shift occurs when one or more of these components changes enough to alter the reachable failure catalogue \(C_D\). Legal review on contracts from a fixed jurisdiction is a different patch from legal review across mixed jurisdictions; medical RAG over a curated knowledge source is a different patch from medical RAG over arbitrary web content. The patch-local claims in this paper apply within a single, operationally fixed tuple of this form, not to “the domain” in any looser sense.

3.2.0.5 Domain patches cap the engineering problem.

A fixed deployment patch restricts the task family, schemas, tools, workflows, and evaluator expectations. Prior work on localized representations [35], [36], cross-domain performance variation [37], and long-tail knowledge [38], [39] supports the weaker claim that model behaviour is strongly domain-dependent; it does not directly measure \(|C_D|\). We therefore treat the patch’s reachable catalogue as finite or effectively capped as a modelling assumption motivated by domain heterogeneity, not as a theorem derived from boundedness. The full evidence breakdown is in Appendix 11.

3.2.0.6 Tightness reading.

If the empirical discovery curve is approximately tight against the bound at the observed corpus scales, the inequality becomes an approximate equality, \(T(q) \approx\exp((q - A_D)/\sigma_D)\). At the conservative calibration \(\sigma_D \approx1.85\), five additional discovered modes cost roughly \(15\times\) more failures, and ten cost roughly \(220\times\). The lower bound ?? holds unconditionally; the multiplicative-cost reading needs tightness.

3.2.0.7 Engineering meaning.

The claim is about newly distinguishable tail modes, not ordinary failures, which remain common inside already-discovered modes. Combined with the head-heavy coverage model of §3.3, the picture is one of asymmetric returns: the head is cheap to find and absorbs most error mass; the tail is expensive to discover and removes less residual error per mode. Full derivation, the Heaps polynomial variant, the saturation regime, and a mode-mediated capability-gain sub-corollary are in Appendix 8.6.

3.3 Coverage by a targeted intervention library↩︎

With the catalogue defined, the next question is how much residual error a library of \(m\) targeted interventions actually closes off. Let \(p_1 \geq p_2 \geq \dots \geq p_{|C|}\) denote the ranked empirical hard-error masses of the local failure modes, with \(\sum_i p_i = 1\). The exact cumulative coverage of the top-\(m\) library is the empirical step function \[\label{eq:Femp} F_{\text{emp}}(m) \;=\; \sum_{i=1}^{m} p_i, \qquad m \in \{0, 1, \ldots, |C|\},\tag{2}\] with \(F_{\text{emp}}(0) = 0\), \(F_{\text{emp}}(1) = p_1\), and \(F_{\text{emp}}(|C|) = 1\) by construction; \(F_{\text{emp}}\) handles every edge case directly and requires no patching. For closed-form analysis we use the continuum log-head approximation \[\label{eq:coverage} F_{\log}(m; |C|) \;=\; \min\!\left(1,\;\frac{\ln m}{\ln |C|}\right), \qquad m \geq 2,\;|C| \geq 2,\tag{3}\] as a planning approximation to \(F_{\text{emp}}\), not as the true distribution. For our ErrorAtlas anchor (\(|C|=17, m=5\)), \(F_{\log}(5;17) \approx56.8\%\); doubling the library to \(m=10\) pushes it to \(F_{\log}(10;17) \approx81.3\%\). A Zipf-1 reference for the same anchor gives \(H_5/H_{17} \approx66.4\%\), concentrating more head mass than \(F_{\log}\), so the log form is conservative relative to Zipf. Proposition 2 below is therefore a closed-form planning approximation to empirical ranked coverage, not a distribution-free theorem; the qualitative polylog conclusion is what survives across cumulative-coverage families (Appendix 8.5). Empirical anchoring of \(F_{\text{emp}}\) to a measured cumulative-coverage curve in any single patch remains an explicit falsifiability test.

One caveat carries through: \(F_{\log}\) tracks ranked L2 categories covered, not L4 capability axes deployed. Under the set-cover model of §3.2, where one capability often sweeps several L2 categories, residual error at a given \(m\) is generally lower than 3 predicts; the bound is valid but loose, and empirically achieved library sizes may be a constant factor smaller than the formula predicts. For \(|C| = 1\), residual error is fully covered or fully uncovered by definition and neither \(F_{\text{emp}}\) nor \(F_{\log}\) is informative. After deploying the top-\(m\) library, the residual per-hard-token error rate is \[\label{eq:eres} e_{\text{res}}(m) \;=\; \bigl(1 - F_{\log}(m; |C|)\bigr)\, e_{\text{hard}}\tag{4}\] under the log-head approximation, or the corresponding \(F_{\text{emp}}\) expression if the per-mode masses are known directly.

3.4 From universal impossibility to patch-local reliability↩︎

The finite-catalogue claim is patch-local. It does not say that all possible LLM failures can be covered by a universal list of fixes. Across all possible tasks, tools, schemas, knowledge sources, workflows, and evaluator expectations, new intervention-distinguishable failures can keep appearing. In that setting, a finite intervention dictionary is not a well-posed reliability target.

The positive result begins only after a deployment patch \(D\) has been fixed. Inside a patch, the task family, schemas, tools, evaluator expectations, and admissible workflows recur. The engineering question then changes from “Can one library cover all possible LLM failures?” to “How large must the local library be to cover enough of the reachable failure catalogue \(C_D\)?”

3.4.0.1 Metric.

This is a worst-case, mode-wise guarantee, not a claim about expected residual error under a fixed probability distribution. Under a distributional metric, an infinite uncovered tail may still have arbitrarily small total mass; the proposition rules out only the worst-case-mode-coverage reading of universal reliability, which is the reading the engineering literature implicitly assumes when it asks for “a fixed list of interventions that covers LLM use.”

3.4.0.2 Proof sketch.

Each new failure is intervention-distinguishable from the failures before it. The sequence therefore generates infinitely many distinct intervention modes, and any finite dictionary must miss some later mode. The full proof, including the precise definition of mode-level coverage, is in Appendix 8.1.

3.4.0.3 Engineering meaning.

Universal reliability is not a finite-library problem. A fixed list of interventions cannot cover open-ended LLM use in general. The rest of the paper is therefore about patch-local reliability, not universal reliability. Proposition 1 is the inoculation against a misread: we are not claiming a fixed list of \(\approx50\) named patterns covers LLM use in general.

The positive result composes \(\beta\)-stratification (Definition 1), the active-mode bound (Assumption 1), and the log-coverage form (Eq. 3 ) into a local budget statement. The result is conditional engineering math, not an unconditional theorem about LLMs: it depends on the log-coverage approximation, the active-mode exposure assumption, and the patch hypothesis that \(C_D\) is finite or effectively capped. Appendix 8.2 states these conditions explicitly before the derivation.

3.4.0.4 Proof sketch.

The library removes cumulative hard-error mass \(F(m; |C|)\). The residual hard-token error rate is therefore \((1 - F)\,e_{\text{hard}}\). Requiring this residual to be at most \(\varepsilon\) rearranges, after substituting the log-coverage form, to the stated bound. The full step-by-step derivation, including the pre-cap and cap regimes, is in Appendix 8.2.

3.4.0.5 Engineering meaning.

Once the patch is fixed, the problem changes. The relevant engineering question is no longer whether arbitrary future failures exist, but how quickly the reachable local catalogue is discovered and how much hard-error mass is removed by the top interventions. The intervention budget for per-hard-token reliability is small and slowly growing, domain-constant in the cap regime, with the exact size determined by the local discovery and rank-coverage curves rather than by a universal prior.

3.4.0.6 Sequence-level caveat.

Proposition 2 bounds residual error per hard decision. A one-shot sequence-level target is strictly stricter because many hard decisions occur in one output: as \(k\) grows the allowable residual per hard decision shrinks, and the required library approaches full-catalogue coverage. In some regimes the non-hard-token error mass alone already exceeds the sequence-level budget, so hard-token interventions cannot meet the SLA by themselves. The full three-regime analysis (and the per-hard-token tolerance \(\tau_{\text{seq}}\) that converts a sequence target into the Proposition 2 bound) is in Appendix 8.3.

A common engineering rule of thumb, “cover the local head with on the order of tens of interventions,” is best understood as a per-hard-decision planning prior, not a derived constant from the formal model. The per-hard-decision residual-error reduction implied by \(F_{\log}\) at typical \(|C|\) and head-heavy mass is large; the sequence-level analogue requires correspondingly more interventions and tighter coverage of the tail.

We call these statements propositions rather than theorems because their force is conditional: they formalise the consequences of the paper’s modelling assumptions rather than deriving a universal law of LLM reliability. The doubly-logarithmic rate is the optimistic special case under logarithmic active-mode exposure; Appendix 8.5 gives the Heaps and saturation variants. Appendix 8.4 expands on the conditional reading.

4 Empirical Evidence↩︎

We collect evidence for three load-bearing claims: errors cluster into a small recurring set (A); each cluster is addressable by one targeted intervention (B); reliability decays sublinearly with output length (C).

4.0.0.1 Claim A: Failure-mode clustering.

The strongest single result is ErrorAtlas [5]: 83 models \(\times\) 35 datasets, \(\gtrsim 10^4\) failures, 17 named categories with a long-tailed, head-concentrated prevalence ordering. Domain-specific Paretos reproduce the shape inside each domain: math (MWPES top-4 categories dominate per model with strong cross-model overlap [7]), code (AssertionError+NameError = 86.35% on HumanEval, replicated across 23 models [6]), multi-hop QA (a single dominant “missing-evidence” mode [8]), agentic tool use (under 20 recurring modes across two studies [9], [12]), and RAG (a handful of distinguishable hallucination types [10]). The catalogue is also stable across models: ErrorAtlas reports a fixed category hierarchy across 83 models; RFMDataset [11] finds “strikingly similar failure mode distributions” across ten advanced reasoning models; EDIT [40] measures a \(\approx4.7\%\) key-step fraction that holds model to model. [41] provide the closest theoretical anchor, proving that the observed aggregate power-law scaling of LLM evaluations requires the per-problem success-rate distribution to be heavy-tailed near \(p=0\), structurally related to Postulate 1.

4.0.0.2 Claim B: Cluster-selective capability interventions.

A dedicated harvest yields 28 capability-elimination citations across six independent axes (arithmetic, code execution, format/structure, perception/grounding, knowledge/RAG, verification), each axis independently confirmed by between three and nine citations. The full table and the three structural patterns (Pattern A by-construction, B strong-empirical-with-class-shift, C moderate-with-shift) appear in Appendix 9.

Arithmetic is the cleanest case: PAL [13] lifts GSM-Hard from 20.1% to 61.5%, and the residuals are problem-comprehension errors rather than arithmetic ones. Format violations admit something stronger still: constrained decoding zeroes invalid-token probability by construction [16]–[18]. Code-logic errors yield to execution feedback (Reflexion + AgentCoder push HumanEval pass@1 from 80% to 96.3%, with residuals in spec-misinterpretation [19], [20]); reasoning steps to process supervision (Math-Shepherd: Mistral-7B MATH 28.6% \(\rightarrow\) 43.5% [21]); hallucinations to RAG, where Acurai [10] reports 100% elimination on RAGTruth, a benchmark-conditional empirical result rather than a general by-construction guarantee. Two further clusters complete the picture: POROver lifts the not-overrefused rate from 57.6% to 82.1% [23]; SAGE-Agent, targeting tool-call clarification, raises When2Call accuracy from 36.5% to 65.2% [24].

The class-shift signature, in which post-intervention residuals belong to structurally different classes, is the empirical content of the cluster-selectivity property that underwrites Proposition 2’s composition step. Capabilities are coarser than error classes: a single Python interpreter removes execution-error components from five named clusters; constrained decoding eliminates format and the structural component of “missing required element” jointly. A negative control sharpens the selectivity claim. DebugBench [42] reports that execution feedback works for syntax and reference errors and is explicitly “unhelpful for logic errors,” exactly where the framework predicts capability provisioning fails.

4.0.0.3 Claim C: Sublinear length scaling.

Direct evidence for sublinear (rather than exponential) decay shows up across very different measurement designs. Loong [27] reports GPT-4o on Chain-of-Reasoning declining \(81.6\% \rightarrow32.9\%\) across the \(10\)K \(\rightarrow\) \(>\)​200K context bins; log-linear over \(\log L\) and not well explained by smooth independent per-token exponential decay. GSM-\(\infty\) [28] attacks the independent-error model from a different angle: exponentially more inference compute yields only linear AUC gains, and DeepSeek-R1 maintains \(10\%+\) accuracy at 130 reasoning operations where \((1-\varepsilon)^{130}\) for any reasonable \(\varepsilon\) predicts near-zero. [43] find a \(\approx40\%\) compositionality gap that is approximately constant across the GPT-3 family, direct evidence against per-step independent compounding (which would predict the gap to grow with chain length).

Structural correlates of the same phenomenon appear elsewhere. [30]’s Anchor LLMs achieve \(\approx99\%\) K/V reduction with only minor accuracy compromise: the effective number of distinct attention-key vectors a model needs is far smaller than \(n\). Think-Prune-Train [44] raises Pass@1 while Pass@20 plateaus, the manifold-transition signature. RULER [29] finds threshold behaviour inconsistent with smooth \((1-\varepsilon)^n\). METR [32] fits logistic-in-log-length, which is mathematically sublinear in length itself.

Self-consistency [45] is consistent with but does not prove clustered structure: Condorcet aggregation of iid Bernoulli chains can yield similar gain magnitudes, so we treat it as suggestive rather than diagnostic. Prominent steep-decay counter-evidence [2], [31], [33], [46] decays over variables distinct from raw token length; per-paper re-audits showing the relocation pattern are in Appendix 10.

5 Practical Implications↩︎

5.0.0.1 Reliability engineering is local patch coverage.

Within a fixed domain patch, Proposition 2 makes reliability a small-catalogue engineering problem rather than an asymptotic scaling problem. A team should initially budget for a library on the order of tens of interventions, then refine from the local mode-discovery curve \(C_{\text{seen},D}(T)\) and the empirical rank-frequency distribution. As a rule of thumb consistent with the head-mass figures in ErrorAtlas, HumanEval, and MWPES, \(\approx50\) named interventions cover the bulk of the per-hard-token failure distribution in many measured domains. This is a planning prior calibrated to current taxonomies, not a universal constant; the local mode-discovery curve sets the actual library size for any given deployment. The same base model in cardiology RAG, legal contract drafting, and code review yields three different libraries because \(C_D\), \(A_D\), \(\sigma_D\) all change with the deployment domain.

5.0.0.2 Per-hard-token vs.sequence-level targets matter for SLA design.

Per-hard-token residual error, the right metric for continuous-correction systems, has a polylog budget. Sequence-level failure probability, the right metric for one-shot systems, is strictly stricter and approaches full-catalogue coverage as \(k\) grows. Production deployments should design SLAs to match the actual cost structure, not read the per-token result as the production SLA.

5.0.0.3 Library design as engineering, not asymptotic theory.

An intervention library can be assembled module-by-module, in any order, as long as modules target distinct failure classes. Layer-separated interventions (constrained decoding + retrieval + process supervision + tool call) compose approximately additively in our data; the [26] caveat applies only to interventions sharing a prompt channel. Library construction is therefore highly parallelisable.

5.0.0.4 Capability libraries are coarser than error-class libraries.

Five named math classes (arithmetic, units, counting, list manipulation, date arithmetic) collapse under one Python interpreter; three code classes (SyntaxError, NameError, most TypeError) collapse under one execution-feedback loop; format violations and the structural component of “missing required element” (together more than \(20\%\) of ErrorAtlas) collapse under one constrained-decoder deployment. A team budgeting for \(\approx50\) interventions per domain can expect to need fewer than 50 capability-axis interventions while covering \(\approx50\) named clusters. The six axes of Appendix 9 are closer to the right unit of engineering accounting than the twelve clusters.

6 Discussion↩︎

6.0.0.1 By-construction elimination as a boundary case.

A subset of the capability-elimination harvest deserves separate accounting. Seven of the 28 citations (Appendix 9; constrained decoders [16], [18], [47]–[49], proof kernels [50], static syntax checks) achieve residual error rate equal to zero by mathematical construction, not statistically. Constrained decoders set \(P(\text{invalid token}) = 0\) at every generation step; the class of grammar-violating outputs is mathematically empty. In this regime the polylog bound of Proposition 2 is loose: covered clusters contribute exactly zero to residual error, not a small positive quantity. The result strengthens, rather than refutes, the framework. And Pattern A applies precisely to structural/verifiable classes (format, syntax, schema, invalid proofs), which sit at the head of the heavy-tailed cluster frequency distribution: the strongest mechanism lands exactly where the catalogue is densest.

6.0.0.2 Counter-evidence relocates, not dissolves.

Five prominent steep-decay papers [2], [31]–[33], [46] are routinely cited as exponential-compounding evidence. Every one decays over a variable distinct from raw token length: compositional graph size [2], number of supporting facts [31], log-time horizon [32], capacity threshold [33], evidence scope [46]. The apparent rapid decay is, in each case, in a quantity our framework already concentrates the action in (\(k_{\text{hard}}\) and \(|C|\)), not in raw sequence length: a relocation, not a dissolution. These regimes remain hard; the framework’s value is directing intervention toward capability provisioning along the actual decay axis rather than toward context-window expansion that does not help. Appendix 10 walks through each paper.

6.0.0.3 Limitations.

Postulate 1 is empirical, not derived; a domain with genuinely Heaps-power-law mode discovery would invalidate the doubly-logarithmic special case while preserving the qualitative polylog conclusion. The coverage form of §3.3 is an empirical best-fit; readers should not interpret the numerical match against anchors as theoretical confirmation. Inter-cluster additivity is approximate; [26]’s caveat on prompt-channel interference tightens the bound in shared-channel settings. Domain narrowness: Postulate 1’s empirical anchor rests on three taxonomies (ErrorAtlas, HumanEval, MWPES), all published 2025–2026 and covering general/code/math; the framework is untested on agentic workflows, long-running scientific reasoning, and multi-turn tool use over millions of tokens, precisely the regimes where \(|C|\) may grow faster than logarithmically. The \(\sigma \approx1.85\) estimate is a single-point calibration, not a discovery-curve fit; a subsample-vs-distinct-modes plot has not been published for any LLM error taxonomy at this writing. \(\beta\) is latent and we report no empirical range. Taxonomy granularity (the L2–L3 gap) is unmeasured. Patch-shift between deployments changes \(A_D, \sigma_D, \beta_D, |C_D|\) simultaneously; libraries calibrated against one patch under-cover the next. Finally, the framework relocates the difficulty of long-context reliability rather than resolving it: where \(k_{\text{hard}}\) grows with task length (adversarial compositional structure, multi-hop chains, long agent horizons), reliability remains hard. The contribution is to name the on-axis intervention, not to make those regimes easy. A falsifiability test the framework passes: DebugBench [42] explicitly catalogues which classes execution-feedback can and cannot repair, and the empirical split (works for syntax/reference; “unhelpful for logic errors”) is exactly what the framework’s selectivity claim predicts.

7 Conclusion↩︎

LLM reliability is often framed as an asymptotic scaling problem. Universal reliability is not a finite-library problem (Proposition 1): an unbounded domain that keeps producing intervention-distinguishable failures cannot be covered by any finite dictionary that guarantees mode-wise residual error below tolerance. The contribution of this paper is to show that, once an operationally bounded deployment patch is fixed, reliability becomes a local engineering problem. Prior work in this line [3] argued that the standard asymptotic framing rests on a false uniformity assumption: errors concentrate at \(\approx5\)–\(10\%\) of tokens. This paper takes the next step: within that sparse set, errors are not only concentrated but also repetitive, clustering into a finite catalogue whose size grows logarithmically with observed failures under the empirical postulate of §3.2, or as a small power under the conservative Heaps alternative (Appendix 8.5). The conditional consequence (Proposition 2) is that a sufficient per-hard-decision intervention budget scales polylogarithmically in sequence length within a fixed domain patch under the log active-mode-exposure assumption, and becomes a domain-constant once the patch ceiling \(|C_D|\) is reached. Sequence-level reliability targets are strictly tighter. Available evidence is consistent with libraries on the order of tens of interventions covering the head of the per-hard-decision failure distribution in many fixed domains. The exact number is patch-indexed and should be estimated from local discovery and rank-coverage curves, not assumed from cross-domain priors. This reframes the question from “can we bound the growth of \(n\)-token error?” to “have we catalogued enough failure modes inside the deployment patch?” The latter is finite and addressable once an operationally bounded patch has been fixed.

The same logarithmic discovery postulate carries an inverse interpretation (Corollary 1, Appendix 8.6): the logarithmic upper bound cannot accommodate linearly more distinct tail modes without exponentially more observed hard-failure events. Combined with head-heavy failure mass, this explains why generic frontier post-training can face diminishing reliability returns in open-ended deployment settings. The high-mass head is discovered early, while the tail becomes increasingly expensive to find and contributes less marginal residual-error reduction. The framework therefore does not say frontier scaling is useless. It says frontier-only reliability is economically misaligned with fixed deployment patches: once \(D\) is known, local adaptation, tools, validators, retrieval, constrained decoding, and process supervision can target recurring failure mass directly, where generic post-training would have to rediscover the same local repairs indirectly across an open-ended task universe.

The open empirical question this paper invites is direct measurement of the mode-rate constant \(\sigma\) on new domains (agentic, scientific, code-in-production), and a corresponding sequence-level validation that measures \(S_{\text{base}}\), \(\beta\), and active-mode exposure directly in deployed systems. The conceptual follow-on goes in a different direction. This paper names the engineering object: a patch-local failure catalogue and the intervention budget that covers its head. It does not say how production systems actually accumulate and govern that library over time. That question, how the deployment-time scaffold (instructions, tools, retrieval, memory, orchestration, governance) learns to cover the local failure topology, is the subject of follow-on work in this line. Reliability engineering, in that frame, is governance of the scaffold rather than scaling of the weights.

8 Formal Proofs, Derivations, and Sensitivity Analysis↩︎

8.1 Proof of Proposition 1: No Universal Finite Intervention Dictionary↩︎

Fix a residual-error tolerance \(\varepsilon\). Say that an intervention covers a failure mode if it reduces the residual error of that mode below \(\varepsilon\). Coverage is mode-level: an intervention that covers a mode covers every failure event in that mode. The two conditions “\(D\) is intervention-unbounded” and “\(|C_D^{\varepsilon}| = \infty\)” are equivalent under mode-level coverage; an unbounded witnessing sequence is constructed by taking one representative per mode, and an infinite catalogue forces the existence of such a sequence.

Let \(D\) be a domain. Call \(D\) intervention-unbounded if it contains an infinite sequence of failures \(f_1, f_2, f_3, \ldots\) such that each new \(f_j\) is not covered by any finite intervention dictionary that covers all earlier failures \(\{f_1, \ldots, f_{j-1}\}\).

8.1.0.1 Step 1: assume the opposite.

Suppose, for contradiction, that \(C_D^{\varepsilon}\) is finite. Then there are only finitely many intervention-distinguishable failure modes in \(D\). Write them as \(C_D^{\varepsilon} = \{c_1, \ldots, c_M\}\) for some finite \(M\).

8.1.0.2 Step 2: what finiteness means.

If there are only \(M\) intervention-distinguishable modes, then after all \(M\) modes have appeared in the sequence, every later failure must belong to one of the already-seen modes.

8.1.0.3 Step 3: same mode means same intervention class.

Modes are defined at intervention resolution \(\varepsilon\). If a later failure belongs to the same mode as an earlier failure, then the intervention dictionary that covers the earlier representative of that mode also covers the later failure below residual tolerance \(\varepsilon\).

8.1.0.4 Step 4: contradiction.

Intervention-unboundedness says exactly the opposite: each new \(f_j\) is not covered by any finite dictionary that covers \(\{f_1, \ldots, f_{j-1}\}\). Therefore \(f_j\) cannot belong to any earlier intervention mode, so each \(f_j\) introduces a new intervention-distinguishable mode. The sequence \(f_1, f_2, f_3, \ldots\) induces infinitely many such modes, contradicting the assumption that \(C_D^{\varepsilon}\) is finite. Hence \(|C_D^{\varepsilon}| = \infty\).

8.1.0.5 Step 5: no finite dictionary covers every mode of the domain.

Suppose, again for contradiction, that some finite intervention dictionary \(\mathcal{I}\) covers every mode of \(D\). Then \(\mathcal{I}\) covers every finite prefix \(\{f_1, \ldots, f_{j-1}\}\) for every \(j\). By intervention-unboundedness, any dictionary covering that prefix fails to cover \(f_j\). Therefore \(\mathcal{I}\) does not cover \(f_j\), contradicting the assumption that \(\mathcal{I}\) covers every mode of \(D\). 0◻

8.2 Derivation of Proposition 2: Patch-Local Sufficient Intervention Budget↩︎

8.2.0.1 Conditions used.

Proposition 2 gives a sufficient, model-implied budget under the following four assumptions:

1. Coverage model. The cumulative hard-error mass covered by the top \(m\) modes is approximated by the log-head form \[F_{\log}(m; |C|) \;=\; \min\!\left(1,\;\frac{\ln m}{\ln |C|}\right)\] on the declared domain \(m \geq 2\), \(|C| \geq 2\). We write \(F\) for \(F_{\log}\) throughout this appendix unless otherwise noted.

2. Non-trivial, attainable target. The residual target satisfies \(\varepsilon \in (0, e_{\text{hard}})\). If \(\varepsilon \geq e_{\text{hard}}\) no intervention is needed; if \(\varepsilon \leq 0\) the target is unattainable unless all residual hard-token error is eliminated.

3. Patch-local catalogue. The result applies only after a deployment patch \(D\) has been fixed and its reachable catalogue is modelled as finite or effectively capped.

4. Sequence-length claim. The doubly-logarithmic rate further requires Assumption 1 and \(k(n) = \Theta(\log n)\). Without these, Proposition 2 still gives a catalogue-size budget but not the same sequence-length scaling.

8.2.0.2 Step 1: define the target.

Let \(e_{\text{hard}}\) be the baseline hard-token error rate and \(\varepsilon \in (0, e_{\text{hard}})\) the target after intervention.

8.2.0.3 Step 2: define the effective catalogue.

For per-sequence scaling, set \(C_{\text{eff}} = C_{\text{active},D}(n)\): how many failure modes can a single sequence of length \(n\) activate? For full deployment-library budgeting, set \(C_{\text{eff}} = C_D\): how large must the library be to cover the recurring failure modes reachable inside \(D\)?

8.2.0.4 Step 3: cumulative coverage.

Let the ranked local failure modes have hard-error masses \(p_1 \geq p_2 \geq \cdots \geq p_{|C_{\text{eff}}|}\) with \(\sum_i p_i = 1\). A library covering the top \(m\) modes removes cumulative hard-error mass \(F(m; |C_{\text{eff}}|) = \sum_{i=1}^{m} p_i\), approximated by the log-coverage form of §3.3.

8.2.0.5 Step 4: residual after intervention.

The uncovered mass fraction is \(1 - F(m; |C_{\text{eff}}|)\), so the residual per-hard-token error rate is \(e_{\text{res}}(m) = (1 - F(m; |C_{\text{eff}}|))\, e_{\text{hard}}\).

8.2.0.6 Step 5: impose the target.

Requiring \(e_{\text{res}}(m) \leq \varepsilon\) and dividing by \(e_{\text{hard}} > 0\) gives \[1 - F(m; |C_{\text{eff}}|) \;\leq\; \frac{\varepsilon}{e_{\text{hard}}}, \qquad\text{i.e.,}\qquad F(m; |C_{\text{eff}}|) \;\geq\; 1 - \frac{\varepsilon}{e_{\text{hard}}}.\]

8.2.0.7 Step 6: substitute the log-coverage form.

Since \(\varepsilon < e_{\text{hard}}\), the required coverage \(1 - \varepsilon/e_{\text{hard}} \in (0, 1)\), so the cap in \(F\) is inactive before saturation. Using \(F = \ln m / \ln |C_{\text{eff}}|\): \[\frac{\ln m}{\ln |C_{\text{eff}}|} \;\geq\; 1 - \frac{\varepsilon}{e_{\text{hard}}}.\]

8.2.0.8 Step 7: solve for \(m\).

Multiplying by \(\ln |C_{\text{eff}}| > 0\) and exponentiating, \[m \;\geq\; \exp\!\left[\left(1 - \frac{\varepsilon}{e_{\text{hard}}}\right) \ln |C_{\text{eff}}|\right] \;=\; |C_{\text{eff}}|^{1 - \varepsilon / e_{\text{hard}}}.\] Taking the ceiling (since \(m\) is integer-valued) yields \(m \geq \lceil |C_{\text{eff}}|^{1 - \varepsilon / e_{\text{hard}}} \rceil\), the bound stated in Eq. ?? .

8.2.0.9 Step 8: sequence-length rate.

For per-sequence scaling, set \(C_{\text{eff}} = C_{\text{active},D}(n)\). Assumption 1 bounds \(|C_{\text{active},D}(n)| \leq \min(A'_D + \sigma'_D \ln h(n),\, |C_D|)\). In the pre-cap regime, i.e., while \(A'_D + \sigma'_D \ln h(n) < |C_D|\), the ceiling has not been reached and \(|C_{\text{active},D}(n)| = O(\ln h(n))\). With \(h(n) = \beta k(n)\) and \(k(n) = \Theta(\log n)\), we get \(h(n) = \Theta(\log n)\) and \(\ln h(n) = \Theta(\log\log n)\), hence \[|C_{\text{active},D}(n)| \;=\; O(\log\log n),\] and substituting into the budget, \[m \;=\; O\!\bigl((\log\log n)^{1 - \varepsilon / e_{\text{hard}}}\bigr).\] This is the doubly-logarithmic pre-cap special case.

8.2.0.10 Step 9: cap regime.

Once active-mode discovery saturates the patch catalogue, \(|C_{\text{active},D}(n)| = |C_D|\), so \(m \geq \lceil |C_D|^{1 - \varepsilon / e_{\text{hard}}} \rceil\), which no longer depends on \(n\). The intervention budget is then domain-constant. 0◻

8.3 Sequence-Level Reliability Derivation↩︎

Proposition 2 gives a per-hard-token residual target. Production systems often care about a stricter target: the probability that the entire sequence is correct.

Starting from the composed reliability of Eq. 1 , and writing the post-intervention hard-token rate as \(e_{\text{res}}\), \[P(\text{correct}) \;=\; (1 - e_{\text{res}})^{\beta k}\,(1 - e_{\text{easy}})^{(1 - \beta) k}\,(1 - e_{\text{non}})^{n - k}.\] Define the non-hard-token survival factor \[\label{eq:Sbase} S_{\text{base}} \;=\; (1 - e_{\text{easy}})^{(1 - \beta) k}\,(1 - e_{\text{non}})^{n - k},\tag{5}\] so that \(P(\text{correct}) = (1 - e_{\text{res}})^{\beta k}\, S_{\text{base}}\). The sequence-level target \(P(\text{correct}) \geq 1 - \varepsilon_{\text{seq}}\) becomes \[(1 - e_{\text{res}})^{\beta k}\, S_{\text{base}} \;\geq\; 1 - \varepsilon_{\text{seq}}, \qquad\text{i.e.,}\qquad (1 - e_{\text{res}})^{\beta k} \;\geq\; \frac{1 - \varepsilon_{\text{seq}}}{S_{\text{base}}}.\] Three regimes follow.

8.3.0.1 Regime (i): non-hard-token errors already violate the target.

If \(S_{\text{base}} < 1 - \varepsilon_{\text{seq}}\), then even setting \(e_{\text{res}} = 0\) cannot meet the target, because the maximum possible survival after eliminating all hard-token failures is only \(S_{\text{base}}\). Hard-token interventions alone cannot meet the sequence-level SLA. This is the route by which the exponential-in-\(n\) concern re-enters and should be diagnosed before any catalogue-budgeting exercise.

8.3.0.2 Regime (ii): hard-token residual error determines feasibility.

If \(S_{\text{base}} \geq 1 - \varepsilon_{\text{seq}}\), the target may be feasible. Taking the \((1/(\beta k))\)-th power of the rearranged inequality and isolating \(e_{\text{res}}\), \[e_{\text{res}} \;\leq\; 1 - \left(\frac{1 - \varepsilon_{\text{seq}}}{S_{\text{base}}}\right)^{1 / (\beta k)} \;=:\; \tau_{\text{seq}}.\] Applying Proposition 2 with \(\varepsilon\) replaced by \(\tau_{\text{seq}}\), \[m \;\geq\; \left\lceil |C_{\text{eff}}|^{1 - \tau_{\text{seq}} / e_{\text{hard}}} \right\rceil.\] As the sequence-level target becomes stricter (\(\varepsilon_{\text{seq}}\) shrinks), \(\tau_{\text{seq}} \to 0\) and the exponent \(1 - \tau_{\text{seq}}/e_{\text{hard}} \to 1\), so \(m \to |C_{\text{eff}}|\). Strict one-shot sequence-level reliability pushes the system toward full-catalogue coverage.

8.3.0.3 Regime (iii): baseline hard-token error is already acceptable.

If \(\tau_{\text{seq}} \geq e_{\text{hard}}\), the baseline hard-token rate already satisfies the sequence target (no intervention is required because \(e_{\text{res}}(0) = e_{\text{hard}}\) from Eq. 4 ).

8.3.0.4 Conclusion.

Per-hard-token reliability is easier than sequence-level reliability. A library that gives a large reduction in residual hard-token error may still be insufficient for one-shot sequence-level guarantees when many hard decisions occur in a single output. This is why the main paper treats the “tens of interventions” rule of thumb as a per-hard-decision planning prior, not a one-shot sequence-level SLA.

8.4 Why these are propositions rather than unconditional theorems↩︎

Proposition 1 is a definitional impossibility result: once intervention-unboundedness is assumed, an infinite intervention-resolution catalogue follows. Its role is not to prove that every unbounded domain necessarily has infinite failure modes, but to show that open-ended domains cannot be assumed to admit finite dictionaries.

Proposition 2 is conditional engineering math. It does not prove that LLM failures universally obey logarithmic mode discovery. It proves that if a bounded patch has a finite or effectively capped reachable catalogue, and if cumulative intervention coverage follows the stated head-heavy form, then a sufficient per-hard-decision intervention budget grows slowly and becomes constant after catalogue saturation. It does not prove that the true minimal intervention library has the same scaling.

The empirical burden therefore lies not in the algebra but in measuring, for each deployment patch, the local discovery curve \(C_{\text{seen},D}(T)\), the per-sequence activation \(C_{\text{active},D}(n)\), the cumulative coverage \(F(m; |C_D|)\), the hard-token fraction \(\beta_D\), and the baseline hard-token rate \(e_{\text{hard}}\). This is why the paper frames the results as a reliability-engineering scaffold rather than as universal theorems about LLM behaviour.

8.5 Heaps Power-Law Variant and Cluster-Count Sensitivity↩︎

A reader who prefers to derive cluster-count growth from standard Heaps’ law rather than from Postulate 1 obtains a qualitatively similar result. Let \(|C|(k_{\text{hard}}) \approxK \cdot k_{\text{hard}}^{b}\) with \(b \in (0, 1)\). Canonical fits give \(b \approx0.5\) for natural-language vocabularies; failure-mode taxonomies plateau much more sharply (ErrorAtlas stabilises at \(|C| = 17\) across \(10^4{+}\) failures), implying a small effective \(b \approx0.24\)–\(0.32\) under a crude no-intercept endpoint read (not a fitted discovery exponent) in our setting. Composing with \(k = \Theta(\log n)\), we get \(|C| = O((\log n)^{b})\), and Proposition 2 becomes \[m = O\!\left((\log n)^{b \cdot (1 - \varepsilon / e_{\text{hard}})}\right).\] This is still polylogarithmic in \(n\) for any \(b \in (0, 1)\) and any \(\varepsilon < e_{\text{hard}}\). The paper’s qualitative claim survives either choice of cluster-count law.

8.5.0.1 Symbolic-form sensitivity across candidate laws.

The polylog conclusion depends on which cluster-count law one accepts; available evidence is consistent with multiple candidates because no subsample-discovery curve has been published for any LLM failure-mode taxonomy at this writing. We therefore report symbolic rates rather than fitted constants. With \(h(n) = \beta k(n)\):

• Logarithmic: \(|C_{\text{active},D}(n)| = O(\log h(n))\). If \(k(n) = \Theta(\log n)\), then \(m = O\bigl((\log\log n)^{1 - \varepsilon / e_{\text{hard}}}\bigr)\).

• Heaps: \(|C_{\text{active},D}(n)| = O\bigl(h(n)^b\bigr)\) with \(b \in (0, 1)\). If \(k(n) = \Theta(\log n)\), then \(m = O\bigl((\log n)^{b\,(1 - \varepsilon / e_{\text{hard}})}\bigr)\).

• Saturating: \(|C_{\text{active},D}(n)| \leq |C_D|\). Then \(m = O\bigl(|C_D|^{1 - \varepsilon / e_{\text{hard}}}\bigr)\), constant in \(n\) once the patch ceiling is reached.

The qualitative conclusion is robust under the \(k(n) = \Theta(\log n)\) regime: \(m\) grows more slowly than any positive power of \(n\) under every candidate, and the directional claim (“a small library covers the head of the failure distribution in the per-hard-token regime”) survives. Only the exponent shifts: doubly-logarithmic under logarithmic discovery, \((\log n)^b\) with small \(b\) under Heaps, constant in the cap regime. The doubly-logarithmic rate is the optimistic special case. If \(k(n)\) grows as a positive power of \(n\), the Heaps variant inherits that power and the polylog-in-\(n\) language fails along that axis; the framework’s intervention prescription still applies, but its asymptotic-rate framing does not.

Numerical constants require a measured discovery curve \(C_{\text{seen},D}(T)\) or \(C_{\text{active},D}(n)\). Existing taxonomies provide endpoint category counts at a single corpus scale (ErrorAtlas at \(|C|=17\) for \(\approx10^4\) failures), not discovery curves. The subsample-discovery measurement remains the explicit empirical test that would either tighten the postulate or fall back to the Heaps variant. Until that measurement exists, the framework’s headline rate should be read as polylogarithmic in the pre-cap regime, domain-constant in the cap regime, with the specific exponent flagged as a falsifiability test rather than a fitted prediction.

8.6 Inverse Discovery Cost↩︎

The body Corollary 1 inverts the upper-bound discovery postulate into a sample-budget lower bound on novel-mode discovery. This appendix gives the algebra, the numerical anchors, the tightness assumption that converts the lower bound into an approximate inverse cost, sensitivity to the Heaps cluster-count alternative of §8.5, the saturation regime, and a separate mode-mediated gain sub-corollary that connects discovery cost to broad capability proxies.

8.6.0.1 Setup.

Let \(q(T) = |C_{\text{seen},D}(T)|\) be the number of distinct failure modes discovered in patch \(D\) after \(T\) observed hard-failure events. Postulate 1 is an upper bound, \[q(T) \;\leq\; A_D + \sigma_D \ln T, \qquad \sigma_D > 0,\] on the discovered catalogue under the logarithmic upper-bound assumption.

8.6.0.2 Inversion (lower bound on \(T\)).

The upper-bound postulate is monotone in \(T\). Taking the inverse direction yields a lower bound on \(T\) required for the cap to accommodate \(q\) discovered modes: if \(q > A_D\) distinct modes have been discovered after \(T\) events, then \[T \;\geq\; \exp\!\left(\frac{q - A_D}{\sigma_D}\right).\] This is the rigorous direction of the corollary. The cap cannot accommodate \(q\) until the sample budget has grown exponentially in \(q\).

8.6.0.3 Tightness assumption.

The lower bound above is unconditional under Postulate 1. The stronger reading is that observed hard failures at scale \(T(q) \approx\exp((q - A_D)/\sigma_D)\) actually deliver \(q\) discovered modes, and that each additional \(\Delta q\) modes raises the sample budget by approximately \(\exp(\Delta q/\sigma_D)\). This stronger reading requires an additional assumption that the empirical discovery curve is approximately tight against the bound at the relevant corpus scales. Without that assumption, the appendix gives a lower bound on \(T\) only. With it, \[\frac{T(q + \Delta q)}{T(q)} \;\approx\; \exp\!\left(\frac{\Delta q}{\sigma_D}\right).\]

8.6.0.4 Numerical anchors under tightness.

At the conservative calibration \(\sigma_D \approx1.85\) (§3.2), \(\exp(5/1.85) \approx14.9\) and \(\exp(10/1.85) \approx222\): five extra modes need roughly \(15\times\) more observed hard failures, ten extra modes need roughly \(220\times\) more. These are tightness-conditional anchors, not unconditional consequences of Postulate 1. The subsample-discovery measurement of §3.2 is precisely the test of whether tightness holds.

8.6.0.5 What this is and is not.

The corollary describes new distinct-mode discovery. Ordinary failures inside already-discovered modes may remain common and cheap to observe; the exponential cost lives on the category-novelty axis. Conflating ordinary failure rate with novel-mode arrival rate would over-claim the result.

8.6.0.6 Heaps alternative.

Under the Heaps cluster-count law of §8.5, \(q(T) = K T^b\) with \(b \in (0,1)\), the corresponding inverse cost is polynomial rather than exponential: \[T(q) \;=\; (q/K)^{1/b}.\] The exponential inverse-cost reading is specific to the logarithmic upper bound; the broader qualitative claim that tail discovery has diminishing returns survives under any concave discovery curve. Sensitivity is therefore: exponential under logarithmic-and-tight, polynomial under Heaps, undefined past the patch ceiling.

8.6.0.7 Saturation regime.

Once \(q(T) \leq |C_D|\) has been saturated, discovery stops; the inversion applies only in the pre-cap regime. Inside saturated patches the corollary’s multiplicative-cost reading is vacuous because no novel modes remain to discover, which is itself a property of the patch and not a failure of the corollary.

8.6.0.8 Mode-mediated capability gain (sub-corollary).

Suppose broad capability or reliability gain \(G\) inside the patch is approximately linear in the number of useful discovered modes, \(G(q) = G_0 + \gamma q\) for some \(\gamma > 0\). Composing with the logarithmic upper bound at tightness, \[G(T) \;\approx\; G_0 + \gamma A_D + \gamma \sigma_D \ln T,\] so \(G\) grows logarithmically in observed hard-failure exposure under tightness. Inverting, \[T(G) \;\approx\; \exp\!\left(\frac{G - G_0 - \gamma A_D}{\gamma \sigma_D}\right).\] A linear gain in mode-mediated broad reliability therefore corresponds to exponential growth in observed hard-failure exposure under the postulate at tightness. We deliberately phrase \(G\) as a mode-mediated capability/reliability proxy, not as “intelligence”: the corollary does not say frontier scaling is useless or that intelligence requires exponential data in any general sense. It says, more narrowly, that for fixed deployment reliability where improvement is mediated by discovering new useful modes, generic open-domain training pays a heavy data tax relative to direct patch-local measurement and intervention.

8.6.0.9 Engineering reading.

The corollary explains, without invoking new mechanisms, two empirical signals: (a) why generic post-training shows diminishing reliability returns once a domain’s head modes are covered, and (b) why patch-local measurement combined with targeted tools, retrieval, validators, constrained decoding, and process supervision often outperforms more frontier-scale data on the deployment SLA. Frontier scaling and patch-local engineering solve different problems: scaling improves the substrate; patch-local engineering removes recurring deployment failure mass. \(\square\)

9 Full Failure-Mode Taxonomy and the Capability-Elimination Harvest↩︎

The intervention literature provides at least one targeted countermeasure for each named cluster of §4. Two organising granularities exist: at the capability level (six axes) the cluster-selectivity property is most clearly visible; at the error-class level (twelve named clusters) the evidence aligns with the taxonomies of §4 but is finer-grained than the underlying capability mechanisms.

9.0.0.1 Six capability axes and three structural patterns.

A dedicated harvest yields 28 quantitatively-anchored citations across six independent capability axes: Arithmetic (Python/symbolic execution), Code Execution (REPL/sandbox feedback), Format/Structure (constrained decoding, FSMs, grammar engines), Perception/Grounding (visual grounding for GUI, charts, tables), Knowledge/RAG (dense retrieval and citation grounding), Verification (proof checkers, learned verifiers, classifier rerouting, process supervision). Each axis is independently confirmed by between three and nine citations. We stratify the 28 by kind of evidence into three patterns:

Pattern A: hard guarantees (by-construction). Seven citations achieve residual error rate equal to zero by construction, restricted strictly to structural/verifiable classes: constrained decoders set \(P(\text{invalid token}) = 0\) at every step [16], [18], [47]–[49]; static syntax checks reject programs with a SyntaxError before execution [6]; proof kernels reject any output failing type-checking [50]. The class of grammar-violating outputs is mathematically empty under these mechanisms.

Pattern B: strong empirical reductions with class-shift signature. Roughly fourteen citations report empirical reductions of \(80\)–\(100\%\) in a named error class, with the post-intervention failure log dominated by structurally different residual classes. Program-of-Thoughts on GSM8K [14]: calculation errors drop from \(30\%\) of failures to \(0\%\), residuals are \(62\%\) reasoning + \(36\%\) misunderstanding. OpenMedCalc [51]: “only interpretation errors were identified.” Acurai [10]: \(100\%\) hallucination elimination on RAGTruth (\(95\%\) CI \(91\)–\(100\%\)), strong empirical rather than by-construction.

Pattern C: moderate reductions (\(60\)–\(80\%\)) with residuals shifting outside the target class. Legal RAG [52]: fabricated citations \(>30\%\) \(\rightarrow\) \(<0.2\%\). GPT-5 SimpleQA with web access [53]: \(47\% \rightarrow9.6\%\) (inter-condition, not within-condition). CRITIC [54]: toxic generation \(-79.2\%\).

9.0.0.2 The 28 citations, organised by axis and pattern.

9.0.0.3 Capabilities are coarser than error classes.

Several entries in the harvest reveal “two-for-one” reductions where one capability addresses multiple named clusters: a single Python interpreter removes the execution-error component of arithmetic, unit conversion, simple counting, list manipulation, and date arithmetic; constrained decoding eliminates by construction both format violations and the structural component of “missing required element”; code execution feedback strongly reduces SyntaxError, NameError, and most TypeError together; RAG strongly reduces factual hallucinations, fabricated citations, and outdated information jointly. The practical capability library required is therefore smaller than the count of named error categories.

9.0.0.4 The twelve named clusters.

Eight of twelve categories have a strong citation with double-digit percentage-point improvement; three (B, C, I) lack a clean single-cluster ablation; one (F) has consistent medium-strength evidence. Category I (missing required elements) splits mechanistically into I.a (structural, absorbed by D’s constrained decoders) and I.b (semantic, absorbed by G/H). Categories B (units) and C (counting) remain technical gaps: B is naturally folded into A; C is the smallest residual gap.

9.0.0.5 Additivity and its limits.

[25] demonstrate that stacking interventions targeting orthogonal failure modes can produce large compound reliability gains: their parallel-consensus framework yields a \(14{,}700\times\) improvement over single-pass baseline, evidence for compound gains from decomposition plus consensus aggregation under their specific parallel-voting regime rather than a direct demonstration that heterogeneous cluster-targeted interventions compose additively without voting. [68] similarly show supra-additive gains when modules target distinct failure modes. [26] reports that schema-level and prompt-level instructions interact non-additively when sharing a prompt channel: interventions on orthogonal processing layers (decoding constraint vs.retrieval vs.training-signal vs.inference-time tool call) compose near-additively, while those sharing a channel may interfere.

9.0.0.6 The irreducible-semantic residual.

Of the 17 ErrorAtlas categories, 13 are addressed under Patterns A, B, or C by one of the six capability axes; four are not: residuals of inappropriate refusal beyond preference optimisation, specification misinterpretation not closed by clarification, problem-decomposition reasoning bottlenecks, and a “user wanted something different” semantic remainder. These are classes where the failure is in choosing what to do, not executing it. Proposition 2’s prediction of \(O(\log)\) rather than \(O(0)\) residual reliability reflects exactly this irreducible core. The framework does not claim \(100\%\) coverage of all failures; it claims polylog-bounded capability-eliminable failures, with the named semantic residual as the floor.

10 Counter-Evidence Re-Audits↩︎

Five prominent papers are routinely cited as evidence that LLM reliability decays steeply with length [2], [31]–[33], [46]. A careful re-reading shows that every one of these papers decays over a variable distinct from raw token length \(n\). Identifying the decay axis is not the same as dissolving the practical concern: where compositional graph size, fact count, or evidence scope grow with problem length, the framework predicts steep failure curves. The contribution is to identify which interventions help (capability provisioning along the actual decay axis) and which do not.

10.0.0.1 [2], Faith and Fate.

GPT-4 multi-digit multiplication accuracy drops from 59% (3-digit) to 4% (4-digit) zero-shot, with the authors theorising “probability of incorrect predictions converges exponentially to \(\approx1\) for abstract compositional tasks.” The decay variable is compositional graph size \(N\), not raw token length: the \(3 \times 3\) graph has on the order of \(d^2\) partial products plus carries, and multi-digit multiplication is engineered so that \(k_{\text{hard}} \approxN\), with every node in the computation graph a hard decision. For natural-language tasks where \(k_{\text{hard}} \ll n\), Dziri’s regime is the boundary case in which our framework reduces to their result. A clean \((1-\varepsilon)^N\) exponential cannot simultaneously reproduce 59% at 3-digit and 4% at 4-digit for any single per-node \(\varepsilon\): the observed drop is locally steeper than per-node iid exponential, consistent with a finite catalogue of failure modes exhausting as \(N\) grows. Where the framework concedes: in adversarial compositional tasks engineered so \(k_{\text{hard}} \approxN\), the framework reduces to Dziri’s regime and does not relieve it; the relocation is informational, not magical.

10.0.0.2 [31].

The abstract reports “performance declines sharply with increased reasoning complexity” and models “effectively utilise only \(10\)–\(20\%\) of the context.” Read in detail, BABILong varies two axes: context length \(n\) (\(0\)K to \(10\)M tokens) and number of supporting facts \(k\) (QA1 \(=1\) fact to QA3 \(=3\) facts). The sharp decline is in \(k\), not \(n\). For QA1 (single-fact), most models “perform well up to 4,000 tokens”: a plateau, not exponential decay. The famous RAG-flat-across-length result (\(60\%\) on single-fact QA independent of context length) is the cleanest possible demonstration that when relevant evidence is in window, length does not matter. Recurrent Memory Transformers maintaining performance to 50M tokens further confirms effective \(k\) is determined by architecture, not raw \(n\). The framework’s concession is narrow but real: for multi-hop tasks whose required fact count grows with task complexity, BABILong’s sharp \(k\)-axis decay is exactly what the framework predicts happens, not a counter-example.

10.0.0.3 [32] (METR).

Per-model success fits a logistic in \(\log(\text{human-task-duration})\): \(S(t) = \sigma(\beta (\log t - \log h))\). Logistic-in-log-length is mathematically sublinear in length itself. The 80%-horizon being \(4\)–\(6\times\) shorter than the 50%-horizon is a steep within-model cliff but compatible by construction with the polylog result: a cliff at a specific capacity threshold is a manifold-transition signature. The famous exponential (capability doubling every seven months) is an inter-model claim about how the horizon \(h\) moves across generations, orthogonal to within-model decay shape. An honest qualifier: the within-model logistic-in-log cliff is steep on a practical scale. Calling it “sublinear in length” is technically correct but engineering-useful only for systems designed to operate well below the 50% horizon.

10.0.0.4 [33], Fano-style upper bound.

This paper theorises super-linear information-demand growth and identifies an “accuracy cliff” at capacity overflow: “when the task’s information demand surpasses the model’s output capacity, performance does not degrade gracefully but instead collapses sharply.” The behavioural prediction is a threshold, not smooth \((1-\varepsilon)^n\). A cliff at a specific capacity threshold is exactly the manifold-transition behaviour our framework predicts at the boundary between covered and uncovered modes; Wan et al.’s mechanism (capacity overflow) is one specific cause, complementary to ours. Where the framework concedes: Wan documents a mechanism by which \(|C|\) effectively exceeds the model’s coverage in a single forward pass; Pattern A interventions (constrained decoding, formal verification) are the kind of response the framework expects but does not automatically supply.

10.0.0.5 [46].

GPT-4o achieves 55.8% pair accuracy across 1,001 minimally-different true/false claim pairs about long fictional books (mean length 127K tokens). The paper reports performance by evidence scope, not by context length: 59.8% on sentence-level retrieval, 47.6% on passage-level, 41.6% on global reasoning. No per-context-length curve is reported. The decay axis is the number of evidence pieces that must be integrated, a \(k\)-axis quantity rather than an \(n\)-axis one. NoCha is therefore not counter-evidence to a sublinear-in-\(n\) claim. Evidence-scope decay is exactly the \(k\)-axis observation the framework predicts cannot be addressed by scaling raw context length; it requires retrieval or process supervision along the actual decay axis. The relocation is informational, not magical.

10.0.0.6 Unifying observation.

Each of the steep-decay counter-papers we re-audit decays over a variable other than \(n\): compositional graph size, fact count, log-time horizon, capacity threshold, or evidence scope. The apparent rapid decay is, in each case, in a quantity our framework already concentrates the action in (\(k_{\text{hard}}\) and \(|C|\)), not in raw sequence length. This is a relocation, not a dissolution. Where \(k_{\text{hard}}\) grows with task length (adversarial compositional structure, multi-hop fact chains, long horizons that force more decisions), reliability remains hard; the framework’s value is directing intervention toward capability provisioning along the actual decay axis rather than toward context-window or compute-budget expansion that does not help.

11 Patch Evidence Detail↩︎

The body’s claim that domain patches cap the engineering problem rests on a triangulation of indirect evidence. None of the items below measures the patch ceiling \(|C_D|\) directly; they support the weaker claim that model behaviour is strongly domain-dependent and that operational neighbourhoods occupy bounded regions of the model’s behavioural space.

The purpose of this evidence is motivational. It supports patch-indexing of \(\sigma_D\), \(A_D\), \(\beta_D\), \(|C_D|\), but the finite (or effectively capped) reachable catalogue remains an empirical modelling assumption to be measured per deployment, not derived from any of the rows above.

12 Empirical Calibration Detail↩︎

Three published LLM error taxonomies anchor the mode-rate parameter \(\sigma\) in the body’s \(\sigma \in [0.87, 1.85]\) range. The simple calibration uses \(|C| \approx A + \sigma \ln T\) with \(A = 0\); this is a deliberately conservative readout that ignores any positive intercept and uses endpoint counts rather than discovery curves.

These are endpoint counts, not discovery curves: they tell us how many named categories a taxonomer assigned at a single corpus scale, not how the count grew with \(T\). They do not prove logarithmic mode discovery. Their role in this paper is twofold: they motivate the empirical postulate of §3.2, and they identify the explicit empirical test that would either tighten the postulate or move the analysis to the Heaps variant of Appendix 8.5: repeatedly subsample failures from a fixed deployment patch \(D\) and plot discovered modes \(|C_{\text{seen},D}(T)|\) against \(T\).

References↩︎