Proof. Since \(\delta_u = 0\), the set of correct outputs \(\{y : V(x,y) = 1\}\) is well-defined for each \(x\) and does not depend on the evaluator. The reward \(r(y) = V(x,y)\) is therefore a deterministic function of \((x, y)\). In contrast, when \(\delta_u > 0\), any reward derived from human preferences inherits the irreducible disagreement encoded in \(C_u\): the reward for the same \((x, y)\) pair varies across annotators, introducing label noise with variance proportional to \(\delta_u\). ◻

Proof. If \(\delta_u > 0\), then \(I(C_u; Y \mid X, C_r) > 0\), so \(H(Y \mid X, C_r) > 0\). The conditional distribution \(P(Y \mid X, C_r)\) assigns positive probability to multiple distinct outputs for the same input, even given all resolvable context. A deterministic verifier \(V(x, y)\) partitions \(\mathcal{V}^L\) into accepted and rejected outputs for each \(x\). Either (a) \(V\) accepts all outputs with positive probability under \(P(Y \mid X, C_r)\), in which case it cannot distinguish among the multiple valid outputs for a given input and therefore provides no verification signal, or (b) \(V\) rejects some outputs that have positive probability under the true distribution, in which case it disagrees with the data-generating process on a set of positive measure. In neither case \(V\) is a reliable verifier. There is no ground truth to verify against, because the correct output depends on the subjective context \(C_u\) that varies across evaluators. ◻

Proof. The Markov chain \(Y \leftrightarrow (X,C) \leftrightarrow X \leftrightarrow T \leftrightarrow \hat{Y}\) yields three applications of the DPI:

Step 1: Decoder loss. Since \(\hat{Y}\) is a stochastic function of \(T\) (i.e., \(\hat{Y} \perp (X, Y, C) \mid T\)), and \(\phi(\hat{Y})\) is a deterministic function of \(\hat{Y}\), the DPI gives \[I(\phi(\hat{Y}); Y) \;\leq\; I(\hat{Y}; Y) \;\leq\; I(T; Y).\] The first inequality holds because \(\phi(\hat{Y})\) is a (possibly lossy) processing of \(\hat{Y}\); the second because \(\hat{Y}\) is generated from \(T\) alone.

Step 2: Encoder loss. Since \(T\) is computed from \(X\) alone (i.e., \(T \perp C \mid X\), and hence \(T\) is a function of \(X\) and possibly independent randomness), the DPI gives \[I(T; Y) \;\leq\; I(X; Y).\]

Step 3: Marginalization loss. Since \(X\) is a marginal of \((X, C)\), the DPI gives \[I(X; Y) \;\leq\; I((X,C); Y).\]

Chaining Steps 1–3 yields ?? : \(I(\phi(\hat{Y}); Y) \leq I(T; Y) \leq I(X; Y)\).

For the ceiling bound, take \(\phi\) to be the identity. Then \(I(\hat{Y}; Y) \leq I(X; Y)\) for any model \(Q_{Y|X}\). Dividing both sides by \(H(Y) > 0\) (which holds for any non-degenerate task) and taking the supremum over all models: \[\sup_{Q_{Y|X}} \frac{I(\hat{Y}; Y)}{H(Y)} \;\leq\; \frac{I(X; Y)}{H(Y)} \;=\; R^{*}. \qedhere\] ◻

Proof. Part (i). At step \(t\), the autoregressive model generates \(\hat{y}_t\) conditioned on the prefix \(\hat{y}_{<t}\), while the true next token \(y_t\) is drawn from \(P(\cdot \mid y_{<t}, X)\). The distributions governing these two draws differ because the prefixes differ.

By Assumption 1, the KL divergence between the true and corrupted continuation distributions is bounded: \[\mathrm{KL}\bigl(P(\cdot \mid y_{<t}, X) \;\big\|\; P(\cdot \mid \hat{y}_{<t}, X)\bigr) \;\leq\; \omega\bigl(d(\hat{y}_{<t}, y_{<t})\bigr) \;=\; \Delta_t.\]

We relate this KL divergence to the increase in error probability via a coupling argument. Construct an optimal coupling \((Z, Z')\) of \(P(\cdot \mid y_{<t}, X)\) and \(P(\cdot \mid \hat{y}_{<t}, X)\) such that \(P[Z \neq Z'] = \mathrm{TV}\bigl(P(\cdot \mid y_{<t}, X),\, P(\cdot \mid \hat{y}_{<t}, X)\bigr)\).

By Pinsker’s inequality, the total variation distance is bounded below: \[\label{eq:pinsker-tv} P[Z \neq Z'] \;=\; \mathrm{TV} \;\geq\; \sqrt{\frac{\mathrm{KL}}{2}} \;\geq\; \sqrt{\frac{\Delta_t}{2}}\,.\tag{6}\]

The error probability under the corrupted prefix decomposes over the coupling. Let \(E_t = \{y_t : d(\hat{y}_{\leq t}, y_{\leq t}) > \eta\}\) denote the error event. Conditioning on whether the coupling agrees: \[\begin{align} \epsilon_t &= P[E_t \mid \hat{y}_{<t}] \notag \\ &= P[E_t \mid Z = Z'] \cdot P[Z = Z'] \;+\; P[E_t \mid Z \neq Z'] \cdot P[Z \neq Z'] \notag \\ &= \epsilon_t^* \cdot (1 - \mathrm{TV}) \;+\; P[E_t \mid Z \neq Z'] \cdot \mathrm{TV}, \label{eq:coupling-decomp} \end{align}\tag{7}\] where we used \(P[E_t \mid Z = Z'] = \epsilon_t^*\) (when the coupling agrees, the corrupted and true distributions produce the same token).

When \(Z \neq Z'\), the generated token differs from what the true-prefix distribution would have produced. However, with tolerance \(\eta > 0\), this different token may still satisfy \(d \leq \eta\), so we cannot set \(P[E_t \mid Z \neq Z'] = 1\). Instead, let \[p_{\mathrm{flip},t} \;:=\; P[E_t \mid Z \neq Z', \hat{y}_{<t}]\] denote the probability that a “flipped” token causes an error. We have \(p_{\mathrm{flip},t} \geq \epsilon_t^*\) (a token from a different distribution is at least as likely to be erroneous as one from the correct distribution), so 7 gives \[\begin{align} \epsilon_t &= \epsilon_t^*(1 - \mathrm{TV}) + p_{\mathrm{flip},t} \cdot \mathrm{TV} \\ &= \epsilon_t^* + \mathrm{TV} \cdot (p_{\mathrm{flip},t} - \epsilon_t^*) \\ &\geq \epsilon_t^* + (1 - \epsilon_t^*) \cdot p_{\mathrm{flip}}(\Delta_t), \end{align}\] where the last step uses \(p_{\mathrm{flip},t} - \epsilon_t^* \geq (1 - \epsilon_t^*) \cdot q\) for a constant \(q > 0\) depending on the task (reflecting that a randomly “wrong” token has probability at least \(q\) of falling outside tolerance), and defines \[p_{\mathrm{flip}}(\Delta_t) \;:=\; q \cdot \mathrm{TV} \;\geq\; q\sqrt{\Delta_t/2} \;\geq\; \tfrac{1}{2}\min(\Delta_t, 1),\] where the last inequality holds for \(q \geq 1/\sqrt{2}\) and \(\Delta_t \in [0, 1]\).

For exact token match (\(\eta = 0\)), we have \(p_{\mathrm{flip},t} = 1\) whenever \(Z \neq Z'\), recovering the simpler bound \(\epsilon_t \geq \epsilon_t^* + (1 - \epsilon_t^*) \cdot \mathrm{TV}\).

Part (ii). In the optimistic case where drift feedback is ignored (i.e., \(\epsilon_t = \epsilon_t^* \leq \epsilon\) for all \(t\)), the probability that the entire sequence stays within tolerance \(\eta\) of the target is \[\mathbb{P}[d(\hat{Y}, Y) \leq \eta] \;\leq\; \prod_{t=1}^{L} (1 - \epsilon_t^*) \;\leq\; (1 - \epsilon)^L \;\leq\; e^{-\epsilon L},\] where the first inequality holds because the sequence is correct only if every step is correct, and the last uses \(1 - x \leq e^{-x}\) for \(x \geq 0\). Hence \(\mathbb{P}[d(\hat{Y}, Y) > \eta] \geq 1 - e^{-\epsilon L}\).

When drift feedback is present, the per-step error \(\epsilon_t\) grows with \(t\) by Part (i): each error increases the drift \(\Delta_{t'}\) for future steps \(t' > t\), which in turn raises \(\epsilon_{t'}\). The product \(\prod_t (1 - \epsilon_t)\) therefore decays strictly faster than \((1 - \epsilon)^L\).

Part (iii). Theorem 1 gives \(R^*_{\mathrm{seq}} \leq R^*\), so we only prove \(R^*_{\mathrm{seq}} \leq \exp(-c \sum_t \Delta_t)\).

By the chain rule for mutual information, conditioning on the input \(X\): \[I(\hat{Y}; Y \mid X) \;=\; \sum_{t=1}^{L} I(\hat{y}_t; Y \mid \hat{y}_{<t}, X).\] We show that each term satisfies \[\label{eq:dpi-step} I(\hat{y}_t; Y \mid \hat{y}_{<t}, X) \;\leq\; I(\hat{y}_t; Y \mid y_{<t}, X).\tag{8}\] At each step \(s < t\), the model generates \(\hat{y}_s\) from \((\hat{y}_{<s}, X)\) and independent randomness \(\xi_s\), so the corrupted prefix satisfies \(\hat{y}_{<t} = g(X, \xi_1, \ldots, \xi_{t-1})\) for some measurable \(g\), with \(\xi_{<t} \perp (Y, C) \mid X\). Conditioning on \((y_{<t}, X)\) therefore screens off \(\hat{y}_{<t}\) from \(Y\): \[Y \;\perp\; \hat{y}_{<t} \;\mid\; (y_{<t},\, X),\] yielding the Markov chain \[\label{eq:valid-markov} Y \;\longleftrightarrow\; (y_{<t},\, X) \;\longleftrightarrow\; (\hat{y}_{<t},\, X).\tag{9}\] The DPI applied to 9 gives 8 .

For each \(t\), define the per-step information loss: \[\delta_t \;:=\; I(\hat{y}_t; Y \mid y_{<t}, X) \;-\; I(\hat{y}_t; Y \mid \hat{y}_{<t}, X) \;\geq\; 0.\] Under Assumption 1 with a Lipschitz modulus \(\omega(x) = \lambda x\), the per-step loss satisfies \(\delta_t \geq c' \Delta_t\) for a constant \(c'\) depending on the local geometry of the conditional distribution.

Summing over \(t\): \[\begin{align} I(\hat{Y}; Y \mid X) &= \sum_{t=1}^{L} \bigl[I(\hat{y}_t; Y \mid y_{<t}, X) - \delta_t\bigr] \notag \\ &\leq \sum_{t=1}^{L} I(\hat{y}_t; Y \mid y_{<t}, X) \;-\; c' \sum_{t=1}^{L} \Delta_t. \label{eq:cond-bound} \end{align}\tag{10}\] The first sum is the teacher-forced mutual information: under teacher forcing, the model receives \(y_{<t}\) at each step and generates \(\hat{y}_t \sim Q_\theta(\cdot \mid y_{<t}, X)\). Each term satisfies \(I(\hat{y}_t; Y \mid y_{<t}, X) \leq H(y_t \mid y_{<t}, X)\), so \[\sum_{t=1}^{L} I(\hat{y}_t; Y \mid y_{<t}, X) \;\leq\; H(Y \mid X).\] Combining: \[\label{eq:IYX-degraded} I(\hat{Y}; Y \mid X) \;\leq\; H(Y \mid X) \;-\; c' \sum_{t=1}^{L} \Delta_t.\tag{11}\]

The bound 11 controls the conditional mutual information. To bound the unconditional \(I(\hat{Y}; Y)\) that appears in the definition of \(R^*_{\mathrm{seq}}\), we use the chain rule identity \[I(\hat{Y}; Y) \;=\; I(\hat{Y}; Y \mid X) \;+\; I(X; Y) \;-\; I(X; Y \mid \hat{Y}) \leq I(\hat{Y}; Y \mid X) + I(X; Y).\] since \(I(X; Y \mid \hat{Y}) \geq 0\). Substituting 11 : \[\begin{align} I(\hat{Y}; Y) &\leq \bigl[H(Y \mid X) - c' \textstyle\sum_t \Delta_t\bigr] + I(X; Y) \notag \\ &= H(Y) \;-\; c' \sum_{t=1}^{L} \Delta_t, \end{align}\] Dividing by \(H(Y)\) and applying \(1 - x \leq e^{-x}\), we get the needed. ◻

Proof. The proof bounds the cumulative drift \(\sum_t \Delta_t\) separately for banded and dense kernels, then substitutes into \(\gamma(L) := \exp(-c \sum_t \Delta_t)\).

Part (i): Banded kernel. Suppose \(K_Y\) has bandwidth \(w\), meaning \(K_Y(t_0, t) \approx 0\) for \(|t - t_0| > w\). An error at position \(t_0\) induces drift \(\Delta_t \leq \lambda K_Y(t_0, t) + \mu\) at subsequent positions. For \(t > t_0 + w\), the first term vanishes: \(K_Y(t_0, t) = 0\), so \(\Delta_t \leq \mu\) (only the baseline drift remains). The error at \(t_0\) can therefore trigger secondary errors only within the window \([t_0, t_0 + w]\), and each secondary error similarly has bounded propagation range \(w\).

We bound the total drift by counting error “waves.” The total number of waves that can propagate through the sequence is at most \(\lceil L/w \rceil\). Within each wave, the accumulated drift is bounded by \[\sum_{s=0}^{w} \bigl(\lambda K_Y(t_0, t_0+s) + \mu\bigr) \;\leq\; \lambda \|K_Y\|_1^{\mathrm{row}} + w\mu,\] where \(\|K_Y\|_1^{\mathrm{row}} = \max_t \sum_{t'} K_Y(t, t')\) is the maximum row sum. An error occurs at rate \(\epsilon\) per step, so the expected number of error-initiating positions in \(L\) steps is \(\epsilon L\). Each contributes drift over a window of size \(w\). The total cumulative drift is therefore \[\sum_{t=1}^{L} \Delta_t \;\leq\; \epsilon L \cdot \bigl(\lambda \|K_Y\|_1^{\mathrm{row}} + w\mu\bigr).\] So that \[\gamma(L) = \exp (-c \sum_t \Delta_t) \;\leq\; \exp\bigl(-c \cdot \epsilon L \cdot (\lambda \|K_Y\|_1^{\mathrm{row}} + w\mu)\bigr) \;=\; \exp(-c_1 \epsilon L),\] where \(c_1 = c(\lambda \|K_Y\|_1^{\mathrm{row}} + w\mu)\) depends on the bandwidth and kernel magnitude but enters only as a constant prefactor—the dependence on \(L\) is linear.

In the banded case, errors do not accumulate across waves: once a wave exits the bandwidth window, it ceases to influence future positions. This locality is what prevents the superlinear drift accumulation that arises in the dense case.

Part (ii): Dense kernel. For dense \(K_Y\), an error at any position \(t_0\) induces non-negligible drift at all subsequent positions. Concretely, for a dense kernel with approximately uniform off-diagonal mass \(K_Y(s,t) \approx \kappa / L\), the drift at position \(t\) accumulates contributions from all prior errors: \[\Delta_t \;\leq\; \lambda \sum_{s < t} K_Y(s, t) \cdot \mathbb{I}[\text{error at } s] \;+\; \mu \;\approx\; \frac{\lambda \kappa}{L} \sum_{s < t} \mathbb{I}[\text{error at } s] \;+\; \mu.\]

To derive the recursion on the expected error rate, we aggregate these drift contributions over all prior positions. Taking expectations over the randomness in the generation process and using linearity, the expected drift at position \(t\) satisfies \[\mathbb{E}[\Delta_t] \;\leq\; \lambda \sum_{s < t} K_Y(s, t) \cdot \bar{\epsilon}(s) \;+\; \mu,\] where \(\bar{\epsilon}(s) := \mathbb{E}[\mathbb{I}[\text{error at } s]]\) is the expected error rate at position \(s\). For a dense kernel with approximately uniform off-diagonal mass \(K_Y(s,t) \approx \kappa/L\), this becomes \(\mathbb{E}[\Delta_t] \leq (\lambda\kappa/L)\sum_{s<t} \bar{\epsilon}(s) + \mu\). By Proposition 2(i), a positive expected drift at position \(t\) raises the per-step error probability above the base rate: \(\bar{\epsilon}(t) \geq \epsilon + (1 - \epsilon) \cdot p_{\mathrm{flip}}(\mathbb{E}[\Delta_t])\). Since \(p_{\mathrm{flip}}\) is nondecreasing and satisfies \(p_{\mathrm{flip}}(x) \geq \tfrac{1}{2}\min(x,1)\), the feedback from \(\mathbb{E}[\Delta_t]\) into \(\bar{\epsilon}(t)\) yields, to leading order in \(\lambda\kappa\), \[\bar{\epsilon}(t) \;\approx\; \epsilon \;+\; \frac{\lambda\kappa}{L} \sum_{s < t} \bar{\epsilon}(s). \label{eq:ebar-recursion}\tag{12}\]

With dense \(K_Y\) (uniform off-diagonal mass \(\kappa/L\)), this becomes \(\bar{\epsilon}(t) \approx \epsilon + (\lambda \kappa / L) \sum_{s < t} \bar{\epsilon}(s)\). Define \(S(t) := \sum_{s=1}^{t-1} \bar{\epsilon}(s)\), so \(S(t+1) - S(t) = \bar{\epsilon}(t) = \epsilon + (\lambda\kappa/L)\,S(t)\). This is a first-order linear recurrence whose solution is \[\label{eq:gronwall-dense} \bar{\epsilon}(t) \;=\; \epsilon\Bigl(1 + \frac{\lambda\kappa}{L}\Bigr)^{t-1}.\tag{13}\] For \(t \leq L\), we have \(\bar{\epsilon}(t) \leq \epsilon\,(1 + \lambda\kappa/L)^{L} \leq \epsilon\, e^{\lambda\kappa}\). The expected error rate grows geometrically within the sequence but remains bounded by a constant (depending on \(\lambda\kappa\)) times the base rate \(\epsilon\).

Substituting into the cumulative drift: \[\sum_{t=1}^{L} \bar{\epsilon}(t) \;=\; \epsilon \sum_{t=0}^{L-1} \Bigl(1 + \frac{\lambda\kappa}{L}\Bigr)^{t} \;=\; \epsilon \cdot \frac{(1 + \lambda\kappa/L)^{L} - 1}{\lambda\kappa / L} \;=\; \frac{\epsilon L}{\lambda\kappa}\bigl(e^{\lambda\kappa} - 1\bigr)\bigl(1 + o(1)\bigr).\] Therefore \[\sum_{t=1}^{L} \Delta_t \;\geq\; c \sum_{t=1}^{L} \bar{\epsilon}(t) \;\geq\; \frac{c\,\epsilon\, L}{\lambda\kappa}\bigl(e^{\lambda\kappa} - 1\bigr)\bigl(1 + o(1)\bigr),\] giving \(\gamma(L) \;\leq\; \exp(-c_2\, \epsilon\, L)\), where \(c_2 = c\,(e^{\lambda\kappa} - 1)/(\lambda\kappa)\) depends on \(\lambda\), \(\kappa\), and the constant from Proposition 2(iii). ◻

Proof. The cross-entropy loss of an autoregressive model averages over positions: \(\mathcal{\mathcal{L}}_{\mathrm{est}}(D) = L^{-1}\sum_{t=1}^L \mathcal{L}_{\mathrm{est}}^{(t)}(D)\). We prove \(\mathcal{L}_{\mathrm{est}}^{(t)}(D) = B_t D_{\mathrm{eff}}^{-\beta} + o(D_{\mathrm{eff}}^{-\beta})\) for a generic position \(t\). The exponent \(\beta\) is position-independent because Assumption 5 bounds the eigenvalues \(\bar{\lambda}_k^{(t)}\) uniformly across \(t\); only the prefactor \(B_t\) varies with \(t\). Averaging over positions gives \(\mathcal{L}_{\mathrm{est}}(D) = B D_{\mathrm{eff}}^{-\beta} + o(D_{\mathrm{eff}}^{-\beta})\) with \(B = L^{-1}\sum_t B_t\).

Step 1: Mode-wise error. Under Assumption 7, the estimation error on eigenmode \(k\) of \(\bar{K}\) is \[\mathrm{err}_k(D) \;=\; \frac{\mathbb{E}_\tau[\bar{f}_k^{\tau\,2}]}{1 + D_{\mathrm{eff}}\,\bar{\lambda}_k / \sigma_{\mathrm{noise}}^2}.\] By Assumption 5, \(\bar{\lambda}_k \geq c_{\min}\bar{\sigma}_k\), and by Assumption 6, \(\mathbb{E}_\tau[\bar{f}_k^{\tau\,2}] = \bar{f}_0^2 k^{-\bar{\mu}}(1 + o(1))\) as \(k \to \infty\). With \(\bar{\sigma}_k = \bar{\sigma}_1 k^{-\nu_{\mathcal{T}}}(1 + o(1))\) from Assumption 4: \[\mathrm{err}_k(D) \;\leq\; \frac{\bar{f}_0^2 k^{-\bar{\mu}}(1+o(1))}{1 + D_{\mathrm{eff}}\,c_{\min}\,\bar{\sigma}_1\,k^{-\nu_{\mathcal{T}}}(1+o(1)) / \sigma_{\mathrm{noise}}^2}.\]

Step 2: Crossover index. Define \(k^*(D)\) as the mode index at which the data term in the denominator equals \(1\): \[D_{\mathrm{eff}}\,c_{\min}\,\bar{\sigma}_1\,(k^*)^{-\nu_{\mathcal{T}}} / \sigma_{\mathrm{noise}}^2 \;=\; 1, \qquad k^*(D) \;=\; \Bigl(\frac{D_{\mathrm{eff}}\,c_{\min}\,\bar{\sigma}_1}{\sigma_{\mathrm{noise}}^2}\Bigr)^{1/\nu_{\mathcal{T}}}.\] Note that \(k^* \to \infty\) as \(D_{\mathrm{eff}} \to \infty\), so the asymptotic forms of \(\bar{\sigma}_k\) and \(\mathbb{E}_\tau[\bar{f}_k^{\tau\,2}]\) apply for all modes near and beyond the crossover.

Step 3: Splitting the sum. We split \(\mathcal{L}_{\mathrm{est}}(D) = S_{\leq}(D) + S_{>}(D)\), where \(S_{\leq}\) sums over learned modes \(k \leq k^*\) and \(S_{>}\) over unlearned modes \(k > k^*\).

Unlearned tail. For \(k > k^*\), the denominator satisfies \(1 \leq 1 + D_{\mathrm{eff}}\bar{\lambda}_k/\sigma_{\mathrm{noise}}^2 \leq 2\), so \(\mathrm{err}_k(D) = \mathbb{E}_\tau[\bar{f}_k^{\tau\,2}](1 + O(k^*/k)^{\nu_{\mathcal{T}}})^{-1}\). Therefore \[S_{>}(D) \;=\; \sum_{k > k^*} \bar{f}_0^2 k^{-\bar{\mu}} \;+\; O\Bigl(\sum_{k > k^*} k^{-\bar{\mu}} \cdot (k^*/k)^{\nu_{\mathcal{T}}}\Bigr).\] The error term is \(O((k^*)^{\nu_{\mathcal{T}}} \sum_{k > k^*} k^{-\bar{\mu}-\nu_{\mathcal{T}}}) = O((k^*)^{-(\bar{\mu}-1)}\cdot (k^*)^{-1})\) when \(\bar{\mu} + \nu_{\mathcal{T}} > 1\), which is \(O(D_{\mathrm{eff}}^{-\beta - 1/\nu_{\mathcal{T}}})\).

For the leading term, by the Euler–Maclaurin formula: \[\sum_{k > k^*} k^{-\bar{\mu}} \;=\; \frac{(k^*)^{-(\bar{\mu}-1)}}{\bar{\mu}-1} \;+\; O\bigl((k^*)^{-\bar{\mu}}\bigr),\] where convergence requires \(\bar{\mu} > 1\). The remainder \(O((k^*)^{-\bar{\mu}})\) is a multiplicative \(O((k^*)^{-1})\) correction to the leading term, i.e., \(O(D_{\mathrm{eff}}^{-\beta - 1/\nu_{\mathcal{T}}})\).

Combining: \[S_{>}(D) \;=\; \frac{\bar{f}_0^2}{\bar{\mu}-1}\,(k^*)^{-(\bar{\mu}-1)} \;+\; O\bigl(D_{\mathrm{eff}}^{-\beta - 1/\nu_{\mathcal{T}}}\bigr). \label{eq:tail95error}\tag{14}\]

Step 4: Substituting the crossover index. Replacing \(k^*\) with its expression from Step 2: \[\begin{align} S_{>}(D) &\;=\; \frac{\bar{f}_0^2}{\bar{\mu}-1}\,\Bigl(\frac{D_{\mathrm{eff}}\,c_{\min}\,\bar{\sigma}_1}{\sigma_{\mathrm{noise}}^2}\Bigr)^{-(\bar{\mu}-1)/\nu_{\mathcal{T}}} \;+\; O\bigl(D_{\mathrm{eff}}^{-\beta - 1/\nu_{\mathcal{T}}}\bigr) \nonumber \\ &\;=\; B \cdot D_{\mathrm{eff}}^{-\beta} \;+\; O\bigl(D_{\mathrm{eff}}^{-\beta - 1/\nu_{\mathcal{T}}}\bigr), \label{eq:tail95substituted} \end{align}\tag{15}\] where \(\beta = (\bar{\mu}-1)/\nu_{\mathcal{T}}\) and \[B \;=\; \frac{\bar{f}_0^2}{\bar{\mu}-1}\,\Bigl(\frac{c_{\min}\,\bar{\sigma}_1}{\sigma_{\mathrm{noise}}^2}\Bigr)^{-(\bar{\mu}-1)/\nu_{\mathcal{T}}}.\]

Note that this identifies the position-level prefactor \(B_t\).

Step 5: Learned-mode contribution. For modes \(k \leq k^*\), the denominator satisfies \(D_{\mathrm{eff}} \bar{\lambda}_k / \sigma^2_{\mathrm{noise}} \geq 1\), so \(\mathrm{err}_k(D) \leq \mathbb{E}_\tau[\bar{f}_k^{\tau\,2}] \cdot \sigma^2_{\mathrm{noise}} / (D_{\mathrm{eff}} \bar{\lambda}_k)\). Therefore \[S_{\leq}(D) \;\leq\; \frac{\bar{f}_0^2\, \sigma^2_{\mathrm{noise}}}{D_{\mathrm{eff}}\, c_{\min}\, \bar{\sigma}_1} \sum_{k=1}^{k^*} k^{\nu_{\mathcal{T}} - \bar{\mu}}.\] We consider two cases.

Case \(\bar{\mu} > \nu_{\mathcal{T}} + 1\): The sum \(\sum_{k=1}^{k^*} k^{\nu_{\mathcal{T}} - \bar{\mu}}\) converges to a constant \(C_1 < \infty\) as \(k^* \to \infty\), since the exponent \(\nu_{\mathcal{T}} - \bar{\mu} < -1\). Then \(S_{\leq}(D) = \Theta(D_{\mathrm{eff}}^{-1})\). In this regime, \(\beta = (\bar{\mu}-1)/\nu_{\mathcal{T}} > 1\), so \(D_{\mathrm{eff}}^{-\beta}\) decays faster than \(D_{\mathrm{eff}}^{-1}\): the unlearned tail \(S_>\) is lower order and \(S_{\leq}\) dominates. The effective scaling exponent saturates at \(\beta_{\mathrm{eff}} = 1\), regardless of how large \((\bar{\mu}-1)/\nu_{\mathcal{T}}\) is. This regime (very concentrated signal with rapidly decaying spectrum) is not the empirically relevant one for language modeling, where \(\beta < 1\) is consistently observed: Kaplan et al. [4] estimate \(\beta \approx 0.095\), Hoffmann et al. [5] estimate \(\beta \approx 0.34\), and the corrected replication of Besiroglu et al. [16] gives \(\beta \approx 0.37\), all well below 1.

Case \(1 < \bar{\mu} \leq \nu_{\mathcal{T}} + 1\): The sum grows as \((k^*)^{\nu_{\mathcal{T}} - \bar{\mu} + 1}/(\nu_{\mathcal{T}} - \bar{\mu} + 1)\). Substituting \(k^* = \Theta(D_{\mathrm{eff}}^{1/\nu_{\mathcal{T}}})\): \[\begin{align} S_{\leq}(D) &= O\!\left(D_{\mathrm{eff}}^{-1} \cdot D_{\mathrm{eff}}^{(\nu_{\mathcal{T}} - \bar{\mu} + 1)/\nu_{\mathcal{T}}}\right) \\ &= O\!\left(D_{\mathrm{eff}}^{(-\nu_{\mathcal{T}} + \nu_{\mathcal{T}} - \bar{\mu} + 1)/\nu_{\mathcal{T}}}\right) \\ &= O\!\left(D_{\mathrm{eff}}^{-(\bar{\mu}-1)/\nu_{\mathcal{T}}}\right) \\ &= O\!\left(D_{\mathrm{eff}}^{-\beta}\right). \end{align}\] Therefore \(S_{\leq}(D) = \Theta(D_{\mathrm{eff}}^{-\beta})\): the same order as \(S_>(D)\). Both the learned and unlearned modes contribute to the leading-order prefactor.

Step 6: Combining. In the empirically relevant case \(1 < \bar{\mu} \leq \nu_{\mathcal{T}} + 1\) (which includes all language modeling settings where \(\beta < 1\)), the estimation loss is \[\mathcal{L}_{\mathrm{est}}(D) \;=\; S_{\leq}(D) + S_>(D) \;=\; (B_> + B_{\leq}) \cdot D_{\mathrm{eff}}^{-\beta} + o(D_{\mathrm{eff}}^{-\beta}),\] where \[\begin{align} B_> &= \frac{\bar{f}_0^2}{\bar{\mu} - 1} \left(\frac{c_{\min}\,\bar{\sigma}_1}{\sigma^2_{\mathrm{noise}}}\right)^{-(\bar{\mu}-1)/\nu_{\mathcal{T}}}, \\[4pt] B_{\leq} &= \frac{\bar{f}_0^2}{\nu_{\mathcal{T}} - \bar{\mu} + 1} \left(\frac{c_{\min}\,\bar{\sigma}_1}{\sigma^2_{\mathrm{noise}}}\right)^{-(\bar{\mu}-1)/\nu_{\mathcal{T}}}. \end{align}\] The total prefactor is \[B \;=\; B_> + B_{\leq} \;=\; \frac{\bar{f}_0^2\,\nu_{\mathcal{T}}}{(\bar{\mu}-1)(\nu_{\mathcal{T}} - \bar{\mu} + 1)} \left(\frac{c_{\min}\,\bar{\sigma}_1}{\sigma^2_{\mathrm{noise}}}\right)^{-\beta}.\] ◻

Proof. The approximation error averages over tasks and positions: \(\mathcal{L}_{\mathrm{approx}}(N) = \mathbb{E}_\tau[L^{-1}\sum_t \mathcal{L}_{\mathrm{approx}}^{(\tau,t)}(N)]\). As in the proof of Theorem 4, the position average does not affect the exponent (by Assumption 5), so we work at a generic position and average over tasks only.

Step 1: Capacity budget. Not all modes are equally costly to represent: modes carrying less variance in the task distribution require more precise parameterization to represent, analogous to the effective degrees of freedom in kernel ridge regression scaling inversely with the kernel eigenvalue [17]. Assumption 8 models this as a capacity cost proportional to \(\bar{\sigma}_k^{-1}\) for mode \(k\). The total capacity budget constrains the number of representable modes \(M_{\mathrm{eff}}\): \[\sum_{k=1}^{M_{\mathrm{eff}}} \bar{\sigma}_k^{-1} \;\leq\; M(N).\]

Here \(M_{\mathrm{eff}}\) is the number of modes the model can represent given that mode \(k\) consumes \(\bar{\sigma}_k^{-1}\) units from the total budget \(M(N)\).

Step 2: Effective number of representable modes. Under Assumption 4, \(\bar{\sigma}_k^{-1} = \bar{\sigma}_1^{-1}\,k^{\nu_{\mathcal{T}}}(1+o(1))\) as \(k \to \infty\). By the Euler–Maclaurin formula: \[\sum_{k=1}^{M_{\mathrm{eff}}} k^{\nu_{\mathcal{T}}} \;=\; \frac{M_{\mathrm{eff}}^{\,\nu_{\mathcal{T}}+1}}{\nu_{\mathcal{T}}+1} \;+\; O\bigl(M_{\mathrm{eff}}^{\,\nu_{\mathcal{T}}}\bigr).\] Setting this equal to \(\bar{\sigma}_1 M(N) = \bar{\sigma}_1 \kappa N^{1/d}\) and solving: \[M_{\mathrm{eff}} \;=\; \bigl(\bar{\sigma}_1\,\kappa\,(\nu_{\mathcal{T}}+1)\bigr)^{1/(\nu_{\mathcal{T}}+1)}\,N^{1/(d(\nu_{\mathcal{T}}+1))} \;\bigl(1 + O(M_{\mathrm{eff}}^{-1})\bigr).\] The correction \(O(M_{\mathrm{eff}}^{-1})\) comes from the Euler–Maclaurin remainder and is negligible for large \(N\).

Step 3: Approximation error from unrepresented tail. Modes \(k > M_{\mathrm{eff}}\) are not represented by the model, contributing their full target energy as approximation error. Averaging over tasks and applying Assumption 6: \[\mathcal{L}_{\mathrm{approx}}(N) \;=\; \sum_{k > M_{\mathrm{eff}}} \mathbb{E}_\tau\bigl[\bar{f}_k^{\tau\,2}\bigr] \;=\; \sum_{k > M_{\mathrm{eff}}} \bar{f}_0^2\,k^{-\bar{\mu}}(1+o(1)).\] By the Euler–Maclaurin formula (as in the proof of Theorem 4, Step 3): \[\sum_{k > M_{\mathrm{eff}}} k^{-\bar{\mu}} \;=\; \frac{M_{\mathrm{eff}}^{-(\bar{\mu}-1)}}{\bar{\mu}-1} \;+\; O\bigl(M_{\mathrm{eff}}^{-\bar{\mu}}\bigr),\] where convergence requires \(\bar{\mu} > 1\) (Assumption 6). Therefore \[\mathcal{L}_{\mathrm{approx}}(N) \;=\; \frac{\bar{f}_0^2}{\bar{\mu}-1}\,M_{\mathrm{eff}}^{-(\bar{\mu}-1)} \;+\; O\bigl(M_{\mathrm{eff}}^{-\bar{\mu}}\bigr).\] The remainder is a multiplicative \(O(M_{\mathrm{eff}}^{-1})\) correction to the leading term.

Step 4: Substitution. Inserting \(M_{\mathrm{eff}}\) from Step 2: \[\begin{align} \mathcal{L}_{\mathrm{approx}}(N) &\;=\; \frac{\bar{f}_0^2}{\bar{\mu}-1}\,\bigl(\bar{\sigma}_1\,\kappa\,(\nu_{\mathcal{T}}+1)\bigr)^{-(\bar{\mu}-1)/(\nu_{\mathcal{T}}+1)}\,N^{-(\bar{\mu}-1)/(d(\nu_{\mathcal{T}}+1))} \nonumber \\ &\qquad +\; O\bigl(N^{-(\bar{\mu}-1)/(d(\nu_{\mathcal{T}}+1)) - 1/(d(\nu_{\mathcal{T}}+1))}\bigr) \nonumber \\ &\;=\; A \cdot N^{-\alpha} \;+\; O\bigl(N^{-\alpha - 1/(d(\nu_{\mathcal{T}}+1))}\bigr), \label{eq:approx95substituted} \end{align}\tag{16}\] with \[\alpha \;=\; \frac{\bar{\mu}-1}{d\,(\nu_{\mathcal{T}}+1)},\] and position-level prefactor \[A_t \;=\; \frac{\bar{f}_0^2}{\bar{\mu}-1}\,\bigl(\bar{\sigma}_1\,\kappa\,(\nu_{\mathcal{T}}+1)\bigr)^{-(\bar{\mu}-1)/(\nu_{\mathcal{T}}+1)}.\] Averaging over positions gives \(A = L^{-1}\sum_t A_t\), and the final result is \(\mathcal{L}_{\mathrm{approx}}(N) = A N^{-\alpha} + o(N^{-\alpha})\), since the remainder \(O(N^{-\alpha - 1/(d(\nu_{\mathcal{T}}+1))})\) is \(o(N^{-\alpha})\). ◻

Proof. The lower bound is immediate from the identity 5 : since \(\mathrm{KL}(P_{Y|X} \| Q_{Y|X}) \geq 0\) for all \(X\), we have \(\mathcal{L}(N,D) = H(Y \mid X) + \mathbb{E}_X[\mathrm{KL}(P_{Y|X} \| Q_{Y|X})] \geq H(Y \mid X)\).

For the convergence, Theorems 4 and 5 give \(\mathbb{E}_X[\mathrm{KL}(P_{Y|X} \| Q_{Y|X})] = O(D^{-\beta}) + O(N^{-\alpha})\). Both terms vanish as \(N, D \to \infty\), so \(\mathcal{L}(N,D) \to H(Y \mid X)\). ◻

Proof. By the identity 5 , \(L(N,D) = H(Y|X) + \mathbb{E}_X[\mathrm{KL}(P_{Y|X} \| Q_{Y|X})]\). Under the kernel regime (Assumption 7), the KL gap decomposes along the eigenbasis of \(\bar{K}\) as \(\mathbb{E}_X[\mathrm{KL}(P_{Y|X} \| Q_{Y|X})] = \sum_{k=1}^{\infty} \mathrm{err}_k(N,D)\), where \(\mathrm{err}_k\) is the mode-wise estimation error defined in Assumption 7 for representable modes and the full projection energy \(\mathbb{E}_\tau[\bar{f}_k^{\tau\,2}]\) for unrepresentable modes. We partition this sum at \(M_{\mathrm{eff}}(N)\) and evaluate each part.

Approximation error (\(k > M_{\mathrm{eff}}\)). This is exactly the sum bounded in Theorem 5: \[\sum_{k > M_{\mathrm{eff}}} \mathbb{E}_\tau[\bar{f}_k^{\tau\,2}] \;=\; A\,N^{-\alpha} + O\bigl(N^{-\alpha - 1/(d(\nu_{\mathcal{T}}+1))}\bigr).\]

Estimation error on representable modes (\(k \leq M_{\mathrm{eff}}\)). We further split this sum at the data crossover \(k^*(D)\).

Learned modes (\(k \leq \min(k^*, M_{\mathrm{eff}})\)): The denominator satisfies \(D_{\mathrm{eff}}\bar{\lambda}_k/\sigma_{\mathrm{noise}}^2 \geq 1\), so \(\mathrm{err}_k \leq \mathbb{E}_\tau[\bar{f}_k^{\tau\,2}] \cdot \sigma_{\mathrm{noise}}^2/(D_{\mathrm{eff}}\bar{\lambda}_k)\). As shown in Step 5 of the proof of Theorem 4, this contributes \(o(D_{\mathrm{eff}}^{-\beta})\).

Unlearned representable modes (\(k^* < k \leq M_{\mathrm{eff}}\), when \(k^* < M_{\mathrm{eff}}\)): The denominator is between \(1\) and \(2\), so \(\mathrm{err}_k = \mathbb{E}_\tau[\bar{f}_k^{\tau\,2}](1 + O((k^*/k)^{\nu_{\mathcal{T}}}))^{-1}\). The leading contribution is a partial tail sum, which we express as the difference of two full tails: \[\begin{align} \sum_{k=k^*+1}^{M_{\mathrm{eff}}} \bar{f}_0^2\,k^{-\bar{\mu}} &\;=\; \underbrace{\sum_{k > k^*} \bar{f}_0^2\,k^{-\bar{\mu}}}_{\text{full tail from } k^*} \;-\; \underbrace{\sum_{k > M_{\mathrm{eff}}} \bar{f}_0^2\,k^{-\bar{\mu}}}_{\text{full tail from } M_{\mathrm{eff}}} \\[4pt] &\;=\; \frac{\bar{f}_0^2}{\bar{\mu}-1}(k^*)^{-(\bar{\mu}-1)} \;-\; \frac{\bar{f}_0^2}{\bar{\mu}-1}M_{\mathrm{eff}}^{-(\bar{\mu}-1)} \;+\; O\bigl((k^*)^{-\bar{\mu}} + M_{\mathrm{eff}}^{-\bar{\mu}}\bigr), \end{align}\] by the Euler–Maclaurin formula applied to each tail (as in Steps 3–4 of the proofs of Theorems 4 and 5). The first term equals \(B\,D_{\mathrm{eff}}^{-\beta}\) by the identification in Theorem 4. The second term equals \(A\,N^{-\alpha}\) by the identification in Theorem 5. The remainder terms are lower order than their respective leading terms.

If \(k^* \geq M_{\mathrm{eff}}\) (data is abundant relative to capacity): All representable modes are learned, and the estimation sum contributes only \(o(D_{\mathrm{eff}}^{-\beta})\). The dominant reducible error is the approximation tail \(A\,N^{-\alpha}\).

Combining. The total reducible error is \[\sum_{k=1}^{\infty} \mathrm{err}_k(N,D) \;=\; \underbrace{\sum_{k \leq \min(k^*, M_{\mathrm{eff}})} \mathrm{err}_k}_{\text{learned representable}} \;+\; \underbrace{\sum_{k > \min(k^*, M_{\mathrm{eff}})} \mathrm{err}_k}_{\text{unlearned or unrepresentable}}.\] The first sum is \(o(\max(N^{-\alpha}, D_{\mathrm{eff}}^{-\beta}))\) (learned modes contribute lower-order error). The second sum is a single tail from \(\min(k^*, M_{\mathrm{eff}})\): \[\sum_{k > \min(k^*, M_{\mathrm{eff}})} \bar{f}_0^2 k^{-\bar{\mu}} \;=\; \frac{\bar{f}_0^2}{\bar{\mu}-1}\bigl(\min(k^*, M_{\mathrm{eff}})\bigr)^{-(\bar{\mu}-1)} + \text{lower order}.\] The binding constraint is whichever of \(k^*\) and \(M_{\mathrm{eff}}\) is smaller:

• When \(k^* < M_{\mathrm{eff}}\) (data is the bottleneck): the tail starts at \(k^*\), giving \(B\,D_{\mathrm{eff}}^{-\beta}\).

• When \(M_{\mathrm{eff}} < k^*\) (capacity is the bottleneck): the tail starts at \(M_{\mathrm{eff}}\), giving \(A\,N^{-\alpha}\).

• When \(k^* \approx M_{\mathrm{eff}}\) (balanced regime): both terms are of the same order.

Therefore: \[\mathcal{L}(N,D) \;=\; H(Y|X) + \frac{\bar{f}_0^2}{\bar{\mu}-1}\bigl(\min(k^*, M_{\mathrm{eff}})\bigr)^{-(\bar{\mu}-1)} + \text{lower order}.\] Expressed in terms of \(N\) and \(D\): \[\mathcal{L}(N,D) \;=\; H(Y|X) + \max\bigl(A\,N^{-\alpha},\; B\,D^{-\beta}\bigr) + o\bigl(\max(N^{-\alpha}, D^{-\beta})\bigr). \qedhere\] ◻

Proof. By Theorem 7, \(\mathcal{L}(N,D) = H(Y|X) + \max(A N^{-\alpha}, B D^{-\beta}) + o(\max(N^{-\alpha}, D^{-\beta}))\). Along the compute-optimal frontier, \(A N^{-\alpha} \asymp B D^{-\beta}\), meaning there exist constants \(0 < c_l \leq c_u < \infty\) such that \(c_l \leq A N^{-\alpha} / (B D^{-\beta}) \leq c_u\). In this regime: \[\max(A N^{-\alpha}, B D^{-\beta}) \;\leq\; A N^{-\alpha} + B D^{-\beta} \;\leq\; (1 + c_u/c_l)\, \max(A N^{-\alpha}, B D^{-\beta}),\] so the \(\max\) and the sum differ by at most a bounded multiplicative constant, absorbed into the prefactors. Substituting: \[\mathcal{L}(N,D) = A N^{-\alpha} + B D^{-\beta} + H(Y|X) + o\bigl(\max(N^{-\alpha}, D^{-\beta})\bigr). \qedhere\] ◻

Proof. Part (i). By the chain rule of mutual information, \(I(X, \tilde{C}; Y) = I(X; Y) + I(\tilde{C}; Y \mid X)\). Since \(I(\tilde{C}; Y \mid X) \geq 0\) by non-negativity, \(I(X, \tilde{C}; Y) \geq I(X; Y)\), and dividing by \(H(Y)\) gives \(\tilde{R}^* \geq R^{*}\). Equality holds if and only if \(I(\tilde{C}; Y \mid X) = 0\), i.e., \(\tilde{C} \perp Y \mid X\). This occurs when \(\tilde{C}\) is a deterministic function of \(X\) or otherwise carries no information about \(Y\) beyond what \(X\) already provides.

Part (ii). We show \(I(\tilde{C}; Y \mid X) \leq I(C_r; Y \mid X) = \delta_r \cdot H(Y)\). Recall the decomposition \(C = (C_r, C_u)\) with \(Y = g(X, C_r, C_u)\). The auxiliary context \(\tilde{C}\) is informative about \(Y\) through its dependence on \((X, C)\). We bound its contribution by decomposing: \[I(\tilde{C}; Y \mid X) = I(\tilde{C}; Y, C_r \mid X) - I(\tilde{C}; C_r \mid X, Y) \leq I(\tilde{C}; Y, C_r \mid X).\] Since \(\tilde{C}\) provides information about \(Y\) only through its relationship with the latent context, we have \[I(\tilde{C}; Y \mid X) \leq I(C_r; Y \mid X) = \delta_r \cdot H(Y),\] where the inequality follows because \(\tilde{C}\) can at most reveal all of \(C_r\) (in which case \(I(\tilde{C}; Y \mid X) = I(C_r; Y \mid X)\)) but cannot reduce the subjective gap: \(I(C_u; Y \mid X, C_r, \tilde{C}) = I(C_u; Y \mid X, C_r)\), which holds by the assumption \(\tilde{C} \perp C_u \mid (X, C_r)\). Dividing by \(H(Y)\): \[\tilde{R}^* - R^{*}= \frac{I(\tilde{C}; Y \mid X)}{H(Y)} \leq \delta_r.\]

Part (iii). Let \(\tilde{C}_1, \ldots, \tilde{C}_n\) be a sequence of auxiliary variables. Define \(\tilde{R}^*_k = I(X, \tilde{C}_1, \ldots, \tilde{C}_k; Y)/H(Y)\). By the chain rule, \[\tilde{R}^*_{k+1} - \tilde{R}^*_k = \frac{I(\tilde{C}_{k+1}; Y \mid X, \tilde{C}_1, \ldots, \tilde{C}_k)}{H(Y)} \geq 0,\] so the sequence \(\{\tilde{R}^*_k\}\) is nondecreasing. It is bounded above by \(I(X, C_r; Y)/H(Y) = 1 - \delta_u\) from part (ii), since the cumulative information from all auxiliary variables cannot exceed the total resolvable information \(I(C_r; Y \mid X)\). When \(\tilde{C}\) collectively resolves all of \(C_r\)—i.e., \(C_r\) is a measurable function of \((X, \tilde{C}_1, \ldots, \tilde{C}_n)\)—then \(I(X, \tilde{C}; Y) = I(X, C_r; Y)\) and \[\tilde{R}^*_{\max} = \frac{I(X, C_r; Y)}{H(Y)} = 1 - \delta_u.\] This is exactly \(1\) when \(\delta_u = 0\) (ideally verifiable tasks) and strictly less than \(1\) when \(\delta_u > 0\) (unverifiable tasks). ◻

Proof. Part (i): The verified model generates \(\hat{y}_t^{\mathrm{ver}}\) conditioned on \((\hat{y}_{<t}^{\mathrm{ver}}, X, v_1, \ldots, v_{t-1})\). The oracle feedback \(v_{<t}\) provides information about the true prefix \(y_{<t}\), enabling the model to correct errors in \(\hat{y}_{<t}\). Formally, the verified prefix satisfies \(d(\hat{y}_{<t}^{\mathrm{ver}}, y_{<t}) \leq d(\hat{y}_{<t}, y_{<t})\) whenever the model uses the oracle feedback to bring its prefix closer to the target (which any rational correction strategy achieves). Since \(\omega\) is nondecreasing (Assumption 1), \(\Delta_t^{\mathrm{ver}} = \omega(d(\hat{y}_{<t}^{\mathrm{ver}}, y_{<t})) \leq \omega(d(\hat{y}_{<t}, y_{<t})) = \Delta_t\).

Part (ii): By the triangle inequality for \(d\) and the monotonicity of \(\omega\), \[\Delta_t - \Delta_t^{\mathrm{ver}} \;=\; \omega(d(\hat{y}_{<t}, y_{<t})) - \omega(d(\hat{y}_{<t}^{\mathrm{ver}}, y_{<t})) \;\leq\; \omega(d(\hat{y}_{<t}, \hat{y}_{<t}^{\mathrm{ver}})),\] where the last step uses the subadditivity of \(\omega\) (which holds when \(\omega\) is concave, a mild regularity condition). The bound says the drift reduction at each step is limited by how much the oracle-guided correction actually changes the prefix.

Part (iii): Substituting \(\Delta_t^{\mathrm{ver}} \leq \Delta_t\) into the degradation bound of Proposition 2(iii): \[\exp\Bigl(-c \sum_t \Delta_t^{\mathrm{ver}}\Bigr) \;\geq\; \exp\Bigl(-c \sum_t \Delta_t\Bigr),\] since \(\exp(-cx)\) is decreasing. The verified model degrades more slowly. ◻

Proof. For each instance \(m\), model \(j\) produces a single output \(\hat{y}_j^{(m)}\) and receives a score \(s_j^{(m)} \in [0,1]\) from the evaluation protocol. When \(R^* = 1\), the correct output is unique and the score is deterministic given the model’s capability, so \(\mathrm{V}(s_j^{(m)}) = 0\). When \(R^* < 1\), the conditional entropy \(H(Y \mid X) = (1 - R^*) \cdot H(Y) > 0\) implies that the target distribution \(P(Y \mid X)\) has nonzero spread: multiple outputs have positive probability for the same input. A model sampling from its learned distribution produces different outputs across runs, and a scoring function that compares against any fixed reference inherits this variation.

To lower-bound the score variance, note that the scoring function \(s = g(\hat{y}, y_{\mathrm{ref}})\) compares the model output against a reference. When \(H(Y \mid X) > 0\), the model’s output distribution has support on multiple tokens at each step. Across independent evaluation runs, the sampled output \(\hat{y}_j^{(m)}\) varies, and because the scoring function is non-constant on the support (any reasonable metric distinguishes at least some pairs of valid outputs), the score variance satisfies \[\label{eq:score-var} \mathrm{V}(s_j^{(m)}) \;\geq\; c'(1 - R^*),\tag{18}\] for a constant \(c' > 0\) depending on the scoring function’s sensitivity and \(H(Y)\).

The benchmark score for model \(j\) is \(\bar{s}_j = M^{-1} \sum_{m=1}^{M} s_j^{(m)}\). By independence across instances, \(\mathrm{V}(\bar{s}_j) \geq c'(1 - R^*)/M\). Write \(\bar{s}_j = \mu_j + \epsilon_j\), where \(\mu_j\) is the true expected score and \(\epsilon_j\) is zero-mean evaluation noise with variance \(\sigma_\epsilon^2 \geq c'(1 - R^*)/M\).

By the classical attenuation formula [48], the expected Pearson correlation between two independent evaluation runs is \[\mathbb{E}[r] \;=\; \frac{\sigma_\mu^2}{\sigma_\mu^2 + \sigma_\epsilon^2} \;=\; 1 - \frac{\sigma_\epsilon^2}{\sigma_\mu^2 + \sigma_\epsilon^2} \;\leq\; 1 - \frac{c(1 - R^*)}{M},\] where \(\sigma_\mu^2 = \mathrm{Var}(\mu_j)\) is the variance of true model qualities across the set of LLMs, and \(c = c'/(\sigma_\mu^2 + c'/M)\).

The Spearman rank correlation \(\rho\) is the Pearson correlation of the ranks. For continuous score distributions, \(\rho\) and \(r\) coincide in expectation; for discrete scores (as in practice), the attenuation formula for \(\rho\) is an approximation that becomes exact as the number of models grows. We use it here as a standard proxy, following [48]. ◻

Proof. During pretraining, the model serves the task distribution \(\mathcal{T}\), and Theorem 5 gives the capacity exponent \(\alpha = (\bar{\mu}-1)/(d(\nu_{\mathcal{T}}+1))\). The model’s \(M(N)\) representable modes are distributed across the eigenmodes of the mixture kernel \(\bar{K} = \mathbb{E}_{\tau \sim \mathcal{T}}[K_Y^\tau]\), with capacity allocated in proportion to the eigenvalues \(\{\bar{\sigma}_k\}\).

Fine-tuning on \(\tau^*\) reoptimizes the model’s parameters under the loss for \(\tau^*\) alone. This does not discard the pretrained spectrum but tilts it toward \(K_Y^{\tau^*}\): modes aligned with \(\tau^*\) are amplified while modes orthogonal to it are attenuated. The effective kernel \(\bar{K}_{\mathrm{ft}}\) becomes progressively more aligned with \(K_Y^{\tau^*}\) as fine-tuning proceeds, approaching \(K_Y^{\tau^*}\) in the limit of full convergence. This tilt has three effects on the spectral structure.

Effect 1: Mode amplification and suppression. Let \(\{(\bar{\sigma}_k, \bar{\phi}_k)\}_{k \geq 1}\) be the eigendecomposition of the pretrained mixture kernel \(\bar{K}\), and let \(a_k^{\tau^*} = \langle K_Y^{\tau^*} \bar{\phi}_k, \bar{\phi}_k \rangle \geq 0\) be the projection of the dependency kernel of \(\tau^*\) onto the \(k\)-th pretrained eigenmode. The pretrained model allocates capacity to mode \(k\) in proportion to \(\bar{\sigma}_k\). We show that fine-tuning on \(\tau^*\) reallocates capacity in proportion to \(a_k^{\tau^*}\).

The cross-entropy loss for task \(\tau^*\) decomposes in the eigenbasis of \(\bar{K}\) as \[\begin{align} \mathcal{L}_{\tau^*}(\theta) = H(Y|X) + \sum_{k \geq 1} a_k^{\tau^*} \epsilon_k(\theta)^2, \label{eq:ce95ft} \end{align}\tag{19}\] where \(\epsilon_k(\theta)\) is the model’s approximation error along mode \(k\), and the weight \(a_k^{\tau^*}\) determines how much each mode contributes to the loss on \(\tau^*\).

Since the terms in the sum in 19 are independent, representing mode \(k\) reduces the loss by \(a_k^{\tau^*} \epsilon_k^2\) regardless of which other modes are represented. Thus, a model with capacity to represent \(M\) modes minimizes \(\mathcal{L}_{\tau^*}\) by selecting the \(M\) modes with the largest \(a_k^{\tau^*}\), setting their \(\epsilon_k=0\) and leaving the rest at their default error. Let \(S_{\mathrm{pre}} = \{k : \bar{\sigma}_k \text{ is among the top } M\}\) and \(S_{\mathrm{ft}} = \{k : a_k^{\tau^*} \text{ is among the top } M\}\) be the mode sets selected by the pretrained and fine-tuned models respectively. Then:

• Modes in \(S_{\mathrm{ft}} \setminus S_{\mathrm{pre}}\) are amplified: they were not represented by the pretrained model (because \(\bar{\sigma}_k\) was too small to justify inclusion in the mixture) but are now represented because \(a_k^{\tau^*}\) is large.

• Modes in \(S_{\mathrm{pre}} \setminus S_{\mathrm{ft}}\) are suppressed: they were represented by the pretrained model (because \(\bar{\sigma}_k\) was large in the mixture) but are now dropped because \(a_k^{\tau^*}\) is small.

• Modes in \(S_{\mathrm{pre}} \cap S_{\mathrm{ft}}\) are retained: they are important under both the mixture and \(\tau^*\).

The number of suppressed modes \(|S_{\mathrm{pre}} \setminus S_{\mathrm{ft}}|\) equals the number of amplified modes \(|S_{\mathrm{ft}} \setminus S_{\mathrm{pre}}|\), since both sets have cardinality \(M\).

12.4.0.1 Effect 2: Catastrophic forgetting.

We quantify the performance degradation on a task \(\tau \neq \tau^*\) after fine-tuning on \(\tau^*\). Recall that the pretrained and finetuned models represents modes \(S_{\mathrm{pre}}, S_\mathrm{ft}\) respectively, with \[\begin{align} S_{\mathrm{pre}} &= \{k : \bar{\sigma}_k \text{ is among the top } M\},\\ S_{\mathrm{ft}} &= \{k : a_k^{\tau^*} \text{ is among the top } M\}. \end{align}\] The loss on task \(\tau\) under a model representing mode set \(S\) is \(\mathcal{L}_\tau(S) = H_\tau(Y|X) + \sum_{k \notin S} a_k^{\tau} \, \epsilon_k^2\), where \(a_k^\tau = \langle K_Y^\tau \bar{\phi}_k, \bar{\phi}_k \rangle\) is the projection of \(\tau\)’s kernel onto mode \(k\), and \(\epsilon_k\) is the default (unrepresented) error for mode \(k\). The change in loss on \(\tau\) due to fine-tuning is \[\begin{align} \mathcal{L}_\tau(S_{\mathrm{ft}}) - \mathcal{L}_\tau(S_{\mathrm{pre}}) &= \sum_{k \notin S_{\mathrm{ft}}} a_k^\tau \, \epsilon_k^2 - \sum_{k \notin S_{\mathrm{pre}}} a_k^\tau \, \epsilon_k^2 \nonumber \\ &= \underbrace{\sum_{k \in S_{\mathrm{pre}} \setminus S_{\mathrm{ft}}} a_k^\tau \, \epsilon_k^2}_{\text{loss from suppressed modes}} - \underbrace{\sum_{k \in S_{\mathrm{ft}} \setminus S_{\mathrm{pre}}} a_k^\tau \, \epsilon_k^2}_{\text{gain from amplified modes}}. \label{eq:forgetting95decomp} \end{align}\tag{20}\] Catastrophic forgetting on a task \(\tau\) occurs when the first sum exceeds the second, i.e., the modes lost were more important for \(\tau\) than the modes gained.

The alignment \(\rho(\tau, \tau^*)\) controls which regime holds. Denote by \(S_{\mathrm{pre}} \setminus S_{\mathrm{ft}} := \{k : \bar{\sigma}_k \text{ large but } a_k^{\tau^*} \text{ small}\}\) the modes important for the mixture but not for \(\tau^*\). Whether they are important for \(\tau\) depends on \(\rho(\tau, \tau^*)\):

• High alignment (\(\rho(\tau, \tau^*) \approx 1\)): The kernels \(K_Y^\tau\) and \(K_Y^{\tau^*}\) share eigenmodes, so \(a_k^\tau\) is large when \(a_k^{\tau^*}\) is large and small when \(a_k^{\tau^*}\) is small. The suppressed modes (small \(a_k^{\tau^*}\)) also have small \(a_k^\tau\), so the first sum in 20 is small. Conversely, the amplified modes (large \(a_k^{\tau^*}\)) also have large \(a_k^\tau\), so the second sum is large. The net effect is \(\mathcal{L}_\tau(S_{\mathrm{ft}}) - \mathcal{L}_\tau(S_{\mathrm{pre}}) \approx 0\), so performance on task \(\tau\) is largely preserved.

• Low alignment (\(\rho(\tau, \tau^*) \approx 0\)): The kernels \(K_Y^\tau\) and \(K_Y^{\tau^*}\) have approximately orthogonal eigenmodes. The suppressed modes (small \(a_k^{\tau^*}\)) may have large \(a_k^\tau\) as they were serving \(\tau\) through the mixture but are irrelevant to \(\tau^*\). The amplified modes (large \(a_k^{\tau^*}\)) have small \(a_k^\tau\), so the gain term is negligible. The net effect is \(\mathcal{L}_\tau(S_{\mathrm{ft}}) - \mathcal{L}_\tau(S_{\mathrm{pre}}) \approx \sum_{k \in S_{\mathrm{pre}} \setminus S_{\mathrm{ft}}} a_k^\tau \, \epsilon_k^2 > 0\), so performance on task \(\tau\) degrades substantially.

To make this precise, we bound the spectral mass of \(\tau\) on the suppressed modes. We work in the eigenbasis \(\{\bar{\phi}_k\}\) of the mixture kernel \(\bar{K}\), and define \(a_k^\tau = \langle K_Y^\tau \bar{\phi}_k, \bar{\phi}_k \rangle \geq 0\).

The decomposition \(\|K_Y^\tau\|_F^2 = \sum_k (a_k^\tau)^2\) and \(\mathrm{tr}(K_Y^\tau K_Y^{\tau^*}) = \sum_k a_k^\tau a_k^{\tau^*}\) hold exactly when \(K_Y^\tau\) and \(K_Y^{\tau^*}\) are simultaneously diagonalizable in the mixture eigenbasis—i.e., when each task kernel’s eigenmodes are well-approximated by the shared basis \(\{\bar{\phi}_k\}\). This is a natural consequence of the task-mixture structure: the mixture kernel \(\bar{K}\) is designed to span the dominant eigenmodes of all tasks in \(\mathcal{T}\), so each task’s kernel is well-represented in this basis up to residual cross-terms. We proceed under this approximation, noting that the residual cross-terms contribute lower-order corrections to the bound below.

Partition the full set of modes into \(S_{\mathrm{ft}}\) (retained by the fine-tuned model) and its complement \(S_{\mathrm{ft}}^c\). The total spectral mass of \(\tau\) decomposes as \[\|K_Y^\tau\|_F^2 \;=\; \sum_{k \in S_{\mathrm{ft}}} (a_k^\tau)^2 + \sum_{k \in S_{\mathrm{ft}}^c} (a_k^\tau)^2.\] We bound the retained mass using the alignment. By Cauchy–Schwarz, \[\sum_{k \in S_{\mathrm{ft}}} a_k^\tau\, a_k^{\tau^*} \;\leq\; \Bigl(\sum_{k \in S_{\mathrm{ft}}} (a_k^\tau)^2\Bigr)^{1/2} \Bigl(\sum_{k \in S_{\mathrm{ft}}} (a_k^{\tau^*})^2\Bigr)^{1/2}.\] Since \(S_{\mathrm{ft}}\) contains the top-\(M\) modes by \(a_k^{\tau^*}\), the second factor satisfies \(\sum_{k \in S_{\mathrm{ft}}} (a_k^{\tau^*})^2 \leq \|K_Y^{\tau^*}\|_F^2\).

For the left-hand side, we decompose the full inner product: \[\mathrm{tr}(K_Y^\tau K_Y^{\tau^*}) \;=\; \sum_k a_k^\tau\, a_k^{\tau^*} \;=\; \sum_{k \in S_{\mathrm{ft}}} a_k^\tau\, a_k^{\tau^*} \;+\; \sum_{k \notin S_{\mathrm{ft}}} a_k^\tau\, a_k^{\tau^*}.\] The second sum runs over modes \(k \notin S_{\mathrm{ft}}\), which by construction have the smallest values of \(a_k^{\tau^*}\) (since \(S_{\mathrm{ft}}\) selected the top-\(M\)). Let \(a_{\max}^{\tau^*}(S_{\mathrm{ft}}^c) := \max_{k \notin S_{\mathrm{ft}}} a_k^{\tau^*}\) denote the largest \(\tau^*\)-projection outside the retained set. Then \[\sum_{k \notin S_{\mathrm{ft}}} a_k^\tau\, a_k^{\tau^*} \;\leq\; a_{\max}^{\tau^*}(S_{\mathrm{ft}}^c) \sum_{k \notin S_{\mathrm{ft}}} a_k^\tau.\] Under the power-law spectrum (Assumption 4), the projections satisfy \(a_k^{\tau^*} \sim k^{-\nu_{\tau^*}}\), so \(a_{\max}^{\tau^*}(S_{\mathrm{ft}}^c) = a_{M+1}^{\tau^*} \sim M^{-\nu_{\tau^*}} \to 0\) as \(M \to \infty\). The remainder is therefore \(O(M^{-\nu_{\tau^*}})\), which is negligible for any model with enough capacity to meaningfully fine-tune on \(\tau^*\). Concretely: \[\label{eq:subset-lb} \sum_{k \in S_{\mathrm{ft}}} a_k^\tau\, a_k^{\tau^*} \;\geq\; \rho(\tau, \tau^*)\|K_Y^\tau\|_F \|K_Y^{\tau^*}\|_F \;-\; a_{\max}^{\tau^*}(S_{\mathrm{ft}}^c) \sum_{k \notin S_{\mathrm{ft}}} a_k^\tau.\tag{21}\] In the regime where the remainder is negligible (i.e., \(M\) is large relative to the effective rank of \(K_Y^{\tau^*}\)), substituting 21 into the Cauchy–Schwarz bound and rearranging gives \[\sum_{k \in S_{\mathrm{ft}}} (a_k^\tau)^2 \;\geq\; \rho(\tau, \tau^*)^2 \|K_Y^\tau\|_F^2 \;-\; O\bigl(a_{\max}^{\tau^*}(S_{\mathrm{ft}}^c)\bigr).\] Therefore the lost mass satisfies \[\sum_{k \in S_{\mathrm{ft}}^c} (a_k^\tau)^2 \;\leq\; (1 - \rho(\tau, \tau^*)^2)\,\|K_Y^\tau\|_F^2 \;+\; O\bigl(a_{\max}^{\tau^*}(S_{\mathrm{ft}}^c)\bigr).\] The forgetting bound follows as before: \[\mathcal{L}^\tau(S_{\mathrm{ft}}) - \mathcal{L}^\tau(S_{\mathrm{pre}}) \;\leq\; \|\epsilon\|_\infty^2 \sqrt{M} \cdot \sqrt{1 - \rho(\tau, \tau^*)^2 + O(a_{\max}^{\tau^*}(S_{\mathrm{ft}}^c))} \cdot \|K_Y^\tau\|_F.\] Since \(a_{\max}^{\tau^*}(S_{\mathrm{ft}}^c) = O(M^{-\nu_{\tau^*}})\) under the power-law spectrum, this reduces to the stated bound with forgetting scaling as \(\sqrt{1 - \rho(\tau,\tau^*)^2}\), up to a correction that vanishes polynomially in \(M\). ◻