1 Introduction↩︎

Quantizing a transformer to low bit-widths is the dominant route to cheap inference, and the formats now being deployed quantize both weights and activations to 4 bits. What remains a design choice is mixed precision: which layers (or tensors) to keep at higher precision under a memory budget. The choice is consequential, since allocations at the same memory budget can differ in output fidelity by tens of percent (Section 7), and it is currently made by per-layer sensitivity heuristics.

These heuristics share a structural premise. Each scores a layer by a sensitivity measured largely in isolation (i.e. with other layers unquantized), such as a Hessian eigenvalue or trace [1], [2], a squared-gradient Fisher score [3], or at most a pairwise interaction term [4], and then aggregates these scores to rank layers or to estimate the loss of a joint configuration. To our knowledge, to what extent the loss of quantizing a set of layers is in fact recoverable from such low-order measurements has never been tested directly. Throughout, the order of a term is the number of layers it involves jointly, so per-layer terms are first-order and pairwise terms second-order. The loss from quantization is a set function defined on all \(2^L\) subsets of \(L\) layers (equivalently, the boolean cube \(\{0,1\}^L\)), which cannot be enumerated for models of any realistic depth.

To measure the order of this function directly, we write \(f(S) = L_{\mathrm{CE}}(S\;\text{quantized}) - L_{\mathrm{CE}}(\text{full precision})\) for the damage of quantizing the layer set \(S\), obtained by evaluating the partially quantized model, and analyze the measured \(f\) through two classical changes of basis. The Walsh transform (the functional-ANOVA decomposition) splits the variance of \(f\) by order under the deployment distribution, the distribution in which each layer is quantized independently with probability \(p\). The Möbius expansion reconstructs \(f\) from signed terms of increasing order. This analysis yields two findings.

The first finding is that the variance of the damage is almost entirely first-order. Across configurations drawn from the deployment distribution, 85–93% of the variance of \(f\) is explained by per-layer effects, rising to 98–99% on exact lattices (blocks of \(L\) layers for which every one of the \(2^L\) configurations is measured) once measurement noise is corrected for (Fig. 1, Section 3). The unexplained remainder therefore serves as a certificate: a single number, measured on full lattices (an identity, by Theorem 1) or forecast in closed form from fitted break-rates at full-network scale (Section 4.3), that states in advance how well any additive model can do, and which we report alongside every result. The additive model attaining this error floor also has a closed form (Section 5).

The second finding is that the variance of \(f\) explained by higher-order terms does not reflect a failure of per-layer structure. The order-\(\ge 2\) share, the fraction of the variance that per-layer effects leave unexplained, equals by Theorem 1 the mean-squared error of the best additive fit divided by the variance (Section 3). This share can be nonzero even when the damage is perfectly described by per-layer terms. If \(f\) is additive after a strictly increasing transformation, that is, it has the form \(g\bigl(\sum_{i\in S} b_i\bigr)\) for per-layer contributions \(b_i\) and a strictly increasing \(g\), then composing with \(g^{-1}\) yields a function whose order-\(\ge 2\) energy is exactly zero. Whether \(f\) belongs to this class is testable on rankings alone (Theorem 2), and we measure the smallest fraction of pairwise configuration comparisons that any additive index must order incorrectly, certified against measurement noise (Section 3.1). This fraction is at most 2%, on some models indistinguishable from zero, and a direct search over transformations agrees. The measured damage is therefore of the form \(f(S) \approx g\bigl(\sum_{i\in S} b_i\bigr)\). This model class recurs throughout the paper. It justifies the form fitted in Section 4, and it caps how much any interaction-aware allocator can improve on additive configuration scoring (Section 7).

We show that a single property, saturation, explains both findings exactly. Because the loss is bounded, one can define the headroom \(h(S) = c - f(S)\), the damage still available below a ceiling \(c\). The measurements are well described by a model in which each quantized layer consumes a fixed fraction \(a_i\) of the remaining headroom, \(h(S\cup\{i\}) = (1-a_i)\,h(S)\), independently of which layers preceded it. Equivalently, \(f(S)=c\bigl(1-\prod_{i\in S}(1-a_i)\bigr)\), which we call the coverage model. The model is fitted to the measured losses of sampled configurations by least squares.

The model class of the second finding is not arbitrary. By a representation theorem of Aczél and Ling [5], [6], any rule that accumulates damage one layer at a time through a fixed operation that is associative, continuous, and strictly increasing can be rewritten in the form \(g\bigl(\sum_i b_i\bigr)\) (App. 11).

Aggregating isolated sensitivities is equivalent to fitting an additive model constrained to predict zero damage at the unquantized network, and this constraint is what fails in the incumbent heuristics: freeing the single intercept parameter raises \(R^2\) from \(0.13\) to \(0.94\) on the exact lattice where FP4 damage is large (Section 3). Because saturation shrinks a layer’s marginal damage as more layers are quantized, a sensitivity measured with every other layer at full precision is inflated by a closed-form factor, the quantitative form of a context dependence known qualitatively [7]–[9], and summing \(L\) inflated coefficients compounds the error to 71–376%, which we measure for isolate-and-sum (i.e. the quantization loss is the sum of the damages from quantizing each layer separately) and for a Hessian-trace HAWQ-v2 (Section 5). A single Aumann–Shapley path integral recovers the additive model’s optimal coefficient evaluated at a layer-specific effective deployment density, and with it the exact ranking of layers, with no sampled configurations, a recovery we prove under the coverage model (Section 6, App. 12).

Because both models predict rather than rank, they can price a configuration, that is, quote its loss before committing to it, which a ranking cannot do. On unseen configurations that protect the most fragile layers and quantize the rest, the quoted loss is accurate to within 4% (App. 15). Against the black-box surrogates used for the same task, random forests [10], Gaussian processes [11], and RBF kernels [12], the \((L{+}1)\)-parameter models are best or tied at small measurement budgets, the regime that matters when every measurement is a forward pass. The strongest of these, the Gaussian process, succeeds for a reason our analysis identifies and fails where the theory predicts, degrading \(2\)–\(5\times\) under extrapolation to differently quantized configurations than those it was fit on, where the additive and coverage models stay within \(2\times\). That direction of failure matters because an allocator must price configurations quantized at various levels (Section 7).

Used as allocators at matched memory, the same fitted models attain the lowest KL divergence among the compared allocators (sensitivity scores, structural rules, and the black-box surrogates converted into allocators) on mixture-of-experts language models across models from 0.6B to 355B parameters (Section 7). Below four bits configurations allocated from the fitted models continue to solve code and reasoning tasks at effective bit-widths where configurations from other methods of the same memory budget no longer produce terminating generations, at both 30B and 235B (Section 7.4).

1.0.0.1 Contributions.

1. A closed-form theory: the coverage model’s exact Möbius and Walsh spectra (Theorems 3 and 4), corroborated by the confirmed divergence of the signed expansion (Fig. 2); the certificate identity (Theorem 1) with an exact \(O(L)\) heterogeneous form (Proposition 5); and the ordinal characterization of additivity up to an increasing transformation (Theorem 2) (Sections 3–4, App. 16).

2. A negative result with a closed-form account: sensitivities measured in isolation, including HAWQ-v2, mis-price a configuration’s loss by 71–376%, and the inflation factor responsible is derived in closed form (Section 5).

3. A fit-free estimator: one Aumann–Shapley path integral recovers the additive model’s optimal coefficient at a layer-specific effective density, and its layer ranking exactly, proven under the coverage model (Section 6, App. 12).

4. An interpretable predictor that is best or tied against RF/GP/RBF/pairwise baselines at small budgets, degrades at most \(2\times\) under configuration extrapolation where the GP degrades \(2\)–\(5\times\), and is validated on LLMs 0.6B–355B and a diffusion DiT (Section 6, Fig. 4).

5. An allocator at matched memory that attains the lowest KL among sensitivity-, rule-, and surrogate-based allocators (17–21% below the production ModelOpt baseline on Qwen3-30B, 13–27% on Qwen3-235B, 20–37% on GLM-4.6) and decides served task survival below four bits.

2 Related work↩︎

2.0.0.1 Mixed-precision allocation via per-layer sensitivity.

AWQ [13] uses activation-aware scaling, HAWQ and HAWQ-v2 [1], [2] a Hessian eigenvalue or trace, BRECQ [14] and GPTQ [15] second-order weight reconstruction, and NVIDIA ModelOpt’s auto_quantize [3], the production method we benchmark, a Taylor/Fisher score combined with a linear program over a sum of isolated per-layer scores. All of these aggregate first- or second-order single-layer signals. In our deployment regime the closest neighbor is FGMP [16], which selects FP8-versus-NVFP4 blocks for weights and activations by a Fisher-weighted perturbation score. Its score belongs to the same Taylor/Fisher family whose production embodiment, ModelOpt, we benchmark throughout, while its intra-tensor block granularity requires co-designed kernels that no serving stack we target provides, so it composes with layer-level allocation rather than competing with it. ScaleBITS [17] performs hardware-aligned bitwidth allocation for weights only.

2.0.0.2 Interaction-aware allocation.

CoopQ [9], first circulated under the name IMPQ, is the closest neighbor among interaction-aware methods. Its SPQE estimator samples layer permutations and progressively quantizes along each, thereby measuring marginal contributions in context (permutation-sampled Shapley, at a cost of roughly \(100L\) quantized evaluations), then fits a linear-plus-pairwise surrogate (\(a^\top q + q^\top K q\)) and allocates by solving a binary quadratic program via MILP linearization. CLADO [4] measures pairwise cross-layer effects by joint-quantization finite differences on CNNs and ViTs and rejects the independence assumption. Both fix the expansion order at two and do not test whether that order suffices. GuidedQuant [18] keeps within-output-channel dependencies but ignores cross-layer terms, and QEP [19] models cross-layer accumulation as a compensation term. Sections 3–4 supply what none of these test, namely the order of the expansion, and Section 5 gives the closed-form account of the isolated-anchor bias they implicitly work around.

2.0.0.3 Configuration-quality surrogates.

For the prediction task itself, MiCo [10] fits a random-forest accuracy predictor for edge-scale networks, AMQ [12] an RBF kernel model for LLM weight-only quantization, and AutoQRA [11] a Gaussian-process surrogate inside a joint search over bit-widths and adapter ranks. These predictor classes are accurate in-distribution but opaque, and we show they degrade \(2\)–\(5\times\) when extrapolating to more heavily quantized configurations (Section 6).

2.0.0.4

Additional related works are surveyed in App. 18.

3 The measured structure of quantization damage↩︎

This section reports the two measurements on which the rest of the paper builds: the decomposition of the damage variance by order, and the ordinal defect, which measures how much of that structure survives every increasing transformation of the loss.

3.0.0.1 Set-up.

For a contiguous block of \(L\) layers (finer units are quantized in Section 7), centered at the network’s mid-depth (App. 14), we measure \(f(S)=\Delta L_{\mathrm{CE}}\) for the full subset lattice of \(2^L\) subsets \(S\), running the model with exactly the layers in \(S\) quantized. The quantizer rounds stochastically, so each \(f(S)\) is the average of several independent rounding draws (16 for the lattices in this section; App. 14). A measure over configurations must be fixed before variances or prediction errors can be defined, and it should weigh the configurations that matter in practice, which are mostly quantized. We therefore analyze \(f\) under the deployment distribution \(\mu_p\), the product measure in which each layer is quantized independently with probability \(p\), and take \(p=0.6\) throughout unless stated otherwise. We expand the measured \(f\) in the Walsh basis \(\{\chi_T\}\), with coefficients \(\hat{f}(T)\) [20]. We denote the order-\(k\) energy \(W_k = \sum_{|T|=k} \hat{f}(T)^2\), and by Parseval’s identity, \(\mathop{\mathrm{Var}}(f)=\sum_{k\ge 1}W_k\), so the ratio \(W_k/\mathop{\mathrm{Var}}(f)\) is the fraction of the loss variance attributable to order-\(k\) structure. The following identity gives these energies their operational meaning.

For any \(f\) on \(\{0,1\}^L\) and any \(p\in(0,1)\), the best degree-\(\le 1\) predictor \(f_1\) under \(\mu_p\) has squared error exactly equal to the high-order Walsh energy: \(\mathbb{E}_{\mu_p}\bigl[(f-f_1)^2\bigr] = \sum_{k\ge 2} W_k\).

This is the \(d=1\) case of low-degree learning [20]–[22].

3.0.0.2 Result (Fig. 1 (a)).

Roughly 90% of the variance is order-1, with order-\(\ge 2\) contributing 7–15% across models and datasets. On a 10-layer Qwen3-8B block, order-1 accounts for \(87.3\%\) of the variance as measured and \(98.1\%\) once the noise floor contributed by the stochastic-rounding draws is subtracted (the split-half estimate of App. 11.1), and on an 8-layer Qwen3-0.6B block for \(93.8\%\) and \(99.2\%\). Across \(\{\)Qwen3-0.6B, Qwen2.5-3B, Qwen3-4B, Qwen3-8B, Llama-3.2-3B, Qwen3-30B-A3B\(\}\times\{\)wiki, code\(\}\) (WikiText-2 and a code corpus, App. 14) the sampled order-1 share is 85–93% when whole layers are quantized at FP4. The concentration is not an artifact of whole-layer units or of small scale (Fig. 1 (a)): on exact \(2^8\) lattices over eight consecutive linear units at W4A4, the order-1 share is 97–99% on models from 0.6B to Qwen2.5-72B, and 90.7% on gpt-oss-120B.

3.0.0.3 The intercept diagnostic (Fig. 1 (b)).

An additive model with no intercept, which is the constraint that aggregating isolated sensitivities imposes (\(f(\emptyset)=0\)), reaches \(R^2=0.13\) on the 0.6B block, where FP4 damage is large, against \(0.78\) on the 8B block, where it is small. Relaxing this constraint raises both to \(0.94\) and \(0.87\). Both fits are least squares under \(\mu_p\) on the full lattice.

The remaining questions are why the small order-\(\ge 2\) content is structured the way it is and what generates it. The answer we propose is saturation (Section 4). Before turning to it, we ask how much of the order-\(\ge 2\) energy reflects structure among layers at all.

3.1 The ordinal defect: additivity up to an increasing transformation↩︎

The order-\(\ge 2\) energy has two caveats as an indicator of structure among layers. First, it is measure-relative. Second, and more fundamentally, it is not invariant under a change of coordinates on the loss values, since replacing \(f\) by \(\psi \circ f\) for a strictly increasing \(\psi:\mathbb{R}\to\mathbb{R}\) can change the energy if \(\psi\) is curved, so an exactly additive \(f\) could acquire nonzero higher-order energy. A principled notion of irreducible structure should therefore quantify over all strictly increasing \(\psi\). For binary factors this is possible, because additivity up to an increasing transformation is a purely ordinal property.

\(f(x) = g(b\cdot x)\) for some \(b\in\mathbb{R}^L\) and strictly increasing \(g\) if and only if some additive index reproduces \(f\)’s ranking of subsets, that is, \(f(x) < f(y) \iff b\cdot x < b\cdot y\) for all \(x,y \in \{0,1\}^L\).

The theorem is the finite binary case of additive conjoint measurement [23], [24]. App. 11 gives the short proof. Throughout, an additive index is a linear score \(x \mapsto b\cdot x\) used only through the ordering it induces on configurations. Whether such an index exists is a linear program, with one inequality \(b\cdot(x-y)>0\) per ordered pair [25]. The smallest witnesses of infeasibility are rank flips, pairs of comparisons that order the same two layers oppositely in two contexts: \(f(S\cup i) > f(S\cup j)\) but \(f(T\cup i) < f(T\cup j)\) forces both \(b_i>b_j\) and \(b_i<b_j\), so a single true flip refutes every strictly increasing \(\psi\). We call the minimal fraction of ranking constraints that any additive index must violate the ordinal defect of \(f\).

3.1.0.1 Measurement.

On full \(2^8\) configuration lattices, measured with 16 stochastic-rounding draws per configuration (App. 14), we bound the defect from both sides, certifying each statistic against measurement noise. For the lower bound we count rank flips. Splitting the 16 draws into two halves gives two independent estimates of every configuration’s loss, and a flip is counted only if it is detected above the noise floor on the first half and recurs with the same sign on the second. The surviving count is then read against the count that the identical pipeline produces on a matched-noise null, a synthetic lattice constructed to contain no true flips (the coverage model of Section 4 fitted to the measured losses, plus noise at each configuration’s measured level). For the upper bound we exhibit an index, since the fraction of ranking constraints violated by any single index bounds the defect from above. The exhibited index minimizes the total slack of the constraints of one half of the draws, again a linear program, and its violated fraction is evaluated both on that half and on the held-out half’s constraints, with the larger of the two quoted (App. 11.1, per-model results in Table 7). In the deployed per-block regime the certified ordinal defect is at most \(1.4\%\) (the largest cell, Llama-3.2-3B). The energy counterpart agrees: minimizing the order-\(\ge 2\) share over all strictly increasing transformations of the loss (the estimator \(E^*\) of App. 11.1) yields an upper bound of \(0\)–\(5\%\) across the deployed cells (last column of Table 7). The zeros are not blindness of the procedure. Applied to the non-deployed per-tensor regime of Section 8, the same procedure finds flips well in excess of its matched-noise null (74 against 18, Table 7), so genuine reversals are detected where a known mechanism produces them, and even there an additive index orders at least \(98\%\) of the comparisons above the noise floor correctly. The measured \(f\) is therefore, to that accuracy, of the form \(g\bigl(\sum_{i\in S} b_i\bigr)\) for a strictly increasing \(g\), and the apparent high-order energy reflects the curvature of \(g\) rather than structure among layers.

4 The reconciling theory: saturation generates both views↩︎

4.1 The coverage model↩︎

Section 3 left two measurements in need of a common explanation: per-layer effects carry most of the variance of the measured damage \(f\), and some additive index reproduces at least \(98\%\) of its pairwise ranking of configurations. This section proposes saturation as the origin of both. We describe the damage with a saturating coverage model, \[f(S) = c\Bigl(1 - \prod_{i\in S}(1 - a_i)\Bigr), \label{eq:coverage}\tag{1}\] with a per-layer break-rate \(a_i\in[0,1]\) and a ceiling \(c>0\). Define the headroom \(h(S) := c - f(S)\), the remaining distance to the ceiling. Imposing the coverage model is exactly the statement that each quantized layer consumes a fixed fraction \(a_i\) of whatever headroom remains, \(h(S\cup\{i\}) = (1-a_i)\,h(S)\) regardless of which layers came before.

This model is grounded in a measured property of the damage. Saturation, as we use the term, is the conjunction of two facts visible on the configuration lattices of Section 3.1: the loss of a heavily quantized configuration is finite, and the marginal damage of one more quantized layer shrinks as more layers are already quantized. Table 8 reports the measurement for each of the five certification cells, where a cell is one model and quantizer setting measured as a full \(2^8\) lattice. Averaging \(f(S\cup\{i\}) - f(S)\) over layers \(i\) and contexts \(S \not\ni i\) of a fixed size, we see that every cell’s average marginal quantization damage is largest when no other layer is quantized and falls once other layers are quantized concurrently. The evidence for coverage is assembled with each claim’s standing in App. 17.

Equivalently, the coverage model is the statement that \(f\) is additive in log-headroom, \(-\log(1-f(S)/c) = \sum_{i\in S} b_i\) with \(b_i:=-\log(1-a_i)\), hence coverage lies in the class \(g\bigl(\sum_{i\in S} b_i\bigr)\) of Section 3.1 with generator \(g(u)=c(1-e^{-u})\). The measurements of Section 3.1 place \(f\) within a \({\sim}2\%\) ordinal defect of this class across measured models.

4.2 The Möbius and Walsh-Hadamard expansions↩︎

The coverage model was introduced to account for the variance and ranking measurements of Section 3, and it is fitted to measured loss values (Section 4.4). It makes a second prediction about a quantity that plays no role in that fit. The Möbius/Harsanyi transform gives signed coordinates \(\varphi(T)=\sum_{R\subseteq T}(-1)^{|T|-|R|}f(R)\) that reconstruct the damage as \(f(T) = \sum_{R \subseteq T} \varphi(R)\). Restricting the reconstruction sums to subsets of size \(\le 1\) is precisely the isolate-and-sum aggregation of Section 5, since \(\varphi(\emptyset) = f(\emptyset) = 0\) and \(\varphi(\{i\}) = f(\{i\})\). Restricting them to size \(\le 2\) adds exactly the pairwise finite differences \(\varphi(\{i,j\}) = f(\{i,j\}) - f(\{i\}) - f(\{j\})\) that CLADO [4] measures as its cross-layer sensitivities.

The Möbius transform of the coverage model is, exactly, \(\varphi(T) = c\,(-1)^{|T|+1}\prod_{i\in T} a_i\) for \(|T|\ge 1\) and \(\varphi(\emptyset)=0\).

Coverage predicts a distinct form for this expansion. Its coefficients are nonzero at every order and alternate in sign, so low-order truncations should overshoot and oscillate. The measured lattices show exactly this behavior (Fig. 2). On Qwen3-0.6B at FP4 (8-layer exact lattice) the partial sums alternate, converging only at \(k=L\). On a 10-layer Qwen3-8B block with the full \(2^{10}\)-configuration lattice the same divergence-then-cancellation appears at larger amplitude, swinging to \(\pm 4.8\) against a true loss of \(0.148\), an overshoot of up to \(32\times\). Because an order-\(k\) Möbius coefficient is an alternating sum of up to \(2^k\) measured values, the partial sums amplify measurement noise, so we certify the swing by the split-half device of App. 11.1: partial sums computed from two disjoint halves of the rounding draws independently reproduce the alternating overshoot (both swing beyond \(\pm 4\)), while the same sums computed from the half-difference table, which contains noise only, stay within \(\pm 1.2\).

Under the \(p\)-biased measure \(\mu_p\), the Walsh coefficients are, exactly, \[\hat{f}(T) = c\,(-1)^{|T|+1}\,(p(1-p))^{|T|/2}\Bigl(\prod_{i\in T} a_i\Bigr)\prod_{i\notin T}(1 - a_i p) \quad\text{for } T \ne \emptyset,\] and \(\hat{f}(\emptyset) = \mathbb{E}_{\mu_p}[f] = c\bigl(1 - \prod_{i}(1 - a_i p)\bigr)\).

Proofs of both transforms are in App. 16.

4.3 The saturation-depth law↩︎

By Theorem 1, the mean-squared error of the best additive predictor equals the order-\(\ge 2\) energy \(\sum_{k\ge 2} W_k\). Dividing by the variance turns it into the order-\(\ge 2\) share \[\mathrm{share}_{\ge 2}:= \frac{\sum_{k\ge 2} W_k}{\mathop{\mathrm{Var}}(f)},\] the fraction of the damage variance that per-layer effects leave unexplained, which is the normalized form in which we report the certificate. The certificate columns of the benchmark and allocation tables quote the share, which is comparable across cells (Sections 6 and 7), while statements in loss units use the absolute energies, namely the additive error floor of Theorem 1 and the sampling variance of the additive-coefficient estimator, which is the order-\(\ge 2\) energy plus noise (App. 14). On a measured lattice the share is computed exactly, but at full-network scale, where no \(2^L\) lattice can be measured, it must be forecast from the \(L+1\) fitted parameters of the coverage model. This subsection derives that forecast, and that of the higher-order Walsh energies, from Proposition 5 in closed form. For a break-rate profile \((a_1,\ldots,a_L)\) define the per-layer energy ratios and their aggregates \[\beta_i := \frac{p(1-p)\,a_i^2}{(1-a_ip)^2}, \qquad \tau^2 := \sum_i \beta_i, \qquad L_{\mathrm{eff}}:= \frac{\bigl(\sum_i\beta_i\bigr)^2}{\sum_i\beta_i^2} \in [1, L],\] with \(\tau\) the saturation depth and \(L_{\mathrm{eff}}\) the participation ratio of the profile, which equals \(L\) exactly when all \(\beta_i\) are equal and approaches \(1\) as one layer dominates. Let \(e_{k} := \sum_{|T|=k}\prod_{i\in T}\beta_i\) be the elementary symmetric polynomials in the \(\beta_i\).

Assume not all \(a_i\) are zero. Under the coverage model,

1. (product form) \(W_k = c^2\bigl[\prod_i(1-a_ip)^2\bigr]\,e_{k}\), hence computable in closed form from the \(L+1\) fitted parameters, which are themselves fit from \(\mathcal{O}(L)\) measured configurations, and, \[\mathrm{share}_{\ge 2}= 1 - \frac{\tau^2}{\prod_i(1+\beta_i) - 1}; \label{eq:hetshare}\tag{2}\]

2. (leading order) \(\mathrm{share}_{\ge 2}= \frac{L_{\mathrm{eff}}-1}{2L_{\mathrm{eff}}}\,\tau^2 + R\) with \(|R| \le \tfrac14\tau^4 e^{\tau^2}\), so to second order in the per-layer amplitudes heterogeneity enters the certificate only through the effective depth \(L_{\mathrm{eff}}\);

3. at fixed \(\tau^2\), \(\mathrm{share}_{\ge 2}\) is Schur-concave in \((\beta_1,\ldots,\beta_L)\), hence \[\mathrm{share}_{\ge 2}\;\le\; 1 - \frac{\tau^2}{(1+\tau^2/L)^L - 1} \;\le\; 1 - \frac{\tau^2}{e^{\tau^2}-1},\] with equality on the left exactly when all \(\beta_i\) are equal.

Combining Theorem 1 with part (ii), for small \(\tau\) the best additive predictor’s mean-squared error is at most \({\approx}\tfrac12\tau^2\,\mathop{\mathrm{Var}}(f)\), so even an exact coverage fit, whose error under the model is zero, can improve on additive prediction by no more than that. The symmetric case is stated as a corollary, with its binomial order spectrum, in App. 16.

4.4 Validation of the certificate↩︎

4.4.0.1 The \(\tau\)-collapse (Fig. 3).

We empirically test the saturation-depth law. Each cell contributes one point, its measured order-\(\ge 2\) share against its fitted \(\tau\), and Proposition 5(iii) requires any coverage-generated profile to fall on or below the parameter-free curve, with balanced break-rate profiles on it. We measured 36 full \(2^8\) lattices, three models (Qwen3-0.6B, Qwen2.5-3B, Llama-3.2-3B) crossed with twelve quantizer settings, integer round-to-nearest at 8/6/5/4/3 bits with per-channel or per-tensor scales and 4-bit floating point (FP4) with per-block or per-tensor scales (App. 14). For each cell we noise-correct the order energies (App. 11.1), fit the coverage model to the same lattice by nonlinear least squares (App. 14), and compare the measured order-\(\ge 2\) share with the fit. The measured share tracks the fitted \(\tau = (\sum_i\beta_i)^{1/2}\) at Spearman \(\rho = 0.958\), and the exact share of Proposition 5(i), computed from the fitted \((c,a_i)\), predicts the measured share at Pearson \(\rho = 0.984\). Ten of the 36 cells exceed the curve, by at most 8 percentage points. The largest exceedances are 8-bit cells whose absolute damage sits at the measurement noise floor, where the noise-corrected share is unstable, and Llama-3.2-3B cells, the one model with certified rank reversals (Table 7), so the exceedances are consistent with share noise plus the residual structure that coverage does not capture. Repeating the comparison on Qwen3-30B-A3B blocks (32 further cells, 8 block positions by 4 quantizer formats) gives Pearson \(r = 0.997\).

4.4.0.2 Out-of-sample validation.

A certificate measured on a full lattice uses only Theorem 1. When enumeration is intractable, we instead assume the coverage model and forecast the certificate from break-rates fitted on sampled configurations. We test the forecasting pipeline directly against held-out lattices, with per-cell results in Table 11 of App. 14. The forecast classifies the near-additive and heavily coupled regimes correctly (App. 14).

5 Why isolated sensitivity mis-prices a configuration: the wrong anchor↩︎

Throughout, a predictor’s anchor is the configuration, or distribution over configurations, at which its coefficients are measured: isolated sensitivities are anchored at the unquantized network, while the coefficients of a fitted additive model are anchored at the deployment distribution.

5.0.0.1 The measurement.

An estimate of a configuration’s loss built from per-layer sensitivities alone is the sum \(\sum_{i\in S} f(\{i\})\), which we call isolate-and-sum throughout. On the fully quantized configuration of the exact whole-layer FP4 lattices (the two lattices of Section 3 and the four certification cells of Table 7) this aggregation overstates the measured loss by 71–346%. The mis-pricing persists on full networks. Predicting \(f\) on configurations that protect a sampled subset of layers and quantize the rest (all 36 Qwen3-8B and 48 Qwen3-30B-A3B layers, protection sizes 1 to \(L/2\)), isolate-and-sum’s median relative error is 51% on Qwen3-8B and 203% on Qwen3-30B-A3B, while additive and coverage models fitted on 80 sampled configurations attain 2–3% on the same held-out protection sets. HAWQ-v2, scoring each layer by its Hessian trace (Hutchinson estimator) times its mean squared quantization perturbation [2], inherits the same anchor: summed over each of the two Section 3 lattices it overstates the fully quantized corner by 89% (Qwen3-8B) and 376% (Qwen3-0.6B), against isolate-and-sum’s 155% and 250% on the same corners.

5.0.0.2 Marginal scale.

The failure is caused by two distinct order-1 coefficients. The in-context slope, the coefficient of the optimal additive fit, is \(w_i = \mathbb{E}_p[f\mid x_i=1] - \mathbb{E}_p[f\mid x_i=0]\), the marginal effect of layer \(i\) averaged over deployment configurations. The isolated slope, the quantity that HAWQ-style scores approximate, is \(s_i = f(\{i\})\), the marginal at the unquantized network. Under coverage, \(s_i = c\,a_i\) while \(w_i = c\,a_i\prod_{j\ne i}(1-p a_j)\), so \[\frac{s_i}{w_i} = \frac{1}{\prod_{j\ne i}(1-p a_j)} \approx e^{\,p\sum_j a_j},\] a ratio that grows exponentially with \(p\sum_j a_j\). We validate the formula directly on the full \(2^8\) lattices, comparing the measured ratio \(s_i/w_i\) against the value the fitted coverage model predicts. The measured medians are 1.9 / 3.0 / 3.9 / 9.2 (Llama-3.2-3B / Qwen3-30B-A3B / Qwen3-8B / Qwen3-0.6B) against predicted 1.8 / 2.3 / 2.5 / 7.7, with more detail in App. 13. The two-slope identity is a one-line corollary of Theorems 1 and 4 (App. 16).

6 An interpretable predictor, in and out of distribution↩︎

Two predictors are fit throughout: the coverage model, and the additive model. Each has \(L+1\) parameters, fit by least squares in loss space on \(N \approx L\) measured configurations. The choice of estimator (joint regression rather than per-coefficient sampling), its consistency under misspecification, and the cost comparison with the \({\sim}100L\) progressive-quantization evaluations of CoopQ’s SPQE estimator and with the fit-free Aumann–Shapley variant are discussed in App. 14.

6.1 Benchmark↩︎

The benchmark task is to predict \(f(S)\) for held-out configurations from \(N\) measured ones. Per (model, corpus) cell, every method is fit on \(N\) configurations sampled from \(\mu_p\) at \(p=0.6\) and scored by median relative error on 120 held-out configurations, against the surrogate predictors of Section 2 (random forest, Gaussian process, RBF kernel model, and CoopQ’s linear-plus-pairwise model), at W4A4 on the six models and two corpora of Table 9 (App. 14, with the error-versus-\(N\) sweep in Fig. 7). At the \(N=L/2\) budget, coverage or additive is best in 5 of 11 cells and within 0.03 of best in 8, with small margins separating the leading methods, while isolate-and-sum, at the same measurement cost, fails at 0.38–0.63 error on the FP4-damaged models.

The deciding comparisons are extrapolation to differently quantized configurations, where coverage degrades at most \(2\times\) against the Gaussian process’s \(2\)–\(5\times\) (Section 6.2), allocation, where the surrogates fail outright (Section 7.3), and pricing, which only the parametric models provide (App. 15).

6.2 Out of distribution: stationary kernels do not encode saturation↩︎

On binary configuration vectors the RBF kernel [26] reduces to \(K(S,S') = \exp(-\gamma\,|S\triangle S'|)\), a stationary kernel in the Hamming metric. Under the uniform measure, kernels of this form are diagonal in the Walsh basis with eigenvalues that depend only on the order \(|T|\) and decay as \(|T|\) grows ([20]; derivation in App. 16), and approximately so at \(p=0.6\), so Gaussian-process regression with this kernel is shrinkage of high-order Walsh coefficients [26], the same low-order preference our measurements justify. This is why the Gaussian process matches our predictors at large \(N\) in distribution, where order-1 terms carry most of the damage variance (Section 3). The difference emerges out of distribution. Stationarity applies the same similarity rule everywhere on the cube, while saturation does not: under coverage the marginal effect \(f(S\cup\{i\})-f(S)=c\,a_i\prod_{j\in S}(1-a_j)\) shrinks as the context \(S\) grows. The stationary prior does not encode this structure. We test this on the two cells of Table 16 (Qwen3-0.6B and Qwen2.5-3B, wiki), fitting every method on configurations drawn at \(p=0.6\) as before and scoring each on 120 test configurations drawn at \(p=0.85\) or at \(p=0.35\). At the \(N=L/2\) budget, coverage’s error grows by \(1.3\)–\(2.0\times\) and the additive model’s by \(1.1\)–\(1.9\times\) relative to their in-distribution errors across the four (cell, shift) pairs, while the Gaussian process’s grows by \(2.1\)–\(4.9\times\) and the random forest’s by \(2.0\)–\(3.9\times\) (Fig. 4).

The heavily quantized direction is the one allocation exercises, hence an explanation for its performance in allocation.

6.3 A diffusion transformer↩︎

The benchmark repeats on the Qwen-Image-2512 diffusion transformer (\(L=60\) blocks, per-block NVFP4), with \(f(S)\) the mean squared error between the quantized and full-precision model outputs at the first denoising step (per-budget results in App. 14, Table 10). The measured order-\(\ge 2\) share is 0.36, above every LLM cell (0.016–0.30, Tables 9 and 16). At small budgets, coverage and random forest edge the other methods, with performance converging across methods as budget size increases.

7 Allocation at matched memory↩︎

7.1 Setup↩︎

Given a memory budget in effective bits, \(\mathrm{eff} = \sum \mathrm{numel}\cdot\mathrm{bits} / \sum \mathrm{numel}\), the allocator assigns each unit (a weight tensor, or a fused group that must share one serving format, App. 14) a bit-width in \(\{16,8,4\}\) to minimize predicted \(\Delta L_{\mathrm{CE}}\) (Section 7.4 extends the alphabet below four bits). Under the coverage model, predicted damage increases in the summed log-headroom costs \(b_i\) of the demotions taken, so minimizing it under a bit budget is a knapsack over those per-demotion costs (App. 13).

7.1.0.1 Converting the baselines into allocators.

The Section 6 baselines are predictors, so each is fit on the same \(N=300\) sampled configurations and run through the same knapsack over its predicted marginals (fit-add for the fitted additive model, gp-alloc and rf-alloc for the surrogates). We fit the coverage and additive scores either with least squares or an Aumann-Shapley path integral. The Aumann–Shapley score is the loss gradient averaged over increasingly quantized contexts, and under the coverage model it equals the optimal in-context coefficient evaluated at an effective deployment density (App. 12).

7.2 Serving against vendor NVFP4 checkpoints↩︎

On the dense Qwen3.6-27B we compare the additive allocation at 5.41 effective bits, derived from measured per-unit step damages on a task-aligned calibration set (App. 14), against the two public NVFP4 checkpoints, NVIDIA’s at 6.31 effective bits and Unsloth’s at 6.73, all served by vLLM on one B200 at concurrency 128. Ours and Unsloth’s checkpoints serve W4A4 NVFP4, quantizing activations at runtime against per-block scales; NVIDIA’s published checkpoint keeps its NVFP4 linears weight-only (W4A16). The additive configuration matches the vendor checkpoints downstream (Table 1), and the smaller footprint is realized as speed: it is the fastest system at every prompt/generation shape, \(1.5\)–\(2.0\times\) the bf16 throughput and 8–24% above the Unsloth checkpoint (Fig. 5).

The comparison repeats at 753B. On GLM-5.2 we compare the additive allocation (4.84 stored weight bits per parameter, 474 GB) against NVIDIA’s published NVFP4 checkpoint (4.76 bits, 465 GB) and the FP8 base model, served by SGLang on \(4\times\)B200 at concurrency 8. Both NVFP4 configurations are W4A4. Accuracy matches at matched stored size (Fig. 6), and the additive configuration reads 28.3 GB of weights per decoded token against the checkpoint’s 45.9, so it decodes \(1.19\times\) faster: the allocation places its low-precision bits on the units decode reads most often.

7.3 MoE language models↩︎

We benchmark our methods against the incumbent methods. The certificate accompanying these results depends on the units it is measured over. On the eight whole-layer FP4 block lattices of Qwen3-30B (Section 4.4) the measured order-\(\ge 2\) share has median 0.013. At the allocation granularity itself (per-linear units over the \(\{16,8,4\}\) alphabet) no lattice is enumerable, and the split-half cross-estimate (coefficient products of two disjoint halves, as in App. 11.1) from the 300 sampled configurations of the fitted baselines, unbiased but not constrained to be nonnegative, gives \(-0.09 \pm 0.10\) on the 30B pool and \(0.24 \pm 0.14\) on the 235B pool. At \(N=300\) the sampling error is of the order of the share itself, so the allocation-granularity certificate is an order-of-magnitude statement, not a precise value. Per-method values are in Tables 2 and 3, each with a paired test over the 16 shared evaluation sequences, Holm-corrected within each budget row at the 5% level. On Qwen3-30B the additive- and coverage-scored allocators lead at every measured budget.

The predictor skill of a black-box surrogate does not transfer: gp-alloc trails the AS and fitted allocators by 3–18% at the contended budgets (the extrapolation failure of Section 6.2 in allocation form), and rf-alloc falls below naive-uniform. CoopQ trails AS by \(+19\) to \(+54\%\) despite 1,205 sampled KL evaluations against AS’s zero, and it scores whole layers rather than the 240 per-linear units of the other methods, the only granularity its measurement design affords, since its permutation-walk cost scales with the unit count.

Table 12 shows AIME-2025 scores for the evaluated methods on Qwen3-30B, with every method between 66 and 76 against the bf16 anchor 73.3 and no significant ordering at any budget. This finding survives both W4A4 and W4A16 quantization regimes.

7.3.0.1 Scale replication (Qwen3-235B, Table 3).

The comparison was re-run at \(8\times\) scale on Qwen3-235B. The in-context family again carries the contended budgets: AS-additive is best at eff 4.2, the fitted additive and coverage variants at 4.5–6.0, and every method outside the family is significantly worse at eff 4.2, 4.5, and 6.0 (at 5.0, naive-uniform, ModelOpt, and gp-alloc are not separated after the Holm correction). The comparison extends once more to GLM-4.6 (355B): a three-level \(\{16,8,4\}\) allocation over its 43,364 units gives AS-coverage calibration KLs of 0.059, 0.045, and 0.036 at effective bits 5, 6, and 7, against ModelOpt’s 0.088, 0.071, and 0.045, margins of 34, 37, and 20%, with AS-additive within 0.006 of AS-coverage at every budget.

7.4 Allocation below four bits↩︎

Below four bits the base quantizer decides whether allocation has anything to work with: under round-to-nearest quantization, every allocator we measured collapses at every sub-4 budget regardless of model scale, so we use an error-compensating base. We construct a per-unit GPTQ [15] ladder on Qwen3-30B, quantizing every unit at 4, 3, and 2 bits (weight-only; details in App. 14).

In Table 5, the additive allocator’s KL sits \(3.0\)–\(6.6\times\) below ModelOpt’s at the servable budgets (\(\ge 2.75\), established by the serving results below). CoopQ and the isolated ranking, which reorders the additive allocator’s own per-unit damages, occupy the same tier, while MC-MoE and int8-outlier sit above ModelOpt.

The construction replicates at scale on Qwen3-235B-A22B (Table 4): of the three constructed allocations the additive one holds the lowest calibration KL at every sub-4 budget, preserves reasoning when served at effective bits 3.5, and degrades through 3.0. At effective bits 2.9 the additive scoring rule appears on both sides of the survival boundary: derived on the evaluation-task corpus itself, the additive and coverage allocations remain functional (42.9 and 41.2 AIME-2025), while the additive allocation of identical memory derived on a separately measured corpus collapses (2.1, as does the random mix at 5.0). The gap opens before the boundary (58.3 against 39.6 at effective bits 3.0), so at this scale the derivation corpus, not the scoring rule, sets the usable budget; every arm collapses at 2.75.

Serving these configurations tests whether the calibration-KL margins are realized as served accuracy. Table 6 reports HumanEval at both scales, and the full reasoning grid is Table 14 (App. 14). The failure mode has a common boundary, a termination threshold: above a model-specific calibration KL a configuration stops emitting its end-of-sequence token, every generation runs to the token cap, and no answer is produced. On the 30B grid the boundary sits near 0.45 nats (rf-alloc at KL 0.449 collapses on AIME-2025 while the additive configuration at 0.415 continues to score, Table 14), and the threshold governs termination rather than accuracy. At effective bits 2.9 the four HumanEval methods order exactly as their calibration KLs (0.215, 0.228, 0.236, and 1.042 giving 73.2, 44.5, 38.4, and 0.0). The accuracy gaps do not scale with the KL gaps: CoopQ sits within 0.013 nats of the additive configuration yet 29 points below it, so which units receive the 2-bit assignments changes served accuracy at fixed memory and near-fixed KL. At 2.75 the ordering itself breaks: the uniform mix has the lowest KL (0.319 against additive’s 0.415) yet scores 19.5 against 54.9.

8 Scope↩︎

8.0.0.1 The activation channel carries the structure.

Measuring the damage set function on the same layers with weights only (W4A16), activations only (W16A4), and both (W4A4) quantized, the weight channel is non-saturating (isolate-and-sum error \(\approx 0.10\), \(\sum_i f(\{i\})/f(\text{all}) \approx 1.0\)) while the activation channel saturates strongly (isolate-and-sum 2.80 against coverage’s 0.098 on the 3B W16A4 cell, Table 16). The saturation therefore lives in the activation channel, which is why the method targets activation and W4A4 quantization.

8.0.0.2 Outlier-borne coupling under coarse activation scaling.

At the deployed NVFP4 per-block format the order-\(\ge 2\) share is 0.07 and the additive and coverage models attain the lowest errors, while under per-tensor activation scales (a coarse setting that is not deployed) the share reaches 0.67 on the 0.6B certification lattice of Section 3.1 (the per-tensor cell of Table 7; the separately measured per-tensor lattice of Table 11 puts its own share at 0.81), and the nonparametric surrogates attain lower error, a breakdown the certificate forecasts. The coupling is outlier-borne: localizing the scale (per-block-256) drops the share to 0.01, clipping the top 0.1% of activations drops it to 0.16, and SmoothQuant-style scaling before quantizing drops it to 0.46. The fitted break-rates themselves are depth-structured rather than outlier-driven (on Qwen3-0.6B they correlate with depth at Spearman \(\rho = +0.68\) and with output RMS at \(\rho = +0.50\), but not with activation kurtosis).

8.0.0.3 Measurement protocol.

Headline cells use 3 seeds (std \(\le 0.03\)), and the per-tensor W4A4 isolate-and-sum failure is also 3-seed stable (0.6B \(20.6\pm 0.3\), 3B \(14.1\pm 0.3\)), so the rankings are robust.

9 Future work↩︎

9.0.0.1 Data randomness in the theory.

We treat \(f(S)=\Delta L\) as deterministic, but it is an expectation over the calibration distribution, with data-dependent break-rates. A two-source variance decomposition, layer-selection variance (the \(p\)-biased Walsh account) plus data-sampling variance, would give a principled law connecting calibration size to prediction variance and a data-aware certificate.

9.0.0.2 Hardware- and framework-aware allocation.

Effective bits equal memory is the wrong cost for realized speedup, which is a property of the serving stack: fusion granularity sets the alphabet (one format per fused qkv or expert block), kernel availability determines if certain bit-widths (e.g. sub-4) are servable. Co-designing the cost function of the allocator with the serving stack would lead to better realized speed gains.

9.0.0.3 Composing allocation with outlier and rotation methods.

Allocation chooses how many bits each unit gets, while rotations (QuaRot, SpinQuant, QuIP#), SmoothQuant, and llm.int8() change how much a unit is damaged at a given bit-width, Our framework operates on whatever base quantizer produces \(f(S)\), so the two compose: allocate over a rotated or outlier-handled base.

10 Conclusion↩︎

Viewed through its signed Möbius expansion, low-bit quantization damage appears to carry irreducible structure at every order, yet in the Walsh basis roughly 90% of its variance is order-1, and a saturating coverage model with one parameter per layer generates both behaviors exactly, the divergent series and the concentrated spectrum alike. Because the order-\(\ge 2\) share equals the best additive predictor’s mean-squared error as a fraction of the variance (Theorem 1), every result is paired with a certificate of when additive machinery suffices for prediction, measured on exact lattices or forecast in closed form from fitted break-rates, and the ordinal defect places the damage within a few percent of a monotone function of a per-layer sum, so the residual interaction is largely the curvature of the loss scale rather than structure among layers.

The practical consequences follow. Sensitivities measured in isolation overstate configuration losses by 71–376%, by a closed-form inflation factor. The fitted coverage and additive models price a configuration’s loss before deployment, are best or tied against black-box surrogates at small measurement budgets, and degrade at most \(2\times\) under extrapolation to differently quantized configurations where Gaussian-process surrogates degrade \(2\)–\(5\times\). One Aumann–Shapley path integral recovers the optimal additive coefficient at a layer-specific effective density, and the ordering of layers exactly, with no sampled configurations. Used as allocators at matched memory, the same models attain the lowest KL among the compared allocators, \(17\)–\(21\%\) below the production ModelOpt baseline on Qwen3-30B, with consistent margins at 235B and 355B. The margins appear where per-unit sensitivities are heterogeneous and budgets are contended, and they vanish in the served \(\ge 4\)-bit regime, which is near-lossless by construction. Below four bits, where an RTN base collapses every allocator, an error-compensating base restores the choice, and allocation decides whether the served model produces terminating generations at all.

11 The model class \(g\bigl(\sum_i b_i\bigr)\): canonicity, and details for the ordinal analysis↩︎

The Walsh–Hadamard expansion of \(f\) is exactly the functional-ANOVA decomposition on the \(p\)-biased Boolean cube: order 0 is the mean, order 1 the per-layer main effects, order \(k\) the \(k\)-way components. The tempting reading, that order-\(\ge 2\) energy measures interaction among layers, does not follow, because ANOVA additivity is additivity in the loss values and the loss saturates, so a model that is additive in the loss must spend order-\(\ge 2\) energy merely to represent the curvature of saturation, even with no coupling among layers. Order-\(\ge 2\) energy therefore conflates real coupling with the shape of a scalar transformation. The class \(L(S)=g\bigl(\sum_{i\in S} b_i\bigr)\), additive after a strictly increasing \(g\), partially disentangles the two, since if a single such \(g\) linearizes \(f\) (that is, \(g^{-1}\circ f\) is order-1), the order-\(\ge 2\) energy was produced by \(g\) rather than by interaction among layers. The residual test that survives this critique is the order-\(\ge 2\) energy after the best transformation, measured in Section 3.1, with protocol details below.

11.0.0.1 Why searching over transformations is canonical.

The class \(g\bigl(\sum_i b_i\bigr)\) is not a parametric convenience. It is exactly the class of damage functions that accumulate one layer at a time by a fixed rule, \(f(S\cup\{i\}) = f(S) \oplus d_i\), with a per-layer increment \(d_i\) that does not depend on \(S\) and an operation \(\oplus\) that is associative, continuous, and strictly increasing with identity \(0\). By the Aczél–Ling representation theorem, every such \(\oplus\) is conjugate to ordinary addition: there is a strictly increasing generator \(g\) with \(x\oplus y = g\bigl(g^{-1}(x)+g^{-1}(y)\bigr)\), unique up to scale [5], [6]. Accumulating damage by any rule with these properties is therefore equivalent to \(f\) being additive after some strictly increasing transformation, and the search of App. 13 ranges over such rules. Coverage is the instance \(x\oplus y = x+y-xy/c\), the probabilistic sum (noisy-OR) [27], [28], whose generator is the log-headroom \(g^{-1}(f) = -\log(1-f/c)\). The essential interaction of Section 3.1 is then precisely what no rule with these properties can absorb, namely the ordinal defect.

11.1 Details for Section 3: the ordinal certification and the noise-corrected energies↩︎

The claim is the finite binary case of additive conjoint measurement [23]–[25], and is stated for completeness.

11.1.0.1 Certification protocol: the estimand and the noise floor.

This and the next three paragraphs detail how the ordinal defect of Section 3.1 is measured and certified. The estimand is the draw-expected damage \(\tilde{f}\): where the measured \(f(S)\) of Section 3 averages 16 stochastic-rounding draws, \(\tilde{f}(S)\) is its expectation over draws, and all randomness below is over those draws. The ranking constraints of \(\tilde{f}\) are the inequalities \(b\cdot(x-y)>0\), one per ordered pair of configurations with \(\tilde{f}(x)>\tilde{f}(y)\). By Theorem 2, an index satisfying every constraint exists exactly when \(\tilde{f}\) is additive up to a strictly increasing transformation, and the ordinal defect is the smallest fraction of the constraints that any index must violate. The complication is that \(\tilde{f}\) is known only up to the draw noise. Both bounds below therefore share two devices. First, the draws are split into two halves, whose means \(\bar f_A\) and \(\bar f_B\) are independent estimates of \(\tilde{f}\) at every configuration, the split-half device of reliability theory [29], [30]. Second, a pair of configurations contributes a constraint on a given half (is eligible there) only if its gap on that half exceeds twice its standard error. A half-mean at configuration \(S\) has standard error \(\hat{\sigma}(S)/\sqrt{8}\), with \(\hat{\sigma}(S)\) the sample standard deviation of the draws, so the gap between configurations \(x\) and \(y\) has standard error \(\sqrt{\hat{\sigma}(x)^2 + \hat{\sigma}(y)^2}/\sqrt{8}\).

11.1.0.2 The upper bound: an exhibited index.

Because the defect is a minimum over indices, the fraction of constraints violated by any single index bounds it from above. The tightest such bound would come from the index minimizing the violated fraction, which is NP-hard to find [31], so we exhibit instead the solution \((b, \xi)\) of the hinge relaxation \[\min_{b,\,\xi} \;\sum_p \xi_p \qquad \text{subject to} \qquad b\cdot(x_p - y_p) \;\ge\;1 - \xi_p, \quad \xi_p \ge 0 ,\] the pairwise-preference construction of ranking with slack variables [32], here without the norm term, so the problem is a linear program, fit on the eligible constraints of half A. The fitted index’s violated fraction is then evaluated twice: in-sample, on the half-A constraints it was fit on, and cross-half, on half B’s own eligible constraints, which the fit never saw. The two evaluations play different roles. With noisy draws the reported number is instead an estimate of the index’s violation rate on \(\tilde{f}\)’s constraints, and the in-sample rate is biased low for that quantity. The cross-half rate removes this adaptation, evaluating a fixed index on independently selected constraints. We report both and quote the larger. On most cells the two agree to within about a tenth of a percentage point.

11.1.0.3 The lower bound: certified witnesses.

The witnesses are rank flips, two contexts that order the same two layers oppositely (the infeasibility witnesses of Section 3.1). For a layer pair \((i,j)\) and a context \(S\) containing neither, the constraint between \(\tilde{f}(S\cup\{i\})\) and \(\tilde{f}(S\cup\{j\})\) involves \(b\) only through the sign of \(b_i - b_j\), which is the same for every context. If, among \(\tilde{f}\)’s constraints, \(m\) contexts order \(i\) above \(j\) and \(n\) order \(j\) above \(i\), then every additive index violates at least \(\min(m,n)\) of them. Summing \(\min(m,n)\) over layer pairs, whose constraint families are disjoint, counts constraints that every index must violate, so this count divided by the number of eligible constraints is a lower bound on the defect, and by Theorem 2 a single opposed pair of contexts already refutes every increasing transformation. To certify a context against noise, it must be eligible on half A and recur with the same sign on half B. The witness count of Table 7 is the sum over layer pairs of the smaller number of certified contexts of each preference direction.

11.1.0.4 The matched-noise null.

The noise floor and the eligible constraint check still pass a small number of false positives, so every statistic above, for both bounds, is read against a matched-noise null lattice pushed through the identical pipeline. The null is the coverage model fitted to the measured means, which is additive after an increasing transformation and so contains no true flips, with the 16 draws regenerated as independent Gaussian samples centered on the fit, scaled by each configuration’s measured per-draw sample standard deviation \(\hat{\sigma}(S)\). This is a parametric bootstrap under the hypothesis of no essential interaction [33]. The Gaussian form is an assumption, and a mild one here, because every statistic consumes the draws only through the two half-means, each an average of 8 draws that are themselves means of per-token losses. The null measures the false-positive rate of the whole procedure, and a cell is certified only where the measured statistic exceeds its null.

11.1.0.5 Per-model detail.

Table 7 reports every cell against its null. In the deployed per-block regime, Qwen3-0.6B and Qwen3-8B sit at or below their nulls and are statistically indistinguishable from zero, Qwen3-30B-A3B has a defect of at most \(0.8\%\), at its null, and Llama-3.2-3B carries a small genuine residue (56 witnesses against a null of 6). In the per-tensor regime the witnesses exceed their null (74 against 18, at \(0.95\) sign replication), which is the outlier coupling of Section 8 appearing as certified rank reversals. An additive index nevertheless still orders at least \(98\%\) of the ranking constraints above the noise floor correctly. Nearly all of the apparent high-order energy in this worst cell is therefore curvature of the loss coordinates, as seen by \(E^\star\).

11.1.0.6 Removing the stochastic-rounding noise floor from measured order energies.

The split of the draws serves a second, separate role for the order energies of Section 3, where the halves are differenced rather than held out. Each measured \(f(S)\) averages \(K\) stochastic-rounding draws and so equals the draw-expected damage \(\tilde{f}(S)\) plus a mean-zero Monte-Carlo error with variance \(\sigma^2(S)/K\), where \(\sigma^2(S)\) is the per-draw variance at configuration \(S\). The Walsh transform is linear, so the measured coefficients are likewise unbiased. Their squares are not, since \(\mathbb{E}[(a+Z)^2] = a^2 + \mathop{\mathrm{Var}}(Z)\) for constant \(a\) and mean-zero \(Z\), so each measured order energy \(W_k\) exceeds the energy of the draw-expectation by the summed error variances of its order-\(k\) coefficients, a floor \(\nu_k > 0\). The per-coefficient error variance is nearly the same for every basis function (independent per-configuration noise is white in an orthonormal basis: exactly even at \(p=\tfrac12\), within a small per-order factor at \(p=0.6\)), so \(\nu_k\) is close to proportional to the number \(\binom{L}{k}\) of order-\(k\) basis functions. All but \(L\) of the \(2^L-1\) non-constant functions have order \(\ge 2\), so the floor’s mass lands almost entirely in the order-\(\ge 2\) energies and deflates the measured order-1 share. To estimate it, we average two disjoint halves of the draws separately, \(\bar f_A\) and \(\bar f_B\), and form the difference table \(d(S)=\tfrac12\bigl(\bar f_A(S)-\bar f_B(S)\bigr)\), in which the draw-expectation cancels exactly and a pure mean-zero error of the same variance \(\sigma^2(S)/K\) remains, so the order energies of \(d\) satisfy \(\mathbb{E}[W^{(d)}_k]=\nu_k\). This requires only that the two halves are independent (fresh rounding draws per configuration and draw). The corrected energies reported in Sections 3 and 4.4 are \(\max(W_k - W^{(d)}_k,\,0)\). Subtracting replicate-estimated error variance from an observed second moment is the classical variance-components device of the analysis of variance [34], applied here to split-half replicates [29], [30].

11.1.0.7 Estimating the transformation-independent energy floor.

For strictly increasing \(g\) let \(E(g)\) be the relative order-\(\ge 2\) Walsh energy of the transformed damage \(g^{-1}\circ \tilde{f}\). By Theorem 1 applied to \(g^{-1}\circ \tilde{f}\), \(E(g)\) is the fraction of the transformed damage’s variance that the best additive model leaves unexplained, and \(E^* := \inf_g E(g)\) is the floor of that fraction over all strictly increasing transformations. A small \(E^*\) therefore certifies that \(g^{-1}\circ \tilde{f}\) is nearly additive for some strictly increasing \(g\), so the measured high-order energy is curvature of the loss coordinates rather than coupling among layers. Estimation must respect two hazards: Monte-Carlo noise spreads nearly evenly across the Walsh basis, as above (247 of 255 basis functions have order \(\ge 2\), so raw noise masquerades as high-order structure), and \(g^{-1}\) is nonlinear, amplifying noise where the inverse is steep. The protocol therefore splits the 16 rounding draws into three parts and fits every transform on part 1 only. The parametric families are fit by nonlinear least squares, and the free monotone transform by alternating a weighted degree-\(\le 1\) fit with isotonic regression against the ordering of \(f\)’s values, which searches all increasing transformations at once and is the fit that estimates \(E^*\): isotonic regression returns the least-squares best monotone nondecreasing fit to a scatter, computed exactly by the pool-adjacent-violators algorithm [35], [36]. Each squared Walsh coefficient of the transformed damage is then estimated by multiplying the coefficient computed from part 2 by the coefficient computed from part 3, an unbiased estimate of the squared coefficient because the two parts’ noises are independent and mean zero. The product is unbiased but not constrained to be nonnegative, so a cell whose true share is near zero can estimate slightly negative, as the \(-0.4\%\) entry of Table 7 does. The headline numbers appear in Section 3.1, and the per-cell values in the last column of Table 7.

11.1.0.8 What depends on the measure.

Being exactly degree-\(\le 1\), raw or after a transformation, is an algebraic property of \(f\) (of its unique multilinear extension), independent of the product measure, since the span of the degree-\(\le k\) \(\mu_p\)-Walsh functions is the same subspace for every \(p\). What depends on \(p\) is the accounting: the best degree-\(\le 1\) projection, its coefficients \(w_i(p)\), and the order-\(\ge 2\) energy are all priced under the chosen measure. Two consequences follow. Theorem 1’s certificate is stated, and must be read, at the deployment \(\mu_p\). The wrong anchor of Section 5 is ultimately a wrong measure, the isolated slope \(s_i\) being the \(p\to 0\) limit of the deployment-point slope \(w_i(p)\).

12 Aumann–Shapley recovers the in-context coefficient at an effective density (proof under coverage)↩︎

There are three per-layer slopes, all gradients of the saturating loss at different anchors: the empty-anchor slope [1], [2], \(s_i = f(\{i\}) = c\,a_i\), inflated by the factor of Section 5; the deployment-point slope, which is the optimal additive coefficient, \(w_i = c\,a_i\prod_{j\ne i}(1 - p\,a_j)\), optimal but requiring a fit at the \(p\)-biased operating point; and the path-average slope [37], [38], which is fit-free, \[AS_i = c\,a_i\int_0^1 \prod_{j\ne i}(1 - t\,a_j)\,dt .\] Completeness, the property that the coefficients sum exactly to the total damage, is the classical efficiency property of the Aumann–Shapley value and does not require the coverage form. Under coverage it is immediate, since \(\frac{d}{dt}\bigl[c - c\prod_j(1 - t\,a_j)\bigr] = \sum_i c\,a_i\prod_{j\ne i}(1 - t\,a_j)\), and integrating over \(t \in [0,1]\) gives \(\sum_i AS_i = f(\text{all})\) with no remainder (measured residual on real models 0.06%). To state what the path integral recovers, write \(w_i(q) = c\,a_i\prod_{j\ne i}(1 - q\,a_j)\) for the optimal additive coefficient when the deployment density is \(q\).

Let \(f\) be the coverage model with break-rates \(a_1, \ldots, a_L \in [0,1)\) and let \(p \in (0,1)\). (i) For every layer \(i\) there is an effective density \(t_i^{*} \in (0, \tfrac12]\) such that \(AS_i = w_i(t_i^{*})\), so the fit-free path integral equals the optimal additive coefficient evaluated at \(t_i^{*}\) in place of \(p\). (ii) For all layers \(i\) and \(k\), \(AS_i > AS_k\) if and only if \(w_i > w_k\), if and only if \(a_i > a_k\), so the path-average and deployment-point coefficients order the layers identically at every deployment density.

Part (i) places the fit-free coefficient in the same family as the fitted one, evaluated at the effective density \(t_i^{*}\) rather than at the deployment density \(p\) (since \(t_i^{*} \le \tfrac12 < p\) at our \(p=0.6\), the path-average coefficient sits above the deployment-point one whenever other layers have nonzero break-rates). Part (ii) shows that this difference in evaluation point never reorders layers. What remains numerical is the magnitude agreement between \(AS_i\) and \(w_i(p)\): the correlation is 0.9997 on coverage-generated data and 0.785 on real models, while the empty-anchor \(s_i\) is inflated by the factor \({\approx}\,e^{p\sum_j a_j}\). Since \(s_i = c\,a_i\) is itself increasing in the break-rate, part (ii) extends to the empty anchor under the coverage model, so the failure of isolated sensitivity documented in Section 5 is a failure of magnitude rather than of order.

13 The log-headroom knapsack, and the generator the data selects↩︎

13.0.0.1 Knapsack certificate.

Because coverage is linear in \(b\), a service-level constraint of the form \(\Delta L_{\mathrm{CE}}\le \varepsilon\) maps to a scalar budget \(\sum_{i\in S} b_i \le B\) with \(B = -\log(1 - \varepsilon/c)\). The cheapest configuration meeting the constraint protects the set \(P\) minimizing protection cost subject to \(\sum_{i\notin P} b_i \le B\) is a 0/1 knapsack, solvable exactly by dynamic programming. The deliverable is a predicted \(\Delta L_{\mathrm{CE}}\) for the chosen configuration, quotable before committing (\(\le 4\%\) error, App. 15), which a ranking structurally cannot produce and which a Gaussian process prices with \(1.6\)–\(2.9\times\) coverage’s error near the heavily quantized corner where optimal configurations live (Section 6.2).

13.0.0.2 Which generator the data selects.

The ceiling \(c\) and membership in the class \(g\bigl(\sum_i b_i\bigr)\) are established empirically, by the marginal-damage measurements of Section 4.1 and the ordinal certification of Section 3.1, while the exponential generator (coverage/noisy-OR), which is what makes the headroom consumption multiplicative, is justified only by fit quality. Refitting the lattices with candidate strictly increasing generators (all with \(L+1\) parameters, \(p=0.6\)) shows that prediction is insensitive to the choice (held-out relative error \({\approx}\,0.064\)–\(0.071\) for all), but the slope calibration, the ratio \(w_{\text{model}}/w_{\text{data}}\) of model-implied to measured in-context slopes, selects one: across the five non-degenerate of six (model, corpus) cells, the probit-\(\sqrt{\cdot}\) ratio is 0.93–1.03, while the exponential ratio spans 0.86 down to \({\approx}0\) (exp, probit, tanh, and rational sit at 0.58–0.67 on Qwen3-0.6B). The saturation rule best supported by the data is therefore the probit-\(\sqrt{\cdot}\) generator \(g(u) = c\,\operatorname{erfc}\bigl(1/\sqrt{2u}\bigr)\), the probability that zero-mean Gaussian corruption of variance \(u\) exceeds a fixed threshold, scaled by the ceiling; the exponential coverage model is its tractable approximation.

14 Experimental setup↩︎

We collect the protocol shared across Sections 3–8 here. Section-specific deviations are noted in place.

14.0.0.1 Models.

LLMs: Qwen3-0.6B, Qwen2.5-3B, Qwen3-4B, Qwen3-8B, Llama-3.2-3B, with the per-linear order-share cells of Section 3 extending to Qwen3-14B, Qwen3-32B, Llama-3.3-70B, Qwen2.5-72B, and gpt-oss-120B; Qwen3-30B-A3B, Qwen3-235B-A22B, and GLM-4.6 (355B MoE) for allocation at scale; Qwen3.6-27B for the vendor-checkpoint serving comparison, with damage measured and ModelOpt calibrated on a chat-templated task corpus (96 MATH-train, 64 gsm8k-train, 96 MMLU-Pro validation prompts); Qwen3-235B and GLM-5.2 (753B) for the scale-robustness checks. Diffusion: Qwen-Image-2512 DiT (\(L=60\) transformer blocks) for the diffusion prediction results. All weights are the public checkpoints.

14.0.0.2 Quantization formats.

The measurement regime throughout is W4A4 with NVFP4 per-block scales. We also analyze W16A4 (activation-only, isolating the channel that carries the non-additivity) and per-tensor activation scaling (a coarse setting that induces interactions of every order among layers). The allocation alphabet is \(\{16, 8, 4\}\) bits (bf16 / FP8 / NVFP4) for the servable studies and is extended to \(\{16,8,4,2,1\}\) for the below-4-bit collapse study. Serving uses three pipelines. NVIDIA ModelOpt PTQ with TensorRT-LLM serves the \(\ge 4\)-bit MoE deployment recipes, and vLLM the vendor-checkpoint comparison of Section 7.2; these paths serve W4A4 ModelOpt exports with runtime activation quantization on the NVFP4 units (NVIDIA’s published 27B checkpoint, which keeps its NVFP4 linears weight-only, is the exception), as does the SGLang-served GLM-5.2 comparison.

14.0.0.3 The damage set function and the lattice.

For a contiguous block of \(L\) layers we form \(f(S)=L_{\mathrm{CE}}(S\;\text{quantized}) - L_{\mathrm{CE}}(\text{bf16})\) by running the model with exactly the units in \(S\) quantized. Every exact lattice in the paper measures such a block centered at the network’s mid-depth (block offset \(\lfloor(L'-L)/2\rfloor\) in an \(L'\)-layer network, so the middle eight of Qwen3-0.6B’s 28 layers), except the Qwen3-30B-A3B extension of Fig. 3, which repeats the measurement at eight block positions. Restricting to a small block is what makes the \(2^L\) enumeration feasible, and the enumerated lattice is a reference against which sampled estimates, fitted models, and the noise corrections of App. 11.1 are certified. The full-network cells (Section 3, the prediction benchmark, and the allocation studies) instead quantize units anywhere in the network, where enumeration is impossible, and sample configurations from the \(p\)-biased measure \(\mu_p\). Each \(f(S)\) is averaged over stochastic-rounding draws (3 seeds for the headline sampled cells, std \(\le 0.03\); 8–16 draws for the exact lattices of Section 3, whose noise treatment is in App. 11.1). The Walsh/Möbius transforms are computed from measured \(f(S)\). The order-\(\ge 2\) energy certificate is the measured \(\sum_{k\ge 2}W_k/\mathop{\mathrm{Var}}(f)\) where a lattice is affordable and, at scale, the closed-form forecast from the fitted break-rates, namely the exact heterogeneous product form \(1 - \sum\beta_i/[\prod(1+\beta_i)-1]\) of Proposition 5 (Section 4.3), whose symmetric summary is \(\tau\) (share \(\approx\tfrac12\tau^2\)). The “\(\tau\)-forecast” label refers to this pipeline. Where a measured share exists for comparison, the forecast understates it (Qwen3-30B: forecast \(3{\times}10^{-4}\) against measured 0.013). Even at that ratio the 235B and GLM-4.6 forecasts (\({\sim}10^{-4}\)) stay below 0.01, so they certify additive dominance, not a precise share.

14.0.0.4 Calibration and evaluation data.

Prediction and interaction-order studies use WikiText-2 (“wiki”) and a Python code corpus, CodeSearchNet functions (“code”). The MoE allocation studies use a task-aligned calibration set: BFCL-v3 (live_simple, live_multiple, simple_python, multiple) tool-use prompts plus RULER validation prompts, tokenized to 16 sequences of 2,048 tokens. The fake-quant metric is the KL divergence \(\mathrm{KL}(\text{bf16} \,\|\, \text{quantized})\) of next-token distributions on the disjoint eval half of this set. Served downstream evaluation uses AIME-2025, GPQA diamond, MMLU-Pro, HumanEval, and gsm8k (Section 7).

14.0.0.5 Predictor fitting.

The coverage model \((c, \{a_i\})\) and the additive model are fit by joint least squares rather than per-coefficient sampling, for the variance and consistency reasons given in the next two paragraphs. Fits use \(N\) sampled configurations. Fits are initialized by linearizing through the generator (ordinary least squares on the log-headroom values at a provisional ceiling) and refined by bounded trust-region least squares in loss space. Fitting in loss space avoids the alternative of transforming the data and then regressing, which is biased because noise pushed through the nonlinear transform no longer averages to zero. The surrogate baselines (RF/MiCo, GP-RBF/AutoQRA, RBF kernel regression/AMQ, CoopQ pairwise) are fit on the same configurations, and GP/RF are additionally converted into allocators (knapsack over their predicted marginals) for the allocator-to-allocator comparison in Section 7. Aumann–Shapley scores are computed fit-free by the path-integral estimator: the loss gradient is integrated by a 4-node midpoint rule along the path that scales weights and unit-input activations linearly from full precision (\(t=0\)) to fully quantized (\(t=1\)), one integral per demotion level (\(16\to 8\) and \(16\to 4\)), with MoE routing frozen along the path since the router’s argmax is not differentiable.

14.0.0.6 Why joint least squares rather than per-coefficient sampling.

The plug-in estimate of each Walsh coefficient from samples, \((1/N)\sum_n f(S_n)\chi_T(S_n)\) [21], is unbiased, but its variance carries the total second moment \(\mathbb{E}_{\mu_p}[f^2]\), because every other coefficient’s energy leaks in through the sampled, non-orthogonal design. Jointly regressing on all \(L+1\) degree-\(\le 1\) terms absorbs that leakage into the fitted terms. Ordinary least squares is the classical control-variate regression estimator, whose asymptotic variance is only the energy of \(f\) orthogonal to the fitted span, that is, the order-\(\ge 2\) energy plus noise, rather than \(\mathop{\mathrm{Var}}(f)\) [39]–[41]. Random-design least squares is stable and near-optimal already at \(N \propto L\log L\) samples [42]. With \({\sim}90\%\) of the variance at order 1 (Section 3), regression gives roughly a \(10\times\) per-coefficient variance reduction at equal \(N\), the reciprocal of the order-\(\ge 2\) share, so the certificate also quantifies the gain from regression. The same regression-over-sampling argument recurs in Shapley estimation (KernelSHAP against permutation sampling, [43]).

14.0.0.7 Statistical footing of the fits.

The coverage fit is ordinary nonlinear least squares: consistent for the best coverage-family approximation of \(f\) even when the model is misspecified [44], [45], and the maximum-likelihood estimate under Gaussian noise when it is correct. Two caveats: the objective is non-convex, so we initialize by linearizing through the generator as described above, and on near-linear data the ceiling is weakly identified (only the products \(c\,a_i\) are pinned down), which perturbs \((c,a_i)\) individually but not predictions.

14.0.0.8 Measurement cost.

Cost is counted in measured configurations, since each is a forward pass over the calibration set. The headline budget is \(N\approx L\) configurations, against roughly \(100L\) progressive-quantization evaluations for SPQE’s permutation-sampled Shapley [9].

14.0.0.9 Prediction benchmark in full.

Table 9 reports the per-cell results of the Section 6.1 benchmark at the small budget, and Fig. 7 the error-versus-\(N\) sweep on the 3B-wiki cell. The certificate column is the held-out residual of an additive fit, the direct Theorem 1 estimate: 300 configurations are sampled per cell from \(\mu_{0.6}\), an additive model is fit on the first 150, and the column reports the mean squared residual on the remaining 150 divided by the damage variance.

14.0.0.10 Certificate estimators.

Table 11 reports the per-cell detail behind the estimator comparison of Section 4.4: the root-mean-squared error, over 8 sampling seeds, of estimating a lattice’s true order-\(\ge 2\) share from \(N\) configurations sampled at \(p=0.6\), for the Proposition 5(i) forecast from the coverage fit and the held-out residual of an additive fit. The residual estimator’s agreement with the forecast, available at no extra measurement cost, is a per-instance check of the coverage assumption itself.

14.0.0.11 Allocation.

Given a memory budget \(\mathrm{eff} = \sum \mathrm{numel}\cdot\mathrm{bits} / \sum \mathrm{numel}\) (kernel-independent, with numel a tensor’s element count), a 3-level \(\{16,8,4\}\) knapsack minimizes predicted \(\Delta L_{\mathrm{CE}}\) per effective bit. A unit is a weight tensor or a fused group that must share one format, such as the q/k/v projections or one MoE layer’s experts. The served recipes additionally tie fusion groups (q/k/v into one qkv_proj, one expert format per MoE layer block) so the per-layer-mixed checkpoint loads in TensorRT-LLM. Baselines are matched to the same \(\mathrm{eff}\). The two fit-free variants price demotions differently: AS-add uses the path-integral coefficients directly, so predicted damage adds across units, while AS-cov converts the same coefficients to break-rates, with the ceiling calibrated from one measured corner (the all-4-bit configuration), so damage composes through the coverage form. The two structural rules use no damage measurements. Naive-uniform lowers every unit together through the \(\{16,8,4\}\) ladder and, if the resulting uniform level falls below the target effective bits, restores the smallest tensors to 16 bits until the next restoration would exceed the budget. Naive-routed places routed experts at the lowest level and protects attention.

14.0.0.12 GPTQ ladder details (Section 7.4).

Grids are symmetric with one scale per 128 input columns, and each linear layer’s Hessian is accumulated from the scoring half of the task calibration set. The all-units GPTQ-4 calibration KL of 0.054 is comparable to NVFP4 RTN at 0.047, and at 2 bits RTN gives 7.0 against GPTQ’s 13.6, so 2 bits is unusable as a uniform level under either quantizer. For the fused expert blocks, all experts in a layer share the Hessian of the block’s full token stream: the first projection’s Hessian is accumulated from the block input, and the second projection’s from intermediate activations recomputed on that input, a standard simplification that ignores the router’s token-to-expert assignment. At the 235B scale the all-units GPTQ-3 calibration KL is 0.115.

14.0.0.13 Sub-4 served detail (Section 7.4).

Table 14 gives the full served grid on the reasoning benchmarks for every method and budget. The reasoning benchmarks are AIME-2025 (30 problems), GPQA diamond (198 questions), and MMLU-Pro (a fixed 1,000-question subset), decoded at temperature 0.6, top-\(p\) 0.95, top-\(k\) 20, with a 32,768-token budget on AIME-2025 and 16,384 elsewhere. The ModelOpt configuration scores at the chance level of each multiple-choice format (23.2 on four-option GPQA, 9.0 on ten-option MMLU-Pro).

14.0.0.14 Joint measurement of the allocation tables.

CoopQ’s quadratic program is solved by greedy plus local search (SCIP unavailable). Its published \({\sim}100L\) budget (\({\sim}24\)k evaluations) was not run (Section 7.3 reports the 1,205- and 942-evaluation tiers). An ablation isolates the cause of its deficit: the additive and coverage models fit on the 942 configurations CoopQ’s permutation walks visited, allocating on its alphabet, land at CoopQ’s level (\(+.004\)–\(.008\), paired-significant), because configurations visited along a permutation walk are efficient for averaging marginal contributions but nearly collinear as regression data, while the same regression on 300 independently sampled configurations lands within \(.002\) of AS at every contended budget (within \(.001\) at eff \(\ge 4.5\)).

14.0.0.15 Hardware and software.

Scoring and fake-quant run on a single B200 (device-map pipeline-parallel across \(8\times\)B200 for the 235B/355B/753B models), and serving runs on \(1\times\)B200 (30B, TP=1) up to \(8\times\)B200 (GLM, TP=8). PyTorch and Hugging Face Transformers handle fake-quant, NVIDIA ModelOpt and TensorRT-LLM handle the \(\ge 4\)-bit deployment recipes. SGLang serves the sub-4 benchmarks from bf16-exported checkpoints and the GLM 5.2 benchmarks (Section 7.4), vLLM serves the Qwen3.6-27B comparison (Section 7.2), and scikit-learn provides the surrogate baselines.

15 Interpretability of the fitted parameters↩︎

15.0.0.1 Budget pricing.

Fitting coverage on \({\sim}200\) configurations of Qwen3-0.6B (all 28 layers, activations quantized at FP4 per-block-32, configurations drawn at \(p=0.6\)), we price unseen configurations of the form “protect the \(m\) most fragile layers and quantize the rest”, comparing the predicted \(\Delta L\), which costs no forward passes once the model is fit, against the measured value (Table 15). Across the deployment range, protecting a few to about half the layers, coverage prices the configuration to within 4%, whereas isolate-and-sum’s prediction is \(2\)–\(3\times\) the truth. The lone breakdown is quantizing a single layer, where the model, fit on configurations drawn at \(p=0.6\), extrapolates far outside its fit distribution and misses by \({\sim}95\%\).

15.0.0.2 The fragility scores are corpus-portable.

Fitting the break-rates \(a_i\) independently on wiki and on code yields strongly agreeing vectors (Spearman \(\rho = 0.74\), Pearson \(r = 0.86\) on Qwen3-0.6B; \(\rho = 0.78\), \(r = 0.93\) on Qwen2.5-3B), with the same deep layers fragile in both corpora. We propose to fit \(a_i\) once on a source corpus, and to deploy on a new corpus refit only the single scalar ceiling \(c\). At small budgets this one-parameter transfer attains lower error than the \((L{+}1)\)-parameter in-domain fit, 0.058 against 0.098 at 9 configurations for code-to-wiki on both 0.6B and 3B, and 0.076 against 0.110 at 36 configurations for wiki-to-code on 3B.

15.0.0.3 Multi-seed robustness.

Table 16 reports median held-out relative error over 3 seeds at the \(N\approx L\) budget. Coverage attains the lowest error on 0.6B and ties the Gaussian process on 3B, isolate-and-sum fails uniformly across seeds, and 3B’s higher order-\(\ge 2\) share (0.26 against 0.6B’s 0.05) is exactly where coverage’s margin over the additive fit is largest, so the certificate forecasts where the saturation structure pays off before any predictor is fit.

16 Proofs (Theorems 3 and 4, Proposition 5, the two anchors, kernel eigenvalues)↩︎

16.0.0.1 Notation.

Under the product measure \(\mu_p\) (each \(x_i \sim \mathrm{Bernoulli}(p)\), independent), the space of functions on \(\{0,1\}^L\) carries the inner product \(\langle g,h\rangle = \mathbb{E}_{\mu_p}[gh]\). With the standardized coordinates \(\phi_i(x) := (x_i-p)/\sqrt{p(1-p)}\) (\(\mathbb{E}[\phi_i]=0\), \(\mathbb{E}[\phi_i^2]=1\)), the Walsh functions \(\chi_T := \prod_{i\in T}\phi_i\) (\(\chi_\emptyset := 1\)) form an orthonormal basis: for \(T\ne T'\), independence factorizes \(\mathbb{E}[\chi_T\chi_{T'}]\) per coordinate, and each coordinate in \(T\triangle T'\) contributes \(\mathbb{E}[\phi_i]=0\). Hence \(f = \sum_T \hat{f}(T)\chi_T\) with \(\hat{f}(T)=\langle f,\chi_T\rangle\), \(\mathbb{E}[f]=\hat{f}(\emptyset)\), and (Parseval) \(\mathop{\mathrm{Var}}_p(f) = \sum_{T\ne\emptyset} \hat{f}(T)^2\) [20].

16.0.0.2 Proof of Theorem 3 (Möbius spectrum).

For \(T\ne\emptyset\), \(\varphi(T) = \sum_{R\subseteq T}(-1)^{|T|-|R|} f(R)\) with \(f(R) = c - c\prod_{i\in R}(1-a_i)\). The constant \(c\) alternates to zero over the subsets of a nonempty \(T\). For the product part, the alternating subset-sum of a product factorizes coordinatewise, \[\sum_{R\subseteq T}(-1)^{|T|-|R|}\prod_{i\in R}g_i = \prod_{i\in T}(g_i-1).\] With \(g_i = 1-a_i\) this gives \(-c\prod_{i\in T}(-a_i) = c\,(-1)^{|T|+1}\prod_{i\in T}a_i\). 0◻

16.0.0.3 Proof of Theorem 4 (Walsh spectrum).

Write \(f = c - H\) with the headroom \(H(x) = c\prod_i g_i(x_i)\), \(g_i(x_i) = 1 - a_i x_i\). Constants are orthogonal to every \(\chi_T\) with \(T\ne\emptyset\), so \(\hat{f}(T) = -\hat{H}(T)\) for \(T\ne\emptyset\), while \(\hat{f}(\emptyset) = \mathbb{E}[f] = c - \hat{H}(\emptyset) = c\bigl(1-\prod_i(1-a_ip)\bigr)\). Because \(H\) and \(\chi_T\) are both products over independent coordinates, the expectation factorizes: \(\hat{H}(T) = c\prod_{i\in T}\mathbb{E}[g_i\phi_i]\cdot\prod_{i\notin T}\mathbb{E}[g_i]\). The one-coordinate integrals (using \(x_i^2=x_i\)): \(\mathbb{E}[g_i] = 1-a_ip\) and \(\mathbb{E}[g_i\phi_i] = -a_i\,\mathbb{E}[x_i(x_i-p)]/\sqrt{p(1-p)} = -a_i\sqrt{p(1-p)}\). Substituting and negating gives \(\hat{f}(T) = c\,(-1)^{|T|+1}(p(1-p))^{|T|/2}\prod_{i\in T}a_i\cdot\prod_{i\notin T}(1-a_ip)\). 0◻

16.0.0.4 Corollary (the two anchors of Section 5).

By Theorem 1 the optimal degree-\(\le 1\) predictor is the projection \(\hat{f}(\emptyset) + \sum_i \hat{f}(\{i\})\chi_{\{i\}}\). Converting \(\chi_{\{i\}}\) back to the \(\{0,1\}\) coordinate gives the optimal additive coefficient \(w_i = \hat{f}(\{i\})/\sqrt{p(1-p)} = \mathbb{E}_p[f\mid x_i=1] - \mathbb{E}_p[f\mid x_i=0]\), the in-context marginal, which under coverage (Theorem 4 with \(T=\{i\}\)) is \(w_i = c\,a_i\prod_{j\ne i}(1-a_jp)\). The empty-anchor slope is \(s_i = f(\{i\}) = c\,a_i\) directly from the coverage model. Hence the inflation factor \(s_i/w_i = 1/\prod_{j\ne i}(1-a_jp) \approx e^{p\sum_j a_j}\) quoted in Section 5. 0◻

16.0.0.5 The symmetric case.

For completeness, the equality case of Proposition 5(iii) with its order spectrum.

16.0.0.6 Walsh eigenvalues of the RBF kernel (Section 6.2).

On binary vectors the squared Euclidean distance collapses to the size of the symmetric difference, \(\lVert x - x'\rVert^2 = \sum_i (x_i - x_i')^2 = |x \oplus x'|\), so the RBF kernel is a function of the XOR alone, \(K(x,x') = \kappa(x \oplus x')\) with \(\kappa(u) = e^{-\gamma \sum_i u_i}\). For any such kernel, the operator \((T_K f)(x) = \mathbb{E}_{x'}[K(x,x')f(x')]\) under the uniform measure (\(p=\tfrac12\)) is diagonal in the Walsh basis. Fix \(x\) and reparametrize the average over \(x'\) by \(u := x \oplus x'\). Hence \[(T_K\chi_T)(x) = \mathbb{E}_{x'}\bigl[\kappa(x \oplus x')\,\chi_T(x')\bigr] = \mathbb{E}_u\bigl[\kappa(u)\,\chi_T(x \oplus u)\bigr].\] The character identity \(\chi_T(x \oplus u) = \chi_T(x)\,(-1)^{\sum_{i\in T} u_i}\) then gives \(T_K\chi_T = \lambda_T\,\chi_T\) with \(\lambda_T = \mathbb{E}_u\bigl[\kappa(u)\,(-1)^{\sum_{i\in T} u_i}\bigr]\). This expectation factorizes over the independent uniform bits, \[\lambda_T \;=\; \prod_{i\in T}\mathbb{E}\bigl[e^{-\gamma u_i}(-1)^{u_i}\bigr] \prod_{i\notin T}\mathbb{E}\bigl[e^{-\gamma u_i}\bigr] \;=\; \Bigl(\tfrac{1-e^{-\gamma}}{2}\Bigr)^{|T|}\Bigl(\tfrac{1+e^{-\gamma}}{2}\Bigr)^{L-|T|} \;=\; \Bigl(\tfrac{1+e^{-\gamma}}{2}\Bigr)^{L}\,\tanh(\gamma/2)^{|T|}.\] The eigenvalues are positive, depend on \(T\) only through \(|T|\), and decay geometrically in \(|T|\) with ratio \(\tanh(\gamma/2) < 1\), which is the high-order shrinkage quoted in Section 6.2. At \(p \ne \tfrac12\) the kernel is still a function of \(x \oplus x'\), but the \(\mu_p\)-Walsh basis is no longer exactly its eigenbasis, which is why Section 6.2 states the diagonalization as approximate at \(p = 0.6\).

17 The assembled evidence for the coverage model↩︎

Table 17 collects the evidence for the coverage model in one place, with each claim’s standing stated explicitly, since the support is deliberately heterogeneous: a representation theorem, certified measurements, a confirmed prediction, and a set of benchmark results. Detailed treatments are at the listed locations.

18 Extended related work↩︎

18.0.0.1 MoE-specific mixed precision.

For mixture-of-experts models, MC-MoE [47] solves a linear program over expert routing and frequency importance. It allocates by an additive importance score and does not analyze the order of the damage or report a validity certificate. The head-to-head comparisons against CoopQ and MC-MoE appear in Section 7.

18.0.0.2 Path-integral sensitivity.

Integrated Gradients [38] and the Aumann–Shapley value [37] are the classical methods. PQI/ReQuant [48] applies the path integral to weight-level sensitivity but assumes per-weight effects sum, and QIG [49] is token-level for vision-language models. We apply the path integral to the layer-selection set function (App. 12).

18.0.0.3 Rotation methods.

QuaRot, SpinQuant, and QuIP# [50]–[52] flatten the representation before quantizing rather than allocating precision. They alter the base quantizer on which any allocator operates, so they compose with allocation (Section 9). A rotation that flattens per-layer heterogeneity also removes the per-layer differences that allocation exploits.

18.0.0.4 Low-order surrogates of subset functions outside quantization.

The analytical approach has precedents in several adjacent literatures: functional ANOVA over configuration spaces [53]; Datamodels [54], whose finding that linear surrogates of training-data-subset effects work well is the data-subset analog of our first-order finding; SPEX [55], sparse Boolean-Fourier surrogates of LLM outputs over input-masking subsets; the Harsanyi-interaction line [56], which applies Möbius machinery to input coalitions; and Walsh surrogates for pseudo-Boolean black-box optimization [57], [58].

18.0.0.5 In-context and isolated measurements.

That an effect measured in isolation differs from the effect in context is known qualitatively from coalition-value pruning [7], the joint-versus-sum analysis of Combinatorial Brain Surgeon for weight pruning [59], the Hydra-effect self-repair mechanism [8], the permutation-sampled marginals of CoopQ’s SPQE estimator [9], and the layer-subset surrogates now used for LLM pruning [60], all of which work around the same anchor problem. What we contribute is the application to mixed-precision quantization, specifically, the closed-form inflation factor under saturation, its measured magnitude on modern LLMs, and a fit-free estimator of the correct coefficient (Section 6), one Aumann–Shapley path integral in place of the SPQE estimator’s \({\sim}100L\) quantized evaluations.

References↩︎

---

1. Work done during an internship at Baseten Labs, Inc.↩︎