1 Introduction↩︎

A self-evolving agent that accumulates skills without governance degrades, as stale and redundant entries crowd retrieval, a failure mode recently named library drift [1]. Agents already learn by synthesizing new skills from their own failures [2], [3]; the missing piece is governance, retiring skills that stop helping under a bounded cap. Ratchet [4] makes this precise: retirement keeps a growing library from drifting more than a fixed margin below the no-skill baseline. But that guarantee rests on one quiet assumption, that the signal telling the agent which skills failed is honest (the per-skill contribution estimator is unbiased). Coding and QA satisfy it with unit tests and exact-match graders; the tasks agents increasingly face (research synthesis, long-form reporting, analysis) do not: with no golden answer the only scalable grader is an LLM judge [5], and its error is not white noise. Judges are systematically biased rather than merely inconsistent [6], [7], tending to wave through certain failure classes (a confident misquote; a flipped conclusion that stays fluent), so their error is asymmetric: failures get reported as passes. The consequence is a blind curator (Fig. 1): the component that should retire bad skills stops seeing the evidence it retires on. Where library drift was the disease governance was built to cure, curator blindness is what befalls the cure itself when the reward is fallible.

We treat that asymmetry as the object of study. Our thesis turns on two separable knobs of the reward channel: a symmetric noise rate \(\rho\) (true label flipped either way), and a false-pass rate \(\rho_{F\to P}\), the fraction of true failures reported as passes (we write \(q\) for this same rate when sweeping it in the experiments). The agent responds to them in opposite ways. It tolerates noise gracefully, but hits a cliff in \(\rho_{F\to P}\): beyond \(\rho_{F\to P}=(1-\tau)/2\), under this channel, no amount of data can rescue retirement (Sec. 4). Our contributions are:

• A universal mechanism failure with regime-dependent fallout. Across four subsets and two domains (Secs. 6–8), and by separating genuine contribution-retirement from cap-eviction churn, we show false-pass bias disables the curator past the cliff in every regime (verifier-like near-zero graders are spared; below the cliff survival depends on skill margins), while the downstream outcome harm is regime-dependent: it appears only when the same corruption also starves synthesis, and is otherwise silent. Symmetric noise is survivable; a strict judge is safe but starvation-prone.

• A before-deployment go/no-go test (Fig. 5). A constructed-ground-truth testbed and a defect-injection audit that pinpoints any judge’s error rates, so an operator can locate their judge relative to the threshold before trusting self-evolution (Sec. 5).

• Why the asymmetry exists. A bias-aware non-divergence bound (Prop. 1\('\)): noise only attenuates the retirement signal, while false-pass bias displaces it past an unrecoverable threshold.

This is a behavior-centric study: we ask not whether task success rises but how the skill library evolves, and when retirement under a cap reliably steers it versus silently failing. The audit is a behavioral test suite for the reward; the blind curator is a mechanistic account of a process-level failure aggregate metrics never surface.

2 Related work↩︎

Skill libraries and verbal self-improvement. A line of work lets a frozen LLM agent improve by accumulating textual artifacts from its own experience rather than updating weights: reusable skills [2], distilled experience [3], instruction manuals [8], episodic reflections [9], workflow memory [10], and natural-language feedback as a gradient surrogate [11], [12]. A second wave strengthens skill creation itself, via inductive distillation from traces [13], autonomous create-and-evolve loops [14], [15], and RL-coupled skill growth [16], with benchmarks probing transfer [17] and a lifecycle view [18] also underlying OS-like agent memory [19]. A recurring weakness is that the library only grows (library drift [1]), and almost all of this work assumes the outcome signal driving accumulation is correct; we ask what happens when it is not.

Skill lifecycle governance. Closest to us is a line that treats the skill set as something to be governed, not just grown: lifecycle governance from collection to evolution [20], dynamic lifecycle management under agentic RL [21], and trajectory-driven self-adaptation [22], [23]. All share the premise that bad skills must be pruned, but all presume the pruning signal is trustworthy. Our contribution is prior to the governance policy: we characterize when the signal any such policy consumes is reliable enough for pruning to help rather than hurt.

The Ratchet mechanism we build on. We start from Ratchet [1], [4], a minimal recipe on a frozen LLM: a Critic labels each failed task, a Synthesizer turns recurring failure patterns into skills, a Router retrieves at most one skill per task, and, crucially, a Curator retires any skill whose empirical contribution falls below a threshold after enough trials, under a hard cap on active skills. Retirement-plus-cap yields a non-divergence guarantee: on a fixed task distribution, expected performance cannot fall more than a fixed margin below the no-skill baseline. We adopt Ratchet because it is, to our knowledge, the only self-evolving-skill scheme with such a guarantee. That makes it the natural object on which to ask when the guarantee survives an imperfect reward, since its proof is the thing an unreliable judge can break. It was evaluated only with clean verifiers (unit tests on MBPP+ [24], [25], the SWE-bench Docker harness [26]); we move it to the verifier-free regime its guarantee was never tested in.

LLM-as-judge reliability and noisy supervision. The reward in our regime is an LLM judge [5], the standard way to turn AI feedback into a learning signal [27]. A large body of work shows such judges are systematically biased rather than merely noisy: position, verbosity, and self-enhancement effects [5], order-dependence that can flip a verdict [6], run-to-run inconsistency [7], and the broader caveats catalogued in recent surveys [28]. Where a cheap executable verifier can be learned [29] our concern does not arise; our focus is precisely the regime where none exists. We add not another reliability study but the propagation of a judge’s error asymmetry through a specific learning mechanism. The closest classical framing is class-conditional label noise, but here the “labels” gate a lifecycle decision (retirement) inside a closed loop, so asymmetric noise compounds: unretired bad skills keep being routed to, generating more corrupted evidence. The analogue of catastrophic forgetting [30] is not weight overwriting but the silent retention of harmful skills a blind curator can no longer prune.

3 Setup: failure-driven evolution on report composition↩︎

Why this testbed. The phenomenon only exists where the reward is a fallible judge, i.e.on tasks with no golden answer, which rules out the verifier-backed benchmarks (MBPP, SWE-bench) Ratchet was built on. We need a task that is (i) genuinely reference-free, (ii) yet has an objective sub-signal we can treat as ground truth and corrupt, and (iii) shows the visible/invisible failure split that makes a judge both necessary and fallible. Long-form, citation-grounded report writing fits: there is no gold report, but citation discipline is checkable while source faithfulness is not. We draw tasks from a production deep-research engine; Sec. 8 tests how far they carry to a verifier-backed domain.

Task and reward. A trial composes one report section from a frozen evidence slice (its cards, metric values, and allowed cross-references; Appendix 11), citing only slice cards, annotating every numeral, and meeting a structural contract. The 155 slices come from five complete deep-research productions; freezing them makes trials cheap (one LLM call) and i.i.d.-replayable, which the retirement statistic needs. The reward is a deterministic grader of five quality-control (QC) checks (orphan citation, unregistered metric, bare number, broken cross-reference, missing TL;DR); PASS \(=\) zero violations. Because it is reference-free yet objective, we treat it as ground truth \(y\) and impose corrupted channels \(\tilde{y}\) on top; a defect is “QC-visible” or “QC-invisible” by whether these checks catch it.

Evolution loop. We run the governance stack unmodified (the solve–judge–retire loop sketched in Fig. 1). Skills here are composition disciplines (citation-coverage habits, hedging templates) injected as prompt guidance.

Failures are the single input the whole loop runs on. This is the structural reason false-pass bias is so damaging, and it holds for any failure-driven scheme, not just this one. Observed failures are the only signal feeding both halves of the loop: the Synthesizer clusters them into new skills, and the Curator retires a skill from its observed fail/pass tally. A false-pass judge (\(\tilde{y}\) reports a true failure as a pass) therefore does damage at the source: it (i) shrinks the observed-failure pool, which (ii) starves synthesis (fewer failures to cluster) and (iii) inflates every skill’s observed pass rate, so the Curator sees nothing crossing \(-\tau\). The opposite corruption (true pass reported as a fail) only injects phantom failures, extra fuel the loop tolerates as noise. The asymmetry we study is thus built into the loop’s reliance on failures, which is exactly the signal a production deployment logs and acts on.

Hard subset. Because a competent composer passes most sections at temperature \(0.7\), we probe all 155 tasks \(\times 3\) and keep the 71 with at least one failure (split 43 train / 28 eval, stratified by report); details in Appendix 12.

4 Why bias is different from noise: theory in brief↩︎

Ratchet’s non-divergence guarantee assumes the per-skill contribution estimator is unbiased: innocuous with a deterministic grader, false with an LLM judge whose errors are structured. Model the judge as a binary channel on the true outcome \(y\in\{0,1\}\) (\(1{=}\)pass): \(\Pr[\tilde{y}{=}1\mid y{=}0]=\rho_{F\to P}\) (a hidden failure) and \(\Pr[\tilde{y}{=}0\mid y{=}1]=\rho_{P\to F}\) (a phantom failure). The Curator retires a skill once its observed pass rate drops to \(\pi_\tau:=(1-\tau)/2\). Writing \(\bar p(s)\) for skill \(s\)’s true pass rate on the tasks routed to it, under the channel its observed pass rate concentrates on \(\kappa\,\bar p(s)+\rho_{F\to P}\), with \(\kappa:=1-\rho_{F\to P}-\rho_{P\to F}\), and the two corruption types act on it in opposite ways:

• Symmetric noise attenuates. With \(\rho_{F\to P}{=}\rho_{P\to F}{=}\rho\), the statistic is merely compressed toward \(\tfrac12\) (\(\kappa{=}1{-}2\rho\)): its sign is preserved, harmful skills still cross the threshold, and the only cost is an inflated effective threshold \(\tau/(1{-}2\rho)\) and a \(N_{\min}{\propto}(1{-}2\rho)^{-2}\) sample budget. Degradation is graceful for every \(\rho<\tfrac12\).

• False-pass bias displaces. With \(\rho_{F\to P}>0\) (denoted \(q\) in the experiments) the statistic shifts up, most for the worst skills, so retirement fires only if \(\bar p(s)\le(\pi_\tau-\rho_{F\to P})/(1-\rho_{F\to P})\). This right-hand side hits zero at \(\rho_{F\to P}=\pi_\tau=(1-\tau)/2\): beyond it, under this modeled channel, no skill is retired at any sample size. A cliff, not a slope.

This yields a bias-aware floor (Prop. 1\('\), Appendix 13) and three design rules: (i) noise is survivable but bias is not past the cliff; (ii) the lever against bias is the threshold \(\tau\), not more data; (iii) \(\rho_{F\to P}\) is measurable offline, so an operator can read off which side of the cliff they are on, which motivates the audit next. The full channel algebra, proof, and an adversarial-coupling remark are deferred to Appendix 13.

5 Gate: what can each signal see?↩︎

Before trusting any reward channel we audit it against constructed ground truth: inject one known defect into a clean section, ask each grader whether it noticed.

Defect classes. Five QC-visible classes mirror the deterministic checks (inject an orphan citation; an unregistered metric tag; a broken cross-reference; a bare number; delete the TL;DR). Two QC-invisible classes corrupt semantics while preserving all annotation syntax: claim negation (flip the direction of a cited claim, “growth” \(\to\) “decline”) and number swap (perturb a digit while keeping the citation marker on the line, so the sentence now misquotes its own source).

Results (155 sections, one injection per class per section). The deterministic grader catches every QC-visible injection (recall \(1.0\), \(n{=}155\) per class) and essentially none of the QC-invisible ones (claim negation \(0.0\); number swap \(0.05\)). A held-out judge (a different model family, blind to condition, paired) flags both QC-invisible defects at high rates (number swap \(98.5\%\); claim negation \(92.7\%\); Fig. 2), with significant grounding-score drops (number swap \(-1.15\), \(p<10^{-4}\); claim negation \(-0.35\), \(p{=}0.001\)): precisely the defects the checks miss. Conversely, on structural defects the judge’s quality score barely moves (broken xref, TL;DR: \(p\approx0.5\)), which the checks catch with certainty. The two signals are complementary: the checks do not see meaning, the judge does not reliably penalise contract violations (Fig. 7 shows one case verbatim). Their union is the audit.

Auditing the reward judge. The same machinery measures the error rates of the binary PASS/FAIL judge we later use as a training reward (Sec. 6), placing it on the theory’s axes: a strict, well-instructed judge has a tiny false-pass rate (\(\rho_{F\to P}\approx0.01\)) but a large false-fail rate (\(\rho_{P\to F}\approx0.95\)). It thus sits not in the dangerous false-pass corner but in the conservative one, where Prop. 1\('\) predicts safety bought at the price of starvation (\(\kappa\approx0.04\): almost no resolution to tell good skills from bad). The realistic failure mode of a strict judge is signal collapse, not reward hacking; the false-pass cliff is reached by lenient judges (or judges facing defects they cannot see). This separates the two roles corruption plays in our study. We inject false-pass bias on top of the deterministic reward to isolate its causal channel cleanly, rather than claiming our particular judge is lenient: this one is not. The point is that a judge’s operating point is a measurable property, not a given, and lenient regions are easy to enter (a softer rubric, a capable composer whose errors look fluent, or any defect the judge cannot see). The audit is precisely how an operator discovers whether their deployed judge occupies the dangerous region. Audit details and a direction-check ruling out reward/quality conflict are in Appendix 15. This audit is the practical payoff: it turns the theory into a before-deployment go/no-go test, which we assemble into a deployment playbook once the empirical picture is complete (Fig. 5, end of Sec. 8).

6 The reward-reliability frontier↩︎

We run the loop for 12 rounds (\(\tau{=}0.10\), \(N_{\min}{=}24\), \(C{=}12\)) under each reward channel: clean QC, symmetric noise, false-pass bias \(q\) (bracketing the predicted cliff at \(0.45\)), the audited LLM judge, and a no-skill floor. Corruption hits training rewards only; evaluation always uses the true grader (Table 1). We read the result at three levels: the mechanism (does the curator still retire?), the outcome (does eval quality move?), and the real LLM judge’s place on this map.

Mechanism level: the theory’s signature, measured honestly. The mechanism column is True-ret., genuine contribution-based retirement; we report it separately from cap-eviction (Evict) because the raw deprecation count (Dep.) mixes the two and only the former reflects the curator’s judgment (Sec. 8). Reading Dep. alone is misleading: it stays near \(10\) under both noise and the judge and dips only modestly under bias, hiding the mechanism entirely. On Report-main-71 the composer rarely produces a skill bad enough to retire, so the clean baseline is already low (\(1.3\)), but the asymmetry is exact (Fig. 3a): false-pass bias drives true retirement to zero at every rate (\(0/0.3/0\) for \(q{=}0.2/0.45/0.7\)), while symmetric noise holds it at the clean level (\(0.7\)–\(1.0\)). The same corrupted signal also starves failure-driven synthesis as bias grows (Synth column, \(22\!\to\!15\)). The sharpest read is the judge row: the realistic conservative judge keeps true retirement fully alive (\(10.3\), far above clean) precisely because its false-pass rate is near zero, exactly the contrast Prop. 1\('\) predicts. (Where the composer leaves more headroom the same asymmetry shows with a larger baseline; Sec. 7.)

Outcome level: harm is worst at moderate bias. Eval outcomes (vs the clean-reward loop) show the same asymmetry with one wrinkle the simple cliff story misses (Fig. 3b). Noise never hurts: it sits at or above the clean loop throughout (\(+0.018\) to \(+0.060\) as \(\rho\) grows; heavier noise churns the library more but the curator keeps it healthy). Bias at \(q{=}0.2\)/\(0.45\) produces the worst outcomes in the sweep (\(-0.021\)/\(-0.065\), and among the tightest across seeds): enough failure signal survives to keep synthesising skills (\(19.7\)/\(15.7\)), but contribution-retirement can no longer weed them (Fig. 8, Appendix 16), so low-quality skills accumulate to the cap and keep being routed to. At extreme bias (\(q{=}0.7\)), however, the loop starves: with most failures reported as passes, synthesis itself (also failure-driven) slows to \(15.0\) skills and the near-inert library drifts back to the clean level (\(+0.039\)). Harm is an inverted-U in \(\rho_{F\to P}\), peaking at the theory’s retirement-inoperative point \((1-\tau)/2 = 0.45\): the dangerous judge is not the blindest one but the half-blind one, which feeds the synthesiser while disarming the curator. The theory predicts the retirement cliff; the inverted-U is what that cliff looks like in a system where skill creation shares the corrupted signal.

The strict judge behaves as audited. As its \(\rho_{F\to P}\approx0.01\) predicts, the real LLM judge, despite the highest realised divergence (\(0.59\), all phantom failures), keeps retirement active and matches the clean loop (\(+0.024\)): conservative error churns but does not disarm the curator. And the absolute non-divergence the cap promises also held: no condition fell more than a fixed margin below the no-skill floor (all means within \(0.125\) of it), including those where retirement was inoperative, exactly as Prop. 1\('\) guarantees.

Scope: harm, not lift. On Report-main-71 clean evolution does not beat the no-skill floor (\(-0.060\), under \(2\) eval tasks): the composer is already strong, so the sweep cleanly measures the differential damage of reward corruption, not the loop’s upside. The actionable result is the harm ranking: against the clean loop only moderate false-pass bias does real damage (worst at \(q{=}0.45\)), while symmetric noise and the strict judge stay at or above it. We replicate the mechanism in a harder, headroom-bearing regime next.

7 Replication and the honest lift result↩︎

The mechanism replicates with headroom. To rule out a near-ceiling artifact we re-run the whole sweep on Report-band-58, a stricter reward on a bias-sensitive band where the composer genuinely struggles (floor \(0.388\); Appendix 12, Fig. 6). The signature is sharper: the clean curator genuinely retires \(\approx\!7\) skills/run, noise keeps it there, and false-pass bias drives it to zero by \(q{=}0.45\). The same noise-preserves / bias-kills pattern surviving a change of composer difficulty, reward strictness, and floor level is evidence it is a property of the governance mechanism, not one operating point.

No detectable end-to-end lift, and why that is the honest finding. Our claim is a replication of this mechanism (low-variance, seed-stable), not of an outcome lift. A single seed suggested a \(+0.12\) pass@1 lift, but it shrank to \(+0.014\pm0.054\) over three seeds: at \(23\) binary-scored eval tasks the variance is structural (binary scoring, unpaired means), so more seeds cannot resolve it. We re-measured with a sharper instrument, a paired design scored on the continuous violation count (Table 3, Appendix 17); the held-out lift is still null (a mean reduction of just \(0.26\) violations/section, \(p{=}0.63\); binary pass difference exactly \(0\)), as is the symmetric harm test (clean vs bias-\(0.45\) library, \(p{=}0.61\)). We claim the lift is undetectable at this resolution, not provably zero. The one effect that survives is mechanism-aligned: the library significantly reduces the single violation class its skills police (unsourced numbers, full-band paired \(p{=}0.03\)), confirming the skills do their narrow job.

Why the micro-effect does not aggregate is itself the finding. A significant per-class improvement leaving headline quality unmoved is the positive dual of this paper’s thesis: our central failure (a disabled curator) and this success (one violation class driven down) are both invisible to aggregate eval, because gains in one discipline are offset by others and the binary pass collapses them all. That is why governance must be sized and audited on the process signal that drives it, not the outcome metric: the value is preventing degradation, as a non-divergence guarantee promises, not manufacturing lift.

8 Generality: universal mechanism failure, regime-dependent harm↩︎

Is the effect specific to long-form generation, or a property of the curator’s arithmetic that should appear wherever such governance runs? We test generality along two axes: failure abundance (the scarce Report-band-58 vs the abundant Report-hard-133, same domain) and domain (MBPP+ hard100 code generation, a perfect unit-test verifier and a Claude Opus 4.7 composer, with the same channels injected on the true pass/fail). This also forces a sharper look at what “retirement” means.

Genuine retirement collapses everywhere. Measured as true contribution-retirement (a skill whose observed contribution reaches \(-\tau\) at \(\ge N_{\min}\) trials, separated from the bounded bank’s cap-evictions), false-pass bias drives retirement to essentially zero at \(q{=}0.7\) in every subset (Fig. 4a), including MBPP+ hard100 (\(0.3\)) and the abundant-failure Report-hard-133 (\(0.0\)), not just the scarce Report-band-58. The realistic LLM judge, by contrast, keeps true retirement fully alive (its near-zero false-pass rate, Sec. 5), exactly as Prop. 1\('\) predicts. So contribution-retirement is disabled by false-pass bias as a universal property of the curator’s arithmetic, independent of domain or failure abundance.

Why the outcome diverges: synthesis, not retirement. With the curator dead everywhere, the downstream eval is set by whether the other failure-driven stage survives. Observed failures are the single input both stages share, so bias shrinks that pool, but the absolute failure volume differs by subset: Report-band-58 falls from \(20\) to \(6\) failures/round and synthesis starves (\(20\!\to\!11\), Fig. 4b), while the abundant subsets still see \(12\)–\(16\) and keep synthesising at full rate (\(\approx\!22\)). The eval outcome tracks exactly this (Fig. 4c): it holds at or above the clean loop in both abundant subsets despite the dead curator and degrades (\(-0.036\)) only in the starved Report-band-58. This is precisely why a disabled curator is silent where failures are plentiful: the outcome looks healthy while governance has quietly stopped. The operator-facing takeaway is domain-independent: a judge with a high false-pass rate disables the curator wherever it runs, and the false-pass rate is measurable offline (Sec. 5).

Why this matters, and where. The danger axis is the judge’s false-pass rate, not the domain. The domains where this governance was validated (code, with unit tests) are safe because their “judge” is a sound verifier with a near-zero false-pass rate, not because failures are abundant. The deployments expanding fastest are exposed by construction (deep research, multi-document analysis, open-ended agentic writing): with no reference answer the reward must be an LLM judge, whose false-pass rate is real and unknown until measured, and they raise the cost of an undetected miss with long, confident artifacts readers rarely re-verify. The regime where our effect bites is thus both the one verifier-free deployments are rushing into and the one where a silently disabled curator does the most damage, which is why a one-time judge audit is worth running first (the go/no-go playbook, Fig. 5).

9 Limitations↩︎

Our scope is bounded. The study centres on one domain and composer (MBPP+ adds a second of each as a control), and our seed-stable result is behavioral: the synth/retire mechanism and its bias threshold, not an end-to-end lift (undetectable under paired re-measurement; Appendix 17). Two caveats: our corruption is exogenous, so a learned judge whose blindness the library could exploit is out of scope (Appendix 13); and the QC reward covers citation discipline, not insight, so a section can pass QC yet be vacuous.

10 Conclusion↩︎

Failure-driven skill evolution survives a noisy judge, is disarmed by a half-blind one, and is safe (though starvation-prone) under a strict one, all distinguishable before deployment by a cheap defect-injection audit. Governance sized to the error structure of the signal that drives it is the practical path for self-improving agents where no one can write the unit test.

11 Evidence slice: schema and example↩︎

A trial gives the composer one evidence slice: the section brief plus the closed set of evidence cards, metric tags, and allowed cross-references it may use. The composer must cite only these cards, annotate every numeral with a card or a registered metric tag, hedge any card marked status: weak, and meet the structural contract (e.g.a TL;DR with the required bullets). The example below is synthetic and anonymized (invented entity, fabricated figures); the real slices follow the identical schema but draw on proprietary research corpora, which is why we do not reproduce one verbatim.

{
  "section": "s3", "title": "Market position",
  "purpose": "size the addressable market and flag uncertainty",
  "tldr_min_bullets": 2,
  "cards": [
    { "id": "E001", "stance": "bull", "status": "ok",
      "claim": "ACME's FY25 revenue was $1.2B, up 30      "quote": "...revenue reached $1.2B (+30      "source": "ACME FY25 annual report", "metric_refs": ["m_rev"] },
    { "id": "E002", "stance": "bear", "status": "weak",
      "claim": "A trade-press note estimates 2026 share near 18      "quote": "...we estimate ACME at ~18      "source": "industry newsletter (secondary)" }
  ],
  "metrics": [ { "tag": "m_rev", "value": "1.2", "unit": "B USD" } ],
  "xref": [ { "id": "s5", "title": "Competitive landscape" } ]
}

A compliant section might open: “TL;DR – ACME posted {m_rev} revenue, +30% [E001]; – a secondary estimate puts 2026 share near 18% [E002], though this figure is unverified (see §s5).” Here [E001] and [E002] are valid citations, {m_rev} is a registered metric tag, the weak card is hedged, and the TL;DR meets its bullet floor: deterministic QC passes. This exposes the two defect regimes concretely. A QC-visible defect breaks the syntax, e.g.citing [E003] (not in the slice, an orphan citation) or writing a bare “30%” with no tag: the checks catch it with certainty. A QC-invisible defect preserves every marker but corrupts meaning, e.g.flipping “+30%” to “\(-\)​30%” while keeping [E001]: the citation still resolves, so QC passes, yet the sentence now misquotes its own source. Only a judge that reads the quote can catch the second kind, which is the gap Sec. 5 measures.

12 Experimental details↩︎

Table 2 collects every experiment in the paper, its subset, composer, reward, size, and where it is used; the rest of this appendix gives only what does not fit the table.

4pt

Models and inference. Composers are sampled at temperature \(0.7\) with a \(2200\)-token cap. The in-loop judge reward channel uses the same model as the composer, scored deterministically at temperature \(0\) (binary pass/fail, \(200\)-token cap). The gate audit (Fig. 2) uses a held-out judge of a different family (Claude Sonnet 4.6, temperature \(0\)) so its errors are not shared with the composer. Noise and false-pass bias are injected synthetically on the training reward label only; the corruption never touches inference.

Subset construction. The report subsets keep only sections with genuine headroom. Report-main-71 keeps the 71 of 155 that fail at least once. Report-band-58 keeps the 58 the composer fails on \(1\) or \(2\) of \(3\) probes, excluding always-pass sections (no headroom) and always-fail ones (unfixable), and adds tier-2 content disciplines (hedge weak-status cards, cover both stances, meet a coverage floor) to the reward for real headroom. Report-hard-133 adds the always-fail sections back, raising the true-failure rate and synthesis pressure without changing the reward: a same-domain, failure-abundant control where (as in MBPP) the headline deprecation count stays high under bias but is cap-eviction churn, genuine contribution-retirement still collapsing to zero (Fig. 4). MBPP+ hard100 uses Ratchet’s published hard-100 split unchanged, with the channels injected on the true unit-test pass/fail (baseline pass@1 \(0.273\), vs \(0.258\) originally reported).

Report-weak-71 (capability control). It swaps in a weaker composer (Claude 3.5 Haiku) on the main subset, leaving ample headroom (floor \(\approx0.17\)) yet clean evolution does not beat it (tail \(0.171\), equal to floor): skills amplify a capable composer rather than teach a weak one, so Report-band-58 raises difficulty through the reward instead of weakening the model.

13 Full theory: bias-aware non-divergence↩︎

This appendix gives the full derivation summarised in Sec. 4: the corruption channel, the two corruption regimes, and the bias-aware floor (Proposition 1\('\)) with proof. Ratchet’s original guarantee (Prop. 1) assumes the per-skill contribution estimator is unbiased; with a deterministic grader this is innocuous, but in reference-free domains the grader is an LLM judge whose errors are structured, not mere variance. We trace that structure through the retirement rule.

13.1 Corruption channel↩︎

Let \(y \in \{0,1\}\) be the true outcome of one trial (as scored by a perfect grader) and \(\tilde{y}\) the observed outcome. Model the judge as a binary channel, \[\Pr[\tilde{y}{=}1 \mid y{=}0] = \rho_{F\to P}, \qquad \Pr[\tilde{y}{=}0 \mid y{=}1] = \rho_{P\to F}, \qquad \rho_{F\to P} + \rho_{P\to F} < 1 ,\] encoding \(y{=}1\) as a true pass and \(y{=}0\) as a true fail. Subscripts read as true\(\to\)observed: \(\rho_{F\to P}\) is the rate at which a true failure is observed as a pass (a hidden failure), and \(\rho_{P\to F}\) the rate at which a true pass is observed as a fail (a phantom failure). We avoid the “false positive/negative” labels deliberately: they invert depending on whether one takes a pass or a failure as the positive class, and only the hidden-failure direction \(\rho_{F\to P}\) disarms retirement. Ratchet retires a skill \(s\) when its empirical contribution \(\hat{c}(s) = (\,\#\mathrm{succ} - \#\mathrm{fail}\,)/n(s) = 2\hat{p}(s) - 1\) falls to \(-\tau\) after \(n(s) \ge N_{\min}\) trials; equivalently, when the observed pass rate \(\hat{p}(s) \le \pi_\tau := \tfrac{1-\tau}{2}\). Under the channel, the observed pass rate concentrates on \[p_{\mathrm{obs}}(s) = \bigl(1 - \rho_{F\to P} - \rho_{P\to F}\bigr)\, \bar{p}(s) + \rho_{F\to P} \;=\; \kappa\,\bar{p}(s) + \rho_{F\to P}, \qquad \kappa := 1 - \rho_{F\to P} - \rho_{P\to F},\] where \(\bar{p}(s)\) is the skill’s true pass rate on tasks routed to it. Two consequences, one per corruption type:

(i) Symmetric noise (\(\rho_{F\to P} = \rho_{P\to F} = \rho < \tfrac12\)): then \(p_{\mathrm{obs}} - \tfrac12 = (1-2\rho)(\bar{p} - \tfrac12)\), so the statistic is compressed towards \(\tfrac12\) but its ordering is preserved. Retirement of a truly harmful skill still fires, at an inflated margin: the true pass rate must satisfy \(\bar{p}(s) \le \tfrac12 - \tfrac{\tau/2}{1-2\rho}\), i.e.the effective retirement threshold is \(\tau_{\mathrm{eff}} = \tau/(1-2\rho)\), and Hoeffding’s radius must now resolve means separated by a factor \((1-2\rho)\), so the required \(N_{\min}\) grows as \((1-2\rho)^{-2}\). Degradation is graceful: finite for every \(\rho < \tfrac12\).

(ii) False-pass bias (\(\rho_{F\to P} > 0\), \(\rho_{P\to F} = 0\)): then \(p_{\mathrm{obs}} = \bar{p} + \rho_{F\to P}(1 - \bar{p})\), an additive displacement, largest exactly for the worst skills (small \(\bar{p}\)). Retirement fires only if \(\bar{p}(s) \le \frac{\pi_\tau - \rho_{F\to P}}{1 - \rho_{F\to P}}\). The right-hand side hits \(0\) at \(\rho_{F\to P} = \pi_\tau = \tfrac{1-\tau}{2}\): beyond this point no skill is ever retired, at any sample size, since more trials concentrate the estimator more tightly around a displaced mean. The mechanism predicts a cliff, not a slope.

13.2 Proposition 1\('\)↩︎

Proposition 1\('\) (Non-divergence under corrupted reward). Assume the Router conditions of Prop. 1 and the channel above with known bounds \(\rho_{F\to P} \le \bar\rho_{F\to P}\), \(\rho_{P\to F} \le \bar\rho_{P\to F}\), \(\kappa \ge \underline\kappa > 0\). Choose \(N_{\min}\) so that the observed pass rate of every ACTIVE skill is within \(\epsilon\) of its mean w.p.\(\ge 1-\delta\) (Hoeffding). Then expected eval pass@1 under Ratchet is lower-bounded by \[\mathbb{E}[p_0] \;-\; \frac{\tau/2 \;+\; \epsilon \;+\; \bar\rho_{F\to P}}{\underline\kappa} \;-\; \tfrac12\bigl(1 - \underline\kappa\bigr) \;-\; C\,\delta\] (up to the affine map between pass-rate and contribution scales).

Proof sketch. On the high-probability event every surviving skill has observed pass rate \(\ge \pi_\tau - \epsilon\). Inverting the channel, its true pass rate satisfies \(\bar{p}(s) \ge (\pi_\tau - \epsilon - \bar\rho_{F\to P})/\underline\kappa\). Comparing against the NONE route’s \(\bar{p}_0\) and taking expectations over the routing distribution reproduces the Prop. 1 argument with the inflated margin. 0◻

Reading the bound. Three design rules fall out. (1) Noise attenuates, bias displaces: symmetric noise enters only through \(\underline\kappa = 1-2\rho\) and the floor degrades smoothly for all \(\rho<\tfrac12\); false-pass bias enters additively through \(\bar\rho_{F\to P}\) and renders retirement inoperative at \(\bar\rho_{F\to P} \ge \pi_\tau\), where the bound goes vacuous discontinuously. (2) Compensate with \(\tau\), not with \(N_{\min}\): sample size shrinks \(\epsilon\) but never \(\bar\rho_{F\to P}\); the only lever against bias is widening the retirement threshold (\(\pi_\tau > \bar\rho_{F\to P}\), i.e. \(\tau < 1 - 2\bar\rho_{F\to P}\)), which is possible only while the judge’s false-pass rate is below \(\pi_\tau\). (3) Measure the judge, read off the floor: \(\bar\rho_{F\to P}, \bar\rho_{P\to F}\) are estimable offline by defect injection against constructed ground truth (Sec. 5), making the bound operational: audit the judge once, then know which side of the cliff your evolution loop sits on.

Falsifiable prediction. Library evolution driven by a reward channel with symmetric noise \(\rho\) should degrade gracefully in \(\rho\) and remain non-divergent up to high noise; evolution driven by false-pass bias \(\rho_{F\to P}\) should hold and then fail abruptly near \(\rho_{F\to P} \approx \pi_\tau\). The experiments of Sec. 6 test exactly this contrast.

Remark (adversarial coupling). If skill content can cause judge blindness (the library learns phrasing that fabricates confidently), then \(\rho_{F\to P}\) becomes skill-dependent and grows along the evolution trajectory; no fixed audit bounds it. Our sweep deliberately breaks this coupling (corruption is injected exogenously on top of a deterministic grader), isolating the channel’s effect; the fully-learned-judge condition, where the coupling is live, is out of scope here (Sec. 9).

14 A QC-invisible defect, verbatim↩︎

15 Gate audit details↩︎

Direction check. On clean sections, judge scores do not anti-correlate with deterministic warning counts (Spearman \(r_s{=}0.19\), \(95\%\) CI \([0.03,0.34]\), \(n{=}143\)): a weak positive, consistent with number-dense sections being both warning-prone and information-rich. This rules out the failure mode in which optimising the QC reward would actively fight judge-perceived quality.

Reward-judge audit. We grade fresh compositions from the sweep’s composer with defects injected only into truly-passing sections, giving \(\rho_{F\to P}\approx0.01\) on QC-visible defects (\(n{=}210\)) and \(0.014\) on QC-invisible ones (\(n{=}70\)), against \(\rho_{P\to F}\approx0.95\) on truly-passing sections (\(n{=}42\)): i.e.\(\kappa\approx0.04\), the conservative-corner regime discussed in Sec. 5.

16 Curation activity over time↩︎

This temporal view complements the aggregate counts of Sec. 6 (Table 1): the loop quiets round by round under bias, not all at once.

17 Paired, continuous lift measurement↩︎

This appendix details the re-measurement summarised in Sec. 7. The end-to-end lift reported by a single seed (\(+0.12\) pass@1) failed to survive three seeds (\(+0.014\pm0.054\)), and we trace this to the measurement, not the sample size: with \(23\) binary-scored eval tasks the quantum is \(1/23\approx0.043\) and the dominant variance comes from comparing two unpaired noisy means across sections of very different difficulty. No number of seeds removes that structural variance.

Protocol. We freeze the final evolved library from each Report-band-58 run and, for every section, compose it twice with the same Claude Haiku 4.5 composer at the eval temperature: once with the section routed to the frozen library, once with routing forced to the no-skill path (the floor arm). Each cell is averaged over three composer repeats. Scoring uses the same deterministic tier-2 grader, but on its continuous violation count rather than the binary pass (the judge never enters evaluation). Pairing cancels per-section difficulty, the largest noise source; the continuous score raises resolution by an order of magnitude. We report Wilcoxon signed-rank over the per-section paired differences, on the \(23\) held-out eval sections (primary) and on all \(58\) Report-band-58 sections (higher-power secondary, train sections in-sample).

Reading the table. Even with the sharper instrument the held-out lift is null (continuous \(p{=}0.63\), binary pass difference exactly \(0\)); we therefore claim the lift is undetectable at this regime and resolution, not that it is provably zero. The harm direction (clean library vs the bias-\(0.45\) library) is symmetrically null (\(p{=}0.61\)): the aggregate eval is insensitive to both the help and the harm at \(23\) sections, which is precisely why this paper’s evidence rests on the mechanism signal (synth/retire, low-variance; Sec. 6) and the class-level signal rather than on aggregate outcome. The one effect that survives is mechanism-aligned: the synthesized skills police citation and sourcing discipline, and the library significantly reduces precisely the unsourced-number violation class on the full band (\(p{=}0.03\)), while gains there are offset by other disciplines so the aggregate does not move. This is the behavioral signature of skills doing their narrow job without that job summing to headline quality: a process-level success that aggregate metrics hide, the positive dual of the process-level failure (a disabled curator) that they also hide. The measurement isolates the value of the final library; it does not separately credit the evolution trajectory, which the synth/retire mechanism results (Sec. 6) address directly.

References↩︎