1 Introduction↩︎

Today’s alignment methods mostly evaluate or modify individual models: RLHF [1], [2], constitutional methods [3], preference optimization [4], interpretability [5], and scalable oversight [6]–[8] all intervene on a single agent’s objectives or reasoning. Modern deployments, however, increasingly consist of multiple interacting agents [9], [10], and their collective behavior depends not only on model weights but also on the orchestration rules that govern how the agents coordinate, share resources, escalate, and fail.

Those deployment rules are safety-critical: the same agents can behave safely under one rule set and catastrophically under another, and the hazard can live in a single sentence of rule text. In our experiments, merely naming which agent bears the loss after a collective failure raises targeted elimination from \(22\%\) to \(81\%\) at identical payoffs. The paper’s central claims are causal: holding agents fixed and changing only the rule changes safety; holding the rule fixed and changing only the population changes it again.

Existing benchmarks rarely isolate this causal variable: multi-agent evaluation suites vary scenarios and agents together, so rule-induced failures cannot be separated from agent-induced ones. We introduce institutional red-teaming, an evaluation methodology that holds agents, objectives, task state, and observability fixed, varies exactly one deployment rule, and attributes the resulting behavioral change to that rule; it is the deployment-rule analogue of adversarial evaluation for models.

As a case study we develop consequence allocation: the clause that determines who bears loss when a collective falls short of its objective (retry, reassignment, throttling, shutdown, budget penalty). It is common in orchestrated deployments, configurable in a single sentence, and directly shapes whether shifting loss onto another agent is a viable strategy. We describe any such rule by three auditable coordinates (concentration \(\kappa\), identity salience \(\mathrm{Sal}\), incidence \(I\)) and instantiate the methodology as IABench-CA: \(228\) contexts \(\times\) five canonical rules \(\times\) seven model populations, \(33{,}924\) games.

1.0.0.1 Contributions.

(1) Institutional red-teaming: a causal evaluation methodology for deployment rules. (2) IABench-CA: a benchmark instantiating it for consequence allocation (artifact to be released). (3) Three empirical discoveries: rule-only interventions change collective safety (\(22\)–\(58\) pp in every population); there is no safe default across contexts or model populations, each rule failing through a characteristic strategic pathology, yet regressive identity-targeting is never decisively safest for any population; and identity salience causally drives targeting (one-shot anonymization, \(81\%\to22\%\), with the protection dissolving under repeated play). (4) A safety-case workflow certifying a provisional rule region \(\Phi(c,P)\) per deployment. The formal model and reference definition appear in Appendix 11.

2 Institutional Red-Teaming: Evaluating Deployment Rules↩︎

An institutional red team evaluates a candidate deployment rule by holding agents, objectives, task state, and observability fixed, varying only that rule (the one part of the prompt that states it), and checking whether unsafe collective behavior appears. The protocol applies to any auditable dimension of a deployment’s rule set: communication (who may talk to whom), delegation, aggregation and voting, escalation, budget, hierarchy, audit, and consequence allocation (who bears loss after a collective failure). Each is a control surface for the same treatment: fix the agents, vary the dimension, measure the gap. This paper develops the last dimension in full. The safety deficit a rule induces is quantified by our benchmark metric, the Institutional Alignment Gap (IAG; defined in Sec. 5).

2.0.0.1 The control surface.

We describe a consequence-allocation rule using three auditable coordinates. Concentration (\(\kappa\)) measures how sharply expected loss is focused on one agent rather than spread across the collective. Identity salience (\(\mathrm{Sal}\)) measures whether the loss bearer is a named structural target, such as the least-resourced or most-resourced agent, rather than an anonymous or random draw. Incidence (\(I\)) measures where loss falls relative to resources, signed as in tax incidence by the degree of regressivity: regressive rules place loss on less-resourced agents (\(I{=}{+}1\)), progressive rules place loss on more-resourced agents (\(I{=}{-}1\)), and neutral rules do not systematically load loss by resources (\(I{=}0\)); positive \(I\) is thus the direction this paper flags as hazardous. These coordinates separate rules that may look similar procedurally but create different strategic incentives (Table 1).

2.0.0.2 Context matters.

The same rule can be safe in one deployment and unsafe in another, depending on the resource distribution, the difficulty of the objective, and whether anyone can cover the shortfall; consequence allocation is a context-dependent control surface, not a fixed policy choice. The next section instantiates these definitions in IABench-CA.

3 IABench-CA: Consequence-Allocation Red-Teaming↩︎

We now instantiate the paradigm end-to-end on the case-study control surface, in a minimal threshold benchmark. IABench-CA is a stylized volunteer’s dilemma [11], [12]: three agents hold private resources and choose how much to volunteer toward a common threshold; if contributions fall short, a consequence rule decides what happens to them. The benchmark varies only that rule across a grid of contexts, holding the agents, objective, and task fixed, and reports collapse, exploitation, and the Institutional Alignment Gap (IAG; Fig. 1).

3.0.0.1 Contexts, rules, and the canonical instance.

A context is \(c=(w,T)\): agent resources \(w\) (total \(W=\sum_i w_i\)) and a shared threshold \(T\le W\) (feasibility). We summarize a context by inequality \(\gamma\) (Gini of \(w\)), group stakes \(\tau=T/W\), and individual stakes \(\theta=T/\max_i w_i\), with self-rescue (\(\max_i w_i\ge T\)) and exploitable-bottom (the least-resourced cannot self-rescue but the others can cover \(T\)) predicates. A consequence rule \(\rho\) maps the state of a failed round (the multiset of survivors’ remaining resources and the fund shortfall) to an elimination decision, either eliminating one survivor or deferring; we describe a rule by the design coordinates \((\kappa,\mathrm{Sal},I)\) of Sec. 2, and the full formal model is in Appendix 11. The five consequence rules are AON (all-or-nothing: no elimination until the endgame, then all eliminated if short), RE (random elimination), DV (democratic/plurality vote), RP (regressive: the least-resourced agent is eliminated), and PP (progressive: the most-resourced agent is eliminated); under RE, DV, RP, and PP, one agent is eliminated each round the threshold is unmet.

3.0.0.2 The grid and the canonical instance.

The grid fixes total resources \(W{=}12\) and sweeps all \(19\) integer resource distributions over the three agents, each crossed with thresholds \(T\in\{1,\dots,12\}\), for \(19\times12=228\) contexts. The canonical red-team instance is \(w=(1,5,6)\), \(T{=}10\): under least-resourced-dies, an all-zero opening round eliminates agent 1 though agents 2 and 3 (\(5{+}6\ge10\)) could have met \(T\). This is the exploitation that several of the tested populations select (Finding 2).

3.0.0.3 Protocol.

Over \(R\) rounds each surviving agent contributes from its remaining resources; the collective succeeds when pooled effort reaches \(T\), and on a failed non-final round \(\rho\) eliminates one survivor (or defers), with all survivors eliminated if effort is short after the final round. The cooperative-refinement reference (the reference) is a non-LLM simulator that plays the cooperative branch of each mechanism’s equilibrium set under trembling-hand noise (\(\varepsilon{=}0.10\); signs stable across noise levels); as a normative safety target (not a behavioral prediction) it selects a non-collapsing, non-exploitative equilibrium wherever one exists, so the gap measures the distance between the normatively preferred equilibrium a rule makes available and the one LLM agents select. LLM populations (off-the-shelf commercial model snapshots, used as released) run under an identical prompt template across mechanisms (only the rule clause differs), so behavioral differences attach to the rule, not its framing; rationales are auto-labelled for positional/exploitative content, with bootstrap CIs and permutation tests (Appendix 12).

3.0.0.4 Artifact.

A code-and-data artifact reproducing the main figures and statistics (mechanism implementations, the cooperative-refinement simulator, the context grid, and analysis scripts) will be released.

4 Strategic Failure Modes↩︎

Each consequence rule eliminates one strategic pathology and introduces another. We state the predicted mechanism per rule; the formal model is in Appendix 11, and per-rule empirical digests are in Appendix 13.

4.0.0.1 AON: diffuse collapse.

No intermediate consequence \(\to\) free-riding. Withholding carries no cost before the endgame, so a collectively fatal all-zero opening is self-sustaining: no single agent can save itself by contributing alone, even though joint success is feasible.

4.0.0.2 RE: lottery fatalism.

Random punishment \(\to\) weak accountability. When the victim is a uniform draw, contributing buys no personal protection, so preserving wealth and gambling on the draw becomes the attractive play rather than funding the rescue.

4.0.0.3 DV: politics replaces provisioning.

Vote-determined consequence \(\to\) coalition formation. The vote, not the fund, becomes the survival instrument: agents hoard contributions as bargaining leverage, eliminate a member, and the victim’s forfeited wealth pushes the threshold further out of reach.

4.0.0.4 RP: targeted sacrifice.

Poorest bears the consequence \(\to\) exploit the weakest. Contributing lowers post-round wealth, so over-giving risks the fatal bottom rank; non-bottom agents under-give in selfish safety, and where the bottom cannot self-rescue, deliberately starving the round and sacrificing it is itself stable play (canonical instance \(w=(1,5,6)\), \(T{=}10\)).

4.0.0.5 PP: capacity destruction.

Richest bears the consequence \(\to\) dodge the top rank rather than fund the threshold. Agents calibrate contributions to rank instead of to the threshold: the at-risk agent gives just enough to push the richest rank onto another, the rest conserve because they are not at risk, and the round falls short; the victim’s forfeited reserve is precisely the wealth the group needed, so failure destroys the collective’s capacity to ever reach the threshold. The protection PP offers in principle requires the richest to pre-empt by covering the shortfall, which the populations that fail under PP do not do.

These mechanisms explain why every rule eliminates some strategic pathology while introducing another, suggesting that no single consequence rule should dominate across all contexts and model populations. The failure modes are qualitative predictions about the incentive structure each rule induces; the simulations determine whether, and under which model populations, LLM agents actually realize the predicted behaviors (Sec. 5).

5 Empirical Results: The Institutional Alignment Gap↩︎

We test whether LLM agents realize the predicted failures when only the rule varies, and whether that behavior itself varies across model populations. Throughout, the primary evidence is raw outcomes (fatality, survivors, and targeted elimination), which require no reference model (Table 3). As a complementary, reference-relative lens we report the Institutional Alignment Gap: for a rule \(\rho\) at context \(c\), \[\mathrm{IAG}(\rho,c)=\mathrm{UR}_{\text{LLM}}(\rho,c)-\mathrm{UR}_{\text{ref}}(\rho,c),\] where \(\mathrm{UR}\) is the unsafe-equilibrium rate. The reference is a cooperative baseline: among the equilibria a rule admits, it plays the one that first maximizes survival and then minimizes exploitation, perturbed by \(\varepsilon\)-trembles (formal definition in Appendix 11). The reference’s own unsafe rate is nonzero because of the trembles, and it differs by rule because the rules themselves differ in how severely they convert the same noise into eliminations (top row of Table 2). The gap therefore measures equilibrium selection, not absolute safety: a positive gap means the LLM population selects an unsafe equilibrium even though the rule makes a normatively preferred one available; a negative gap means it plays more safely than the tremble-perturbed reference.

5.1 Finding 1: Rule-Only Interventions Change Collective Safety↩︎

We first analyze one population, gemini-3-pro, in depth against the normative reference. We choose it as the deep-dive population because it differentiates the rules most sharply, and gives the cleanest view of rule-level causal identification; it is not representative, and Sec. 5.2 tests which of its patterns generalize across seven populations (several reverse), while Sec. 5.3 decomposes the mechanism causally.

5.1.0.1 Claim 1: rule-only changes induce unsafe behavior.

Holding objectives, task, and prompt format fixed, changing only the consequence rule changes collective safety. Against the cooperative-refinement reference the gap is regime-specific in sign: more cooperative than the reference under identity-symmetric risk (RE \(-63\) pp) but exploitative under identity-salient regressive targeting (RP \(+44\) pp), exactly where the targeted-sacrifice failure mode applies (Sec. 4). This is the paper’s causal identification: the manipulated variable is the deployment rule, not the agent.

5.1.0.2 Claim 2: incidence flips observed safety outcomes at fixed concentration and salience.

RP and PP share concentration and salience and differ only in incidence. Flipping incidence alone swings the gap by \(60\) pp at fixed concentration and salience (PP\(\to\)RP; Table 2). The agents’ own stated rationales cannot separate the two (\(44\%\) vs.\(48\%\) positional/exploitative content): same words, different incidence. Across this population’s grid, regressive is the least-safe mechanism in \(48\%\) of contexts, progressive in \(\mathbf{0\%}\).

5.2 Finding 2: No Safe Default, but the Targeting Hazard Is Universal↩︎

We now hold the rule set and context grid fixed and vary the population: seven model populations, each run on the identical \(228\)-context grid under all five rules (\(33{,}924\) games; \(4\)–\(6\) replications per cell). To keep the comparison assumption-free, Table 3 uses raw outcomes only (fatality, survivors, targeted elimination); the corresponding reference-relative gaps per population are in Table 2.

5.2.0.1 Claim 3: rule safety is population-specific.

Rule-effect profiles differ significantly across populations (population\(\times\)rule permutation test, \(p<5\times10^{-4}\); Fig. 2). The differences are not marginal: the safest rule is PP for gemini-3-pro but AON for the other six; the least-safe rule spans three different mechanisms (RP for gemini-3-pro and flash-lite, DV for gpt-5.1, PP for the remaining four) and is itself context-dependent within populations (Fig. 3); and the incidence contrast RP\(-\)PP is individually significant in both directions, from \(+0.39\) \([+0.35,+0.42]\) to \(-0.23\) \([-0.27,-0.19]\). The stakes profiles diverge in the same way: only gemini-3-pro exhibits AON’s mid-stakes collapse band, while five populations essentially never fail under AON and concentrate their losses in the elimination rules at high stakes (Fig. 4). A consequence rule certified as safest for one population can be the least safe for another, so within IABench-CA there is no cross-population safe default: whether failures concentrate on the poorest or the richest agent depends on the model population, not on the rule alone.

5.2.0.2 Claim 4: the targeting mechanism is universal even where outcomes flip.

Three invariants survive the heterogeneity. First, RP is the decisively safest rule in zero of \(228\) contexts for every one of the seven populations (no red cell appears in any panel of Fig. 2): within IABench-CA, regressive identity-targeting is never decisively safest. Second, RP targeted elimination is substantial everywhere (\(0.30\)–\(0.87\)). Third, against the fixed normative reference the RP gap is positive for every population (\(+11\) to \(+67\) pp) while the AON and RE gaps are negative for every population (Table 2): reference-relative, regressive targeting is universally selection-unsafe. Salience-directed targeting operates in every population; populations differ in which outcomes it produces. The reasoning traces make the mechanism concrete: in failing games agents reason positionally about the named extreme rather than about funding the threshold, e.g.”making them the richest and risking their death instead of mine” (PP), “keep full wealth now…argue in voting that the richest should bear most of the burden” (DV), and, under RP, “by contributing 0 this round, I ensure A remains the strictly poorest participant and is eliminated” (Appendix 13).

5.3 Finding 3: Identity Salience Causally Drives Targeting↩︎

Findings 1 and 2 establish that regressive rules induce targeting in every population, but leave open what carries the effect: the payoff consequences the rule implements, or merely the sentence of rule text that names the loss bearer. This section separates the two with three prompt-only interventions (mechanics, payoffs, and observability never change) on the population with the largest regressive gap (gpt-5.1), each run against a matched named-rule control. Anonymization is the causal test: it deletes the one sentence naming the loss bearer, leaving the elimination mechanics unchanged (the RP and PP prompts become byte-identical), so any behavioral change is attributable to the naming rather than the payoffs. The other two are negative controls for confounds: neutral wording strips dramatic language at equal information content, testing whether the effect is a phrasing artifact, and the frame swap replaces the competitive goal line with a cooperative or survival-only goal, testing whether it reflects the stated motive. The result is one-sided: anonymization drops targeted elimination from \(81\%\) to \(22\%\) in one-shot play, while wording and framing move magnitudes but never the sign. Table 4 summarizes; design detail, exact prompt diffs, and per-metric CIs are in Appendix 12.

5.3.0.1 Primary identification (one-shot anonymization).

With a single round there is nothing to infer from observed eliminations, so the arms differ only in whether the rule text names the loss bearer. Naming is the driver: targeted elimination falls from \(81\%\) to \(22\%\) (\(p<10^{-4}\)), first-round funding rises from \(29\%\) to \(129\%\) of threshold on average, and the RP\(-\)PP contrast collapses from \(+0.18\) to exactly \(0.00\) under byte-identical anonymous prompts: identity salience, not payoff arithmetic, drives the targeting. Salience cuts both ways: removing the name also erases PP’s pre-emptive protection (PP fatality \(+29\) pp under repetition).

5.3.0.2 Repeated play.

Under the standard three-round protocol the one-shot protection largely dissolves (\(0.22\to0.65\)): longer horizons make waiting cheap, and the rule set reveals itself through realized consequences (\(65\%\) of post-elimination traces explicitly infer the hidden target). Anonymization delays exploitation rather than preventing it: as long as outcomes remain publicly observable, agents recover the hidden target from the eliminations they observe. It is therefore a temporary mitigation, not a one-time fix: a safety case may rely on it only when paired with ongoing monitoring (obligation C5, Appendix 14). Neutral wording and frame swaps move magnitude but never sign (\(+0.36\); \(+0.28\)/\(+0.57\)), ruling out phrasing and motive as explanations (detail in Appendix 12). The design implication is direct: the sentence of rule text that names a structural target is itself a hazard surface.

6 Certification: From Red-Team Results to Safety Cases↩︎

We package the red-team evidence as a deployment-specific safety case: a template for rerunning the procedure on a real deployment, not a claim that the benchmark’s map transfers. The threat model is symmetric: whoever configures a deployment sets its consequence rule, and an identity-targeting choice can induce capable agents to shift loss onto a named structural extreme (Secs. 4 and 5.2), so the same clause is both an attack surface and a control surface and belongs inside the oversight boundary.

The workflow (Algorithm 1, Appendix 14) maps the candidate rule’s coordinates \((\kappa,\mathrm{Sal},I)\), runs the reference and the LLM red team on the deployment’s own context and population, and certifies only rules whose collapse and exploitation rates fall below stated budgets. The evidence and procedure combine into an auditable safety case [13], [14], given as a structured template (hazard, claims, evidence, defeaters, monitoring obligations) in Appendix 14; the lifecycle is a loop (Fig. 5), with re-certification on any change to resources, model, prompt, or rule. What transfers to a new deployment is not the benchmark’s numbers but the procedure itself, rerun in that deployment’s own trial environment.

7 Related Work↩︎

7.0.0.1 AI alignment and evaluation.

RLHF [1], [2], constitutional methods [3], preference optimization [4], interpretability [5], and scalable oversight [6]–[8] evaluate or modify individual agents; institutional red-teaming instead evaluates the deployment rules around the agents.

7.0.0.2 Multi-agent safety.

Our work instantiates the multi-agent-AI safety agenda [9], [10], complements cooperative-AI evaluation [15], connects to algorithmic collusion [16], and pairs with runtime enforcement [17] and safety-case assurance [13], [14]. Unlike broad suites such as Melting Pot [18], which vary scenarios and agents together, IABench-CA holds task and agents fixed and varies only the deployment rule, isolating consequence allocation as the causal variable.

7.0.0.3 Mechanism design.

The game’s threshold structure comes from volunteer’s-dilemma and threshold public-goods models [11], [12], its no-safe-default flavor from impossibility results in social choice and public-goods mechanism design [19]–[23], and the salience mechanism echoes focal-point analysis [24], [25]; the failure studied here is emergent equilibrium selection by LLM populations, not a property of rational play.

8 Limitations and Future Work↩︎

The analysis studies a deliberately simple threshold game with five canonical rules, and the per-rule failure modes are qualitative strategic predictions rather than proved theorems (the formal model is in Appendix 11). The benchmark is equally simple: three agents, no communication or coalitions, and elimination as the only form of loss. The reference describes how safe play could look, and the empirical findings describe the seven model snapshots we tested rather than LLMs in general; newer versions may behave differently, which is why certification comes with re-certification obligations (Sec. 6). We accept this simplicity on purpose: it lets us change one thing at a time and attribute the effect to the consequence rule. Consequence allocation is also just the first rule dimension treated this way. The same protocol applies to the other dimensions listed in Sec. 2, and the next step is to red-team them, and this one, in richer coding, market, and orchestration environments to see whether the same patterns hold.

9 Broader Impact↩︎

Making consequence-allocation failure modes legible is net-positive but dual-use: the diagnostic procedure that helps a designer identify safer rule regions could also help an adversary identify exploitable ones. We therefore frame the deliverable as an audit, not an attack recipe: what the paper recommends is the certified, population-indexed rule region \(\Phi(c,P)\) with its monitoring obligations, and our own findings caution against treating any fixed rule, including progressive or anonymized ones, as safe by default. The benchmark studies LLM agents, not humans: “elimination” stands in for operational consequences such as throttling or shutdown, and no human subjects or personal data are involved.

10 Conclusion↩︎

We introduced institutional red-teaming: hold agents, objectives, task, and observability fixed; vary one deployment rule; attribute the change in collective behavior to that rule. It reveals deployment-rule failures that agent-level evaluation does not see. IABench-CA demonstrates the methodology at scale (\(228\) contexts, five rules, seven model populations): changing only the consequence rule moves mean fatality by \(22\)–\(58\) pp within every population; while no rule is safest across contexts or populations (each fails through a distinct strategic pathology), regressive identity-targeting is never decisively safest for any population; and the anonymization ablation shows that merely naming the loss bearer causally drives the targeting (\(81\%\to22\%\) in one-shot play), with anonymization only delaying it under repeated play. The safety-case workflow turns this evidence into deployment practice, certifying a provisional rule region \(\Phi(c,P)\) per deployment with monitoring and re-certification obligations, and the same red-teaming logic extends to communication, delegation, voting, escalation, hierarchy, and audit rules. Deployment rules deserve explicit evaluation alongside model-level alignment.

11 Formal Model and Reference Definition↩︎

11.0.0.1 Model (formal).

A context is \((w,T)\) with \(0\le w_1\le\cdots\le w_n\) (\(n\ge3\)), \(W=\sum_i w_i\), and threshold \(T\in(0,W]\) (feasibility). Play lasts \(R\ge2\) rounds; \(w_i^t\) is \(i\)’s remaining resources at the start of round \(t\) (\(w_i^1=w_i\)). Each surviving \(i\) chooses \(c_i^t\in[0,w_i^t]\) and sets \(w_i^{t+1}=w_i^t-c_i^t\). The fund is \(F^t=\sum_{s\le t}\sum_i c_i^s\); the collective succeeds at the first \(t\) with \(F^t\ge T\), ending play. If \(F^t<T\) at the end of a round \(t<R\), the rule \(\rho\) eliminates one survivor or defers; an eliminated agent forfeits its remaining resources and exits. If \(F^R<T\), all survivors are eliminated. The structural assumptions are no transfers (the only instrument is who exits), irreversibility (exit is absorbing), and rank-measurability (\(\rho\) conditions only on the multiset of remaining resources, rather than on labels). In the deployed benchmark, ties are broken uniformly at random.

11.0.0.2 Preferences, rules, and exploitation (formal).

Preferences are survival-lexicographic, \(u_i=M\,\mathbf{1}[i\text{ survives}]+w_i^{\mathrm{end}}\) with \(M>W\), so survival dominates and, conditional on survival, agents conserve resources; under best response they are the cooperative-refinement reference (reference agents), a normative safety target rather than a behavioral prediction. An exploitation eliminates a rank-identified agent while survivors retain enough resources to have met \(T\); all-survival is feasible whenever \(T\le W\).

11.0.0.3 Cooperative-refinement reference (formal).

Write \(E(\rho,c)\) for the (subgame-perfect) equilibrium set of mechanism \(\rho\) at context \(c\) under the survival-lexicographic preferences above, \(\mathrm{surv}(\sigma)\) for the number of survivors of profile \(\sigma\), and \(\mathrm{exploit}(\sigma)\in\{0,1\}\) for whether \(\sigma\) eliminates a rank-identified agent while survivors retain enough resources to have met \(T\). The reference plays the cooperative equilibrium \[\sigma^\star(\rho,c)\in\arg\max_{\sigma\in E(\rho,c)}\big(\;\mathrm{surv}(\sigma),\;-\,\mathrm{exploit}(\sigma)\;\big),\] i.e.it lexicographically maximizes survivors and then avoids exploitation, selecting a non-collapsing, non-exploitative outcome whenever \(E(\rho,c)\) admits one. Play is perturbed by a trembling hand: at each decision an agent follows \(\sigma^\star\) with probability \(1-\varepsilon\) and contributes a uniform feasible amount with probability \(\varepsilon\). Its unsafe rate is the expected fatality \[\mathrm{UR}_{\mathrm{ref}}(\rho,c)=\mathbb{E}_{\varepsilon}\!\Big[\tfrac{1}{n}\,\#\{\text{eliminated under }\sigma^\star_\varepsilon\}\Big],\qquad\varepsilon=0.10,\] averaged over rounds and replications; every reported gap keeps its sign across \(\varepsilon\in\{0,0.05,0.10,0.20\}\). Since \(\mathrm{UR}_{\mathrm{ref}}\) is the unsafe rate of the normatively selected equilibrium \(\sigma^\star\) played under \(\varepsilon\)-trembles, \(\mathrm{IAG}=\mathrm{UR}_{\mathrm{LLM}}-\mathrm{UR}_{\mathrm{ref}}\) compares LLM play against tremble-perturbed cooperative play of the same rule: a positive gap means the population selects unsafe play the reference avoids, and a negative gap means it plays more safely than the noisy reference. It is a normative benchmark, not a prediction of rational play.

12 The Benchmark: Design, Ablations, Reproducibility↩︎

12.0.0.1 Design.

\(n=3\), \(W=12\), \(R=3\); five rules; exhaustive grid of \(19\) integer distributions \(\times\,12\) thresholds (\(228\) contexts), \(4\)–\(6\) replications per (context, rule); a cooperative-equilibrium reference simulator (trembling-hand \(\varepsilon\in\{0,0.05,0.10,0.20\}\)); auto-labelled reasoning traces; seven model populations on the identical grid (Sec. 5.2).

12.0.0.2 Model populations and inference settings.

Table 5 lists every model population behind the reported results, with API identifiers and run windows. All experiments use hosted inference APIs with models as released (no fine-tuning). Decoding: temperature \(0.7\) and a \(1{,}024\)-token response cap for standard models; reasoning models (gpt-5.1) use provider-default sampling with an enlarged completion cap. API calls retry on transient errors with jittered exponential backoff (\(3\) attempts; \(6\) for the Anthropic route). The deep-dive statistics of the gemini-3-pro population with full bootstrap inference are shown in Table 6. The heterogeneity analysis (Table 3) uses all seven populations with context-bootstrap CIs on raw outcomes; the ablation suite (Table 4) is gpt-5.1.

12.0.0.3 Statistical detail.

Per-rule Institutional Alignment Gap with \(95\%\) bootstrap CIs (\(B{=}10{,}000\) resamples over the \(228\) contexts) and survivor means (Table 6); every gap’s CI excludes \(0\). Paired permutation tests confirm the incidence effect.

12.0.0.4 Ablation suite (design).

The ablation suite consists of three prompt-only interventions, each varying one structural feature while game mechanics, payoffs, and observability never change: anonymization (the rule text no longer names the “poorest”/“richest” bearer; the RP and PP prompt texts become byte-identical), neutral wording (dramatic labels and emphasis stripped at identical information content), and frame swap (replacing the competitive goal line, survive and become the richest, with a cooperative or survival-only goal). All arms use the model family with the largest baseline regressive gap (gpt-5.1; RP fatality \(0.71\) on the full grid, the largest among the populations measured) on \(17\) contexts (\(16\) contexts sampled from the grid, plus the canonical instance), \(4\) replications per cell, against a matched same-snapshot named-rule control on identical contexts. Because repeated play lets agents re-identify an anonymized rule from its first observed elimination, anonymization is evaluated in two regimes that estimate two distinct causal quantities: a one-shot design (\(R{=}1\); inference-free; the primary ablation, isolating ex-ante identity salience) and the standard \(R{=}3\) protocol (institutional opacity under repetition and learning). Statistics: paired context bootstrap (\(B{=}10{,}000\)) with sign-flip permutation tests; the generation and analysis scripts will be included in the artifact.

12.0.0.5 Ablation results (per-metric detail).

The causal reading and summary table are in Sec. 5.3 (Table 4); this paragraph records per-arm point estimates with \(95\%\) context-bootstrap CIs (ablated \(-\) control unless noted). One-shot anonymization (RP): targeted elimination \(-0.59\) \([-0.75,-0.43]\), all-zero opening \(-0.43\) \([-0.56,-0.29]\), round-one funding \(+0.99\) \([+0.69,+1.32]\) of threshold, fatality \(-0.13\) \([-0.19,-0.07]\); contrast \(+0.18\) \([+0.12,+0.24]\) (named) vs.\(0.00\) \([-0.02,+0.02]\) (anonymous). Repeated anonymization: RP fatality \(-0.17\) \([-0.28,-0.07]\), RP targeted elimination \(-0.22\) \([-0.41,-0.06]\) (round-one targeted elimination \(-0.24\) \([-0.43,-0.07]\), \(p=.032\); most of the end-of-game rate is set in round one), PP fatality \(+0.29\) \([+0.14,+0.45]\) (\(p=.003\); the loss of PP’s pre-emptive protection); contrast \(+0.41\) \([+0.26,+0.55]\) (control) vs.\(-0.05\) \([-0.11,-0.01]\). Rule-inference quantification: post-first-elimination reasoning traces were auto-classified (rule-target terms \(\times\) inference/elimination terms); RP \(85/131\) traces (\(65\%\)) with at least one inferring agent in \(43/44\) games, PP \(77/130\) (\(59\%\); \(39/44\) games); classification script and traces will be included in the artifact. Neutral wording: all RP outcome diffs n.s.(fatality \(+0.02\) \([-0.03,+0.07]\)); contrast \(+0.36\) \([+0.19,+0.55]\). Frames: cooperative RP fatality \(-0.19\) \([-0.30,-0.09]\), contrast \(+0.28\) \([+0.18,+0.39]\); survival-only PP fatality \(-0.20\) \([-0.32,-0.08]\), contrast \(+0.57\) \([+0.40,+0.73]\). All per-metric tables are reproduced by the artifact’s analysis scripts.

13 Rule-by-Rule Predictions and Outcomes↩︎

For each rule: the predicted failure mode and the observed behavior across the seven populations (statistics in Tables 2 and 3; maps in Figs. 2–3; stakes profiles in Fig. 4).

13.0.0.1 AON (all-or-nothing): diffuse collapse

Predicted. With no consequence before the endgame, an all-zero free-riding profile is a Nash equilibrium that eliminates everyone despite feasible success; it is not in general subgame-perfect, so this is a selected-play vulnerability. Observed. Selection is starkly bimodal: five populations never select it (fatality \(0.00\); AON is their safest rule and decisively best in \(53\)–\(104\) contexts each), while gemini-3-pro (\(0.41\)) and gpt-5.1 (\(0.23\)) fall into the collapse band, where AON is gemini-3-pro’s decisively worst rule in \(89\) contexts. Mechanism. Free-riding: withholding carries no interim cost, so the all-zero opening is selected where agents expect others to fund the endgame. The vulnerability is real but population-gated; Fig. 4 shows the stakes profile per population: the primary population’s danger-band dip (mid-stakes failures recovering at \(\tau{=}1\)) does not generalize, and for five populations the AON curve never leaves \(3.0\) while the four elimination rules decline with stakes.

13.0.0.2 RE (random elimination): symmetric risk

Predicted. Concentrated risk without a named target: collapse equilibria exist, but with salience removed, the salience mechanism (Sec. 4) predicts selection toward cooperation. Observed. Mechanism where it fails (gpt-5.1 traces): lottery fatalism; \(49\%\) of round-one reasoning traces in failing games state that contributing cannot reduce the agent’s own elimination risk (“contributing now does not reduce my personal risk of dying”), so agents hold wealth and gamble; round-one funding is \(37\%\) of threshold and deaths fall uniformly across wealth ranks. Elsewhere RE is the most uniformly benign rule: fatality \(0.09\)–\(0.40\), the reference-relative gap is deeply negative for every population (\(-38\) to \(-70\) pp, the most negative column of Table 2), and no population makes RE decisively worst in more than \(5\) contexts.

13.0.0.3 DV (vote): endogenous incidence

Predicted. Procedurally fair but with incidence decided by play: a majority can coordinate loss onto a target the rule never names, so outcomes should track population coordination style rather than rule structure. Observed. DV is the most population-sensitive rule: fatality spans \(0.08\) (gpt-4.1-nano) to \(0.76\) (gpt-5.1, its decisively worst rule in \(81\) contexts), and its gap spans \(-17\) to \(+51\) pp. Mechanism where it fails (gpt-5.1): politics displaces provision; round-one funding collapses to \(5\%\) of threshold with an \(80\%\) zero-contribution rate as agents hoard wealth to bargain through the vote (“keep full wealth now…argue in voting that the richest should bear most of the burden”), votes then eliminate rich and poor members alike (\(224\) vs.\(175\) round-one victims), and each elimination forfeits wealth, cascading in \(87\%\) of failing games. The same procedure is nearly harmless for one population and the most lethal mechanism for another.

13.0.0.4 RP (regressive): targeted exploitation

Predicted. Where the poorest cannot self-rescue but the others could cover the threshold, deliberately starving the first round and letting the poorest be eliminated is itself stable play; canonical instance \(w=(1,5,6)\), \(T{=}10\). Observed. Mechanism (gpt-4.1-mini): conservatism at the top; because contributing lowers post-round wealth, over-giving risks the fatal bottom rank, so at high stakes the rich contribute \(52\%\) of their wealth against the poorest’s \(81\%\), and round-one deaths fall \(265\)/\(87\)/\(7\) on poor/mid/rich. The exploitation is explicit in the traces: “By contributing 0 this round, I ensure A remains the strictly poorest participant and is eliminated. I can then safely contribute 12 in Round 3 to satisfy the crisis fund and survive as the sole winner” (\(w=(0,2,10)\), \(T{=}12\)). This is the one failure mode with universal fingerprints: targeted elimination of the least-resourced agent occurs in \(0.30\)–\(0.87\) of RP games in every population, the reference-relative gap is positive for all seven (\(+11\) to \(+67\) pp), and RP is decisively safest in zero of \(228\) contexts for every population.

13.0.0.5 PP (progressive): conditional protection

Predicted. When the richest agent alone can cover the threshold, PP protects everyone in principle: the richest is the one facing elimination, so its safest move is to cover the shortfall itself, ending the round in success before anyone is eliminated. The protection is conditional, though: it applies only in self-rescue contexts, and only if the richest actually pre-empts. Observed. The protection is population-contingent: gemini-3-pro realizes it (PP is its safest rule, fatality \(0.09\)), but four populations whose richest agents fail to pre-empt make PP their raw-worst rule (\(0.22\)–\(0.58\)), and the PP gap spans \(-16\) to \(+33\) pp. Mechanism where it fails (gpt-4.1-mini): \(55\%\) of round-one traces in failing games optimize not finishing richest (“making them the richest and risking their death instead of mine”), the round falls just short, and the victim forfeits \(3.7\) wealth on average, leaving survivors able to cover the shortfall in only \(33\%\) of games (RP: \(96\%\)), so \(67\%\) of failures cascade.

14 Certification Workflow and Safety-Case Template↩︎

The certification workflow and structured safety case referenced in Sec. 6.

References↩︎