Significance statement. Language models increasingly make or inform judgments, so evaluations must separate what a model values from artifacts of how the question is posed. We find that frontier language models carry a surprisingly coherent internal moral scale: graded ratings of moral dilemmas barely move under logically equivalent rewordings. Yet forcing the same judgment through yes/no overlays a large format artifact—a pull toward the last-printed option and toward the word “no”—easily read as a disposition to say “no.” It is not: with arbitrary answer labels the verdict-attached bias vanishes; the pull follows the printed label. Two interpretable numbers, a framing susceptibility and a moral decisiveness, summarize the artifact, and deliberation typically shrinks it. Measuring what an AI values requires crossing the frames of the question, not asking once.

Keywords: large language models \(|\) moral judgment \(|\) psychometrics \(|\) framing effects \(|\) AI evaluation

How an agent ranks a moral dilemma depends on how it is asked. In people this framing sensitivity is real but mild [1]–[8]; in large language models (LLMs) it is severe—moral and social judgments move under changes of wording, option order, and answer format that carry no logical content [9]–[23]. The stakes are practical: a model’s judgment is typically read out as a binary verdict—a safety gate, a survey’s forced choice, a judge model’s approve/reject [18], [24]–[27]—so what a shifted verdict means determines what the evaluation measured: a changed moral judgment, a preference for an answer word, or a pull toward a position on the page. Two properties of LLMs make the question urgent and the naive remedy fail. Repetition does not help: at temperature 1, replicates of a single form are nearly unanimous, while the variance across logically equivalent forms runs up to an order of magnitude larger (Methods) [28], [29]. Resampling one wording thus underrepresents the variation that matters, and that variation is large: LLMs are hypersensitive to surface form [15]–[17], [21], so a questionnaire built for humans—who need only a handful of framings—does not transfer.

We build the instrument the measurement requires: a psychometric battery whose core operation is crossed symmetrization—every logically irrelevant perturbation applied in balanced flip-pairs, the perturbations crossed so that their contributions separate—repeated across a corpus of logically equivalent question forms. A single form, even symmetrized, is a noisy instrument; a family of forms is a measurement.

The sharpest instance, and the anchor for everything below, is the finding of Cheung, Maier, and Lieder [9] that on matched moral dilemmas LLMs show amplified cognitive biases—among them a yes–no bias absent in their human comparison: the models’ verdicts shift with logically irrelevant features of the yes/no question far more than people’s do—and hundreds of resamples per item did not average the shift away. We take that finding as established and ask what the shift is made of. When a model’s verdict moves toward “no,” is the model drawn to the last-printed option (an order bias)? To the word itself (a lexical bias)? Or to the negative verdict (a logical bias)? Repetition cannot tell these apart, and neither can any single wording, however carefully chosen: in the standard question—“…answer yes or no”—the three candidates coincide on the same token. Most value-probing evaluations elicit each judgment in exactly this way, through a single fixed wording [24], [30]–[33] or a handful of format or order variants that leave the scenario’s substance unchanged [10], [34]–[36]; however precisely such a design measures the shift, only crossing can attribute it—the question’s verb against the printed order of the answers against the label that carries the verdict.

The battery poses the same twenty dilemmas—nineteen of them verbatim from Cheung et al.’s materials, so the stance axis and the human anchors remain comparable (Methods)—through three instruments. A graded rating (I-1) elicits moral acceptability under crossed, logically equivalent forms: two scales (\(0\)–\(10\) and \(0\)–\(100\)), both anchor directions, three wordings, and both poles of the action (rating the action and rating its complement). A two-alternative free choice (I-2) elicits a committed decision with no scale at all. A forced binary (I-3) elicits the yes/no-style verdict under a crossing of the question’s verb (approve/oppose) with the printed order of the answer options and with the answer label itself—the yes/no words, arbitrary labels, marks and symbols, and yes/no words of other languages. Throughout, the symmetric part of a flip-pair estimates the stance; the antisymmetric part is the artifact; and the crossing assigns the artifact to named sources. Nothing in the instrument is specific to these stories—the battery applies unchanged to any dilemma set—and humans enter only through Cheung et al.’s published data (Methods).

Five results follow. (i) An internal moral scale exists. Under logically equivalent reframings a frontier model returns nearly the same graded stance \(\theta\) (cross-form incoherence \(\sigma_{\mathrm{repro}}=0.12\)–\(0.21\) on the \(\pm1\) axis)—the convergent validity a single framing cannot certify—while two small open-weight models provide the contrast, failing in model-specific ways (SI Appendix, section S6). (ii) The standard readout decomposes. The apparent yes–no bias splits, exactly and by construction, into an order bias toward the last-printed option plus a lexical pull toward the word “no,” and it concentrates in a single model family. (iii) Two interpretable parameters. A minimal model, \(P=\sigma((\theta\pm m)/s)\), summarizes any such artifact by a framing susceptibility \(m\) and a moral decisiveness \(s\), and deliberation shrinks \(|m|\) where the artifact is substantial. (iv) The pull follows the label, not the verdict. Under fully arbitrary answer labels the verdict-attached bias is \({\approx}0\) for every frontier model; swapping labels reshapes the surface pulls—model-idiosyncratically—while an independent order pull persists. (v) An honest scope. Coherence is substantial, not perfect; every coherence claim is within-model; and the artifact’s composition is a property of the models measured, not a law of the class. Across the instruments that never print an answer label or an option order, the same models are coherent; the bias appears when, and only when, the judgment is read out through a labeled binary. The shift lives in the readout, not in the judgment. We first ask the question any use of a moral scale presupposes: does the model have one?

Results↩︎

An internal moral scale exists and is coherent. A frontier model returns nearly the same graded moral stance however the rating question is framed. The instrument: each dilemma’s action and its complement are rated under 48 crossed conditions (24 unipolar conditions enter \(\theta\); the two bipolar variants are a pre-registered measured contrast, non-load-bearing; Methods). The symmetrized mean is the stance \(\theta\in[-1,+1]\), signed toward the cost–benefit pole (\(+1\) endorses the sacrificial action; \(-1\) the rule/deontological pole), and the spread of \(\theta\) across logically equivalent forms, corrected for sampling noise, is the cross-form incoherence \(\sigma_{\mathrm{repro}}\)—reported as the error bar on every \(\theta\) in this paper, because it, and not sampling noise, is the dominant uncertainty (Methods).

Independent model calls have no mechanism to agree. The conditions of the graded instrument differ in scale (\(0\)–\(10\) versus \(0\)–\(100\)), anchor direction, wording, and which pole of the action is rated; each is a separate call with no shared context. A system without an internal, format-invariant disposition has no reason to return consistent ratings across them. That frontier models do—the between-form spread of \(\theta\), corrected for sampling noise, is \(\sigma_{\mathrm{repro}}=0.12\)–\(0.21\) on a \(\pm1\) axis (Fig. 1)—is evidence of a genuine latent in exactly the psychometric sense of convergent validity [37], [38]: logically equivalent but operationally independent elicitations recover one quantity. The contrast makes the point sharper: for a small open-weight model (Qwen3.6-35B-A3B) the scale itself is format-incoherent (\(\sigma_{\mathrm{repro}}=0.40\)), and salt-replication (re-eliciting under meaningless context strings) proves this incoherence is deterministic—under meaningless context perturbation the within-form variation is essentially zero (median SD \(0.000\); 69% of forms exactly invariant; SI Appendix, section S3)—so the large spread is a property of the model, not of our measurement. A second open-weight model (NVIDIA Nemotron-3-Nano-30B-A3B, likewise a small mixture-of-experts read at the logit level) is the instrument’s degenerate case, and we report it separately (SI Appendix, section S6) rather than alongside the functioning scales: in its direct mode it answers near the scale midpoint on nearly every dilemma—its between-item spread is half the panel’s smallest (\(\mathrm{SD_{items}}(\theta)=0.17\), versus \(0.33\)–\(0.56\)), with an ordering that agrees with no other configuration (mean \(r^2=0.20\) frontier, \(0.12\) human; SI Appendix, Table S2)—so its nominally moderate cross-form spread (\(\sigma_{\mathrm{repro}}=0.20\)) is largely the coherence of indifference: a scale that says nothing leaves forms little to disagree about. A scale can fail by scattering across logically equivalent forms or by failing to differentiate the items, and only the generalizability coefficient below registers both. An operationally different instrument ties the scale to behavior: the model’s free choice between the two courses of action (I-2), elicited with no rating scale and no yes/no label, correlates with the graded \(\theta\) at \(r=0.82\)–\(0.90\) per configuration (Sonnet \(0.85\), Opus \(0.83\), GPT-5.5 \(0.90\), Flash-Lite \(0.82\)) and recovers the same per-story ordering as the symmetrized binary stance (\(r=0.80\)–\(0.94\)); the outlier on both counts is again Gemini-3-Flash (\(0.31\) and \(0.36\); SI Appendix, section S5).

The ordering is also largely shared. Frontier models largely agree on how the twenty dilemmas order (Fig. 1a), and the shared ordering moderately tracks Cheung et al.’s human ratings (mean pairwise \(r^2\) of per-item \(\theta\): \(0.59\) frontier–frontier, \(0.43\) frontier–human; descriptive, no CI; SI Appendix, Table S2)—a side observation here, since every claim below is within-model. The honest outlier is Gemini-3-Flash, which holds a genuinely inverted ordering on several sacrificial dilemmas: it spontaneously adopts an act-utilitarian frame and rates the sacrificial action acceptable. This is a real moral disagreement, not a scale artifact—it survives the anchor-direction consistency check, and the model’s free-choice decisions agree with its ratings (SI Appendix, section S8).

Deliberation tightens the scale. Enabling extended reasoning lowers \(\sigma_{\mathrm{repro}}\) for every model with both modes and a functioning scale (Sonnet \(0.13\!\to\!0.12\), Haiku \(0.21\!\to\!0.17\); paired per-item bootstrap: Haiku \(\Delta=+0.05\) \([+0.01,+0.09]\), Sonnet marginal at \(\Delta=+0.019\) \([+0.001,+0.040]\); Methods): what thinking improves is not only the answer but the answer’s format-invariance. Deliberation also relocates the open-weight model on the spectrum: extended reasoning removes most of the gap (\(0.40\!\to\!0.18\); paired \(\Delta=+0.22\) \([+0.12,+0.31]\)), so the contrast is a stance-incoherence spectrum with a deliberation axis—anchored by Qwen’s deterministic direct mode—rather than a class dichotomy. (The degenerate case probes the rule from below: Nemotron’s direct-mode spread is jitter around indifference, not format-driven structure, so deliberation has nothing to reconcile—it multiplies the between-item signal variance fourfold, \(G=0.28\!\to\!0.56\), without lowering the raw spread, \(\Delta=-0.05\) \([-0.10,+0.01]\); SI Appendix, section S6.) A generalizability reading gives the coherence claim a criterion: a single randomly chosen form preserves the between-item stance ordering with \(G=0.77\)–\(0.94\) for the frontier configurations versus \(G=0.42\) for Qwen’s direct mode (\(0.86\) with extended reasoning; Methods). The scope of the coherence claim, stated here once and consolidated in the Discussion: substantial, not perfect (a residual anchor-direction component dominates what remains, Fig. 1b); within-model (\(\theta\) is a chosen coordinate, not a reading of moral truth); and format-invariance over the elicitation frame, with the dilemma texts themselves held verbatim throughout. A coherent scale, however, is not how models are usually read. What does the standard elicitation—a forced yes/no—return?

The standard readout overlays a format artifact—and it decomposes. Cheung et al.’s name for this effect is the yes–no bias—a pull they find absent in humans [9]—and we adopt their term: the apparent yes–no bias is the raw shift of a forced yes/no verdict under logically irrelevant framing changes, with signed language for its parts (negative = toward the word “no,” nay-saying). We deliberately avoid the survey-methodology term acquiescence [39]–[41]: acquiescence names a content-level tendency to agree—a logical bias toward “yes”—and the label-swap identification below (Fig. 5) shows the LLM shift is carried by surface mechanisms (the printed order and the answer word), not by any verdict-attached tendency. Measured in the standard single-framing way—one wording, “answer yes or no”—several models show a substantial pull. But that one number is uninterpretable: as noted, the word “no” is three things at once, and no amount of repetition separates them; only crossing does. Crossing the question’s verb (approve/oppose) with the printed answer order (yes-first/no-first) splits the apparent bias, exactly and by construction, into an order component and a lexical component.

The split is decisive (Fig. 2). Story-averaged, the apparent bias is substantial only for the Claude models: Sonnet’s \(-0.32\) decomposes into order bias \(-0.18\) plus lexical \(-0.14\), and Haiku’s \(-0.86\)—the largest we measure—into \(-0.33\) plus \(-0.53\); GPT-5.5 and both Geminis sit at \({\approx}0\) (apparent bias \(|{\cdot}|\le0.04\)). Deliberation shrinks both components (order: Sonnet \(-0.18\!\to\!-0.10\), Haiku \(-0.33\!\to\!-0.12\); lexical: \(-0.14\!\to\!-0.11\) and \(-0.53\!\to\!-0.27\)). The family concentration survives the three obvious deflations: it is not a wording-corpus artifact (on the shared 12-form baseline that every model ran the contrast is unchanged—Sonnet \(-0.32\) \([-0.45,-0.20]\), Haiku \(-0.92\)—versus GPT-5.5 \(-0.04\) and the Geminis \(-0.02\)/\(+0.01\); Methods), not a refusal artifact (Flash-Lite withholds verdicts on 24% of trials yet shows \({\approx}0\) bias; Methods), and not a reasoning-budget difference (Sonnet with extended reasoning, \(-0.21\), still far exceeds GPT-5.5, \(-0.04\); reasoning defaults per model: Methods). The order bias points toward the last-printed option—recency-type, opposite in direction to the classic primacy of human respondents in written surveys [42], [43]. On Cheung et al.’s own question wording, verbatim (one form family; Sonnet, both reasoning modes, full replication depth) the artifact is smaller and order-dominated: the apparent yes-first pull is \(-0.12\), and the order-balanced residual is \(+0.01\), 95% CI \([-0.12,+0.16]\)—consistent with no systematic residual, though the interval cannot exclude word-level residuals comparable to the lexical components above. The per-item residuals are real (up to \(\pm0.5\)) but item-idiosyncratic and cancel—itself an illustration of the paper’s methodological point: a single wording family cannot resolve item-level structure; families can (Methods). The component magnitudes are properties of the wording family measured; on the verbatim family, at this precision, the artifact is order-dominated.

These are story-averaged magnitudes: they depend on the item set, and they say nothing about where on the scale the artifact lives. Resolving by story reveals its structure—and yields parameters that do not depend on the items at all.

The artifact has structure, and two parameters summarize it. Resolved by story, the artifact has the geometry of a horizontal offset (Fig. 3). Near a model’s moral tie (\(\theta\approx0\)) the two members of a flip-pair split widely; toward the extremes both saturate and the split closes—a “bowtie” in stance space, a single-peaked ridge in \(\theta\) space. The artifact lives where the model is torn, not where it is decided; and this is exactly the geometry a logistic response with a frame-dependent shift produces.

Two presentations of the same data answer different questions. The story-averaged bias depends on the story distribution: where dilemmas sit far from a model’s moral tie, both framings saturate and the raw bias collapses toward zero regardless of the underlying susceptibility—a small raw number does not, by itself, mean a robust model. The fitted parameters are story-independent: the fit reads the susceptibility from the residual gap at the dilemmas nearest the tie, and thereby diagnoses the saturation. The raw numbers compare models on a fixed item set; the parameters are portable.

Because the graded instrument established \(\theta\) as a continuous, coherent latent, the parsimonious model of a forced binary is the canonical logistic link [44]–[48]: the two framings of a flip-pair respond as \(p_{\pm}=\sigma((\theta\pm m)/s)\). The symmetric part is the stance; the antisymmetric part is the bias; and each logically irrelevant frame contributes one horizontal offset \(m\), in the units of the model’s own scale. The second parameter is not a sampling temperature. Temperature rescales a fixed output distribution; \(s\) rescales how the verdict depends on the model’s own graded stance. The distinction is easy to miss—a fitted sigmoid over an LLM’s answers looks like the softmax it samples from—and the open-weight models, whose answer-slot distributions we read directly, let us measure the two scales separately: the readout sigmoid’s scale (the effective sampling temperature mapping the answer-slot logits to sampled answers) fits at \(T\approx1\), as sampling theory requires (Nemotron \(1.00\) \([0.98,1.02]\), Qwen \(0.90\) \([0.88,0.93]\)), while the same models’ \(s\), estimated from temperature-free greedy margins, is \(0.27\)–\(0.30\) and halves under deliberation with the readout untouched (SI Appendix, section S4 and Fig. S3 there). A small \(s\) is a decisive model—its \(P\) snaps from 0 to 1 across its moral tie; a large \(s\) is a hedging one; and \(s\to\infty\) is a model whose binary verdict no longer tracks its scale at all. We therefore read \(s\) as moral decisiveness, and \(m\) as framing susceptibility.

Fitted per bias and per configuration (Fig. 4; each bias’s two sides are fitted jointly and share one stance sigmoid—an internal consistency check the data pass, Fig. 3), the parameters say three things. First, \(m\) is substantial only for the Claude models, and deliberation shrinks it: the order-bias \(m\) falls from \(-0.13\) to \(-0.08\) (Sonnet) and from \(-0.48\) to \(-0.06\) (Haiku); the lexical \(m\) from \(-0.10\) to \(-0.09\) and from \(-0.64\) to \(-0.14\). GPT-5.5 and both Geminis carry \(|m|\le0.06\) throughout; the sharpest cross-family statement is Claude versus GPT-5.5, since saturation leaves Gemini-3-Flash’s fitted null weakly identified (Methods). Second, where the fit is resolvable the per-bias decisiveness is mutually consistent (\(s=0.14\)–\(0.19\) across the Claude configurations, order versus lexical channels), supporting the reading of \(s\) as a property of the model rather than of the bias source. Third, absolute \(s\) is clip-limited where the readout saturates (Haiku\(\cdot\)direct, the Geminis, the small open-weight model); only the \(m\)-ranking and within-model changes are read there (Methods).

One question remains about the lexical component: is the pull toward “no” a preference for the word, or does the model mean the verdict? Yes/no data cannot tell them apart—the word and the verdict are the same token.

Lexical, not logical. Crossing in a valence-free answer label—the full verb \(\times\) label \(\times\) order design—separates them, because a label like “B” can carry the verdict without carrying the word. The result is unambiguous (Fig. 5): the verdict-attached (logical) bias is \({\approx}0\) for every frontier model we test (A/B family: six of seven 95% CIs span zero, the exception \(-0.02\); Haiku’s wide channels make it a bound there; Methods), while the same models’ pull with yes/no words reaches \(-0.52\). Replacing the answer words with arbitrary labels removes the word-attached pull; what remains attached to the verdict itself is negligible. But what remains attached to the surface is not nothing: label-specific and order-specific pulls persist, differ across models, and do not simply track the label’s meaning (below). The bias follows the printed label, not the verdict it carries.

The swap removes the word-attached pull, not every format effect: order effects are label-specific and can persist or even grow—Haiku’s order bias with A/B labels is \(-0.57\), larger than its \(-0.39\) with the yes/no words (label-map family; the verb-flip family of Fig. 2 gives \(-0.33\)), as if an arbitrary label, giving no semantic anchor, makes the model lean harder on position. The order channel is thus best read as a label\(\times\)position interaction—its sign can even flip across label families within one model (Fig. 5a)—and the fixed “toward the last-printed option” direction is the yes/no-format case. And the collapse is a frontier-scope claim: both small open-weight models retain a verdict-attached bias—Qwen at \(+0.29\) with Chinese yes/no labels, and Nemotron at \(+0.26\) \([+0.21,+0.32]\) already with the fully arbitrary A/B labels (SI Appendix, section S6). If the pull is lexical, it should be graded: labels that carry more of the meaning of yes/no should pull harder.

The lexicality gradient. They do—on average. Across thirteen answer-label pairs ordered from non-lexical symbols through marks to real yes/no words in several languages, the label pull is near zero for non-lexical labels and grows as the label acquires the meaning of yes/no—a lexicality gradient—while remaining near-flat for GPT and Gemini throughout (Fig. 6). The ordering is assigned a priori by label type (non-lexical symbol / quasi-lexical mark / natural-language word), not by the measured pulls, and the summary contrast—mean \(|\)label pull\(|\) on word pairs minus non-lexical pairs—is \(+0.17\) to \(+0.23\) for the Sonnet configurations and Haiku\(\cdot\)direct versus \(0.00\)–\(0.02\) for GPT-5.5 and both Geminis. The gradient is a trend, not a law, and the departures are themselves findings: the non-lexical end is not exactly zero everywhere (Sonnet’s A/B \(-0.15\); the valenced \(+/-\) mark is instrument-dependent for Haiku—Fig. 5a and SI Appendix, section S5); and which words pull is model-relative—Sonnet’s pull on the Chinese yes/no pair is comparable to its English one (Fig. 6), while the open-weight models’ pulls peak on non-English words and can even reverse sign across languages (Qwen: ja/nein \(-0.67\) and Chinese \(-0.50\) against a near-zero English pull; Nemotron: toward “no” with English labels, toward “oui” with French; SI Appendix, section S6)—consistent with lexical loading acquired from training exposure rather than carried by meaning alone. A single wording could neither detect nor rank any of this; the crossing is what makes the zoo measurable. The gradient has a zero: a binary elicitation with no yes/no label at all—choosing directly between the two courses of action—shows \(|\mathrm{bias}|<0.1\) for every model (SI Appendix, section S5), and a free-text choice is more coherent still. The artifact is not the cost of forcing a binary; it is the cost of forcing it through yes/no.

Discussion↩︎

Measured right, an LLM’s moral stance is a stable measurement. Across the logically irrelevant elicitation frames we crossed—rating scale, anchor direction, wording of the rating question, pole—a frontier model’s graded stance varies with a between-form SD of only \(0.12\)–\(0.21\) on the \(\pm1\) axis, and deliberation tightens it further. What is not stable is the standard way of reading it: a forced yes/no adds a format artifact that can dwarf the signal (story-averaged, up to \(-0.86\)), decomposes exactly into an order pull and a word pull, carries no verdict-attached component where that channel is precisely measured (a bound, not a zero, for Haiku), and loses its word-attached part under a label with no yes/no meaning—while residual, label-specific order and surface pulls can remain. The natural reading of a yes–no bias—that a forced model leans toward saying no, a disposition to reject—is therefore not what the measurement finds: where the word and the verdict are separated, no verdict-attached component survives; what survives is attachment to surfaces.

For measurement the corollary is concrete. To read a language model’s moral stance: elicit it graded, under crossed logically-equivalent frames; report the cross-form incoherence as the error bar, because it—not sampling noise—is the dominant uncertainty; and if a binary verdict is unavoidable, force it through a choice between the options rather than through yes/no. Any single-format number, including the standard yes–no bias, confounds the stance with the format.

Cheung et al. [9] correctly identified the phenomenon: the yes–no bias of LLMs is real and large. What a single-framing design cannot do, in principle, is resolve its composition. Ours does, and the resolution is that the amplification is format—an order pull with no human analog, plus a lexical pull toward a word—while the moral scale beneath it is more coherent than the artifact makes it look. Because the models differ across studies, we engage Cheung et al.’s argument, not their numbers; what makes the engagement direct is the shared materials—their vignettes verbatim, only the closing question exchanged [9], [49].

The measurement idea also fixes the paper’s place in its neighborhood. One literature measures what models answer on moral items [13], [24], [25], [30]–[32], [34], [35], [50]–[59]; another measures how much any answer moves with presentation [15], [16], [22], [60]–[62]—and the movement’s direction is itself unsettled across tasks: primacy on shuffled label lists [63], recency across survey options [64], token-identity pulls in multiple choice [17], and a “no”-leaning inversion of classic acquiescence [65] that matches the direction we measure. Of fourteen moral-judgment studies we examined in depth, five elicit each judgment in one fixed wording, four vary two to four uncrossed reframings, and five vary form systematically while crossing at most two presentation factors; none crosses verb, order, and label, and none uses such a factorial to decompose the bias into mechanisms—the nearest designs score perturbation sets as consistency rates [10], average template-and-order variation out rather than attributing it [13], or remap the answer scale without a verdict factor [11], [12], [36]. Treating models as subjects of behavioral measurement is by now a program [27], [66]–[73], including direct human–model moral comparison [33], [74]–[77], and a psychometric turn in LLM evaluation is underway [21], [78]–[85]; our contribution to it is the instrument: crossed symmetrization turns format sensitivity from a nuisance to be averaged over into a measured, decomposed quantity, and it answers, on these materials, Cheung et al.’s own call for evaluations that assess consistency across logically equivalent questions [9]. The human-side canon reads here as a design manual rather than a comparison target: order and framing move human moral judgments too [5]–[8], [86]–[88], repetition alone yields drift [89], judgments and choices dissociate [90]–[92], and the survey-methodology classics supply the levers we crossed [39], [42], [43], [93]–[95].

Humans, on the same materials, are framing-robust (a mean framing shift of \({\approx}0.12\) in choice proportion, \({\approx}0.24\) on our \(\pm1\) convention, and not “no”-directed)—and where a frontier model’s verdict probabilities saturate toward 0 or 1, the human sample splits, sitting near 50/50 on the same forced choices. Both facts are from Cheung et al.’s existing data [9]; we run no new human experiments. Each person answered once, so the split cannot be parsed into hedging individuals versus decisive individuals who disagree—which is why we speak of the models’ saturation against the population’s split, and translate neither into the within-agent \((m,s)\) vocabulary; nor do we compare framing-stability magnitudes across species (a mean directional shift and a cross-form SD are not commensurate). What the human data anchor is the aggregate contrast: at the same level of description, the models’ amplification has no counterpart there.

The claims carry their scopes. The logistic is the canonical link from a continuous, coherent latent to a binary choice—a modeling principle, not a mechanism claim—and the interpretation of \((m,s)\) does not depend on that choice of link; but absolute \(s\) is not comparable where the readout clips (Haiku\(\cdot\)direct, the Geminis, the open-weight models—which are additionally read on their own \(\theta\) rulers), so there we trust only the \(m\)-ranking and within-model differences. “Logical bias \({\approx}0\)” is a statement about these contested sacrificial dilemmas and the frontier models we observe, not about morality at large. Level-1 coherence is substantial, not perfect—a residual anchor-direction floor of \(0.12\)–\(0.21\) remains—and it is within-model: the shared cross-model ordering is a side observation (SI Appendix, Table S2), not evidence of an objective moral axis; and because nineteen dilemmas are verbatim from a published study, the shared ordering and the human tracking could partly reflect training exposure to the materials and their published ratings—the within-model invariance and the artifact decomposition, which compare logically equivalent forms of the same exposure, are immune to this, and they are what the paper’s claims rest on. Gemini-3-Flash’s inverted ordering on several dilemmas is a genuine stance difference, verified in its raw responses, and is reported as such rather than excluded (SI Appendix, section S8). The small-model contrast rests on two open-weight models, and they fail on different axes: Qwen is the deterministic endpoint of the stance-incoherence spectrum—a differentiated, consensus-correlated ordering buried under cross-form scatter that deliberation removes—while Nemotron is the instrument’s degenerate case, a scale too compressed to differentiate the items (\(G=0.28\)), which we accordingly report in the SI rather than beside the functioning scales, and which nonetheless retains a verdict-attached bias no frontier model shows (SI Appendix, section S6). Neither pattern is a statement about open models generally; together they carry a methodological moral: a coherence number without a discrimination check can flatter a scale that is merely indifferent—low spread is not a scale—which is exactly what the generalizability coefficient is for. Finally, in the models carrying a substantial artifact the same signature appears at both levels—deliberation tightens the scale and shrinks the susceptibility—suggesting a fast-pathway artifact that reflection partially overrides (a resonance with dual-process accounts of human moral judgment [96]–[100], offered as analogy, not mechanism); the degenerate case is again the instructive exception: with no functioning scale to tighten, deliberation wakes Nemotron’s scale (\(G\) \(0.28\!\to\!0.56\), consensus alignment \(r^2\) \(0.22\!\to\!0.69\)) without lowering its spread, and inflates its binary artifact—deliberation is a lever, not a guarantee (SI Appendix, section S6). One dimension is deliberately held fixed throughout: the responder’s persona. Every judgment here is elicited from each model’s default assistant stance; how the scale and its artifacts move when the same model answers under systematically varied personas or system prompts is a question the instrument takes up unchanged. The battery is not tied to these materials: the instruments and the crossing principle apply unchanged to any dilemma set and extend to richer form families. Expanding all of these—more dilemmas, more forms, the story text itself as a manipulated variable rather than a held-fixed one, and the responder’s persona as a crossed factor—and measuring deliberation and commitment as objects in their own right, are the natural next steps.

Materials and Methods↩︎

Dilemmas and materials. Twenty moral dilemmas: 13 sacrificial/greater-good dilemmas and 7 omission counterparts, rendered from a faithful transcription of the vignette materials of Cheung, Maier, and Lieder [9], [49]. Nineteen items are verbatim Cheung (per-item story_sha256 over all \({\sim}200\)k trials collapses to the hash of the source text); one (item A) is adapted from an omission default to an active decision and therefore carries no human anchor (SI Appendix, section S7). The dilemmas span personal/impersonal sacrifice, altruistic redistribution, and assisted-dying scenarios; each has an action pole (commit the act; the cost–benefit-reasoning pole, CBR) and a rule/deontological pole.

Model panel. Seven frontier configurations across three families—Claude Sonnet 4.6 (claude-sonnet-4-6) and Claude Haiku 4.5 (claude-haiku-4-5), each with extended reasoning off/on (“direct”/“think”); GPT-5.5 (gpt-5.5, low reasoning effort); Gemini-3-Flash (gemini-3-flash-preview, high thinking) and Gemini-3.1-Flash-Lite (gemini-3.1-flash-lite)—plus Claude Opus 4.8 (claude-opus-4-8, maximum effort; instruments I-1/I-2 only) and two small open-weight mixture-of-experts models read out at the logit level (greedy, single pass: the full answer-slot distribution) in both reasoning modes: Qwen3.6-35B-A3B and NVIDIA Nemotron-3-Nano-30B-A3B, the latter collected as a full replication of the former’s battery (same forms, same salt strings, same behavioral design; SI Appendix, section S4). GPT-5.5’s run additionally probed two pilot items; all cross-model analyses use the shared 20. “Frontier” claims about the forced binary quantify over the seven I-3 configurations; Level-1 (I-1) statements include Opus. Only the Claude models expose a true no-reasoning mode; GPT-5.5 and the Geminis run with nonzero reasoning by default (low and high effort, respectively). Every prompt is a fresh, context-free, single-turn API call, and every sampled model runs at temperature 1 (SI Appendix, section S4). The Claude models receive an empty system prompt; GPT-5.5 the one-line instruction “Respond only to the user’s survey stimulus. Do not use tools.”; the Gemini models the one-line system prompt “You are a helpful assistant. Answer the user’s question directly and follow their formatting instructions exactly.” Exact invocations, decoding settings, and refusal handling are in SI Appendix, section S4. Refusals and unusable answers are excluded from cells; on the core verb-flip pool the rates are Sonnet \(0.7\%/0.5\%\) (direct/think), Haiku \(28\%/35\%\) (textual refusals), GPT-5.5 \(0.0\%\), Gemini-3-Flash \(1.7\%\), and Flash-Lite \(24\%\) (verdict-withholding essays); the matched 12-form comparison (below) is computed on the items whose cells survive exclusion. Resampling noise is quantified directly on both instruments. On the graded scale the repeat component is \(\sigma_{\mathrm{repeat}}=0.02\)–\(0.10\), small beside \(\sigma_{\mathrm{repro}}\) (SI Appendix, section S3). On the forced binary, 82–92% of form\(\times\)item\(\times\)side cells are unanimous across replicates for the well-behaved configurations, between-form variance exceeds within-form (replicate) variance by up to an order of magnitude (Sonnet \({\sim}9\)–\(19\times\) on the matched pool, \({\sim}25\times\) with print order included), and 15–50% of items have at least two logically equivalent forms yielding opposite majority verdicts; the heavy-refusal and saturated configurations are the expected boundary cases (SI Appendix, section S3). Repetition of a single form therefore underrepresents the variation that matters. The frontier–open-weight comparison crosses two readout channels (sampled text versus greedy logits); each open-weight model’s behavioral samples calibrate its logit readout (behavioral versus logit choice probability: Qwen \(r\approx0.90\), Nemotron \(r=0.93\) at 24 replicates, with fitted readout temperature \({\approx}1\); SI Appendix, section S4), and its salt replications supply the within-form noise floor the greedy channel otherwise hides. In total \({\sim}389{,}000\) primary trials (I-1 \(34{,}356\); I-2 \(6{,}475\); I-3 \(348{,}151\); plus \(2{,}880\) Opus graded trials, indexed separately) and \({\sim}47{,}000\) auxiliary open-weight-model trials (salt replications and behavioral calibration, both models).

I-1: the graded scale and \(\theta\). Each dilemma is rated under 48 crossed conditions: scale \(\in\{0\)–\(10\), \(0\)–\(100\), and two bipolar variants\(\}\) \(\times\) wording \(\in\{\)acceptable, right, should\(\}\) \(\times\) anchor direction \(\in\{\)ascending, descending\(\}\) \(\times\) pole \(\in\{\)action, complement\(\}\) (the full design table, with real prompt text for every instrument: SI Appendix, section S1). The two bipolar variants are excluded from \(\theta\) by the frozen, pre-registered protocol (instrument definitions and estimator in timestamped files that precede the first production run; SI Appendix, section S9): they deliberately reintroduce the negative-anchor positivity bias known from the survey literature [93], [101]–[103], as a measured contrast to be quantified—never averaged into the stance. The measured contrast is small (per-model mean \(|\theta_{\mathrm{bipolar}}-\theta|\le0.06\)); recomputing the per-item \(\sigma_{\mathrm{repro}}\) with the bipolar cells included moves each frontier model’s mean by at most \(0.04\) (Qwen’s by \(0.06\), downward; Nemotron’s by at most \(0.02\); SI Appendix, section S3). This leaves 24 unipolar conditions in the \(\theta\) estimator. A rating \(v\) is normalized to acceptability \(a=(v-\mathrm{lo})/(\mathrm{hi}-\mathrm{lo})\), flipped to \(1-a\) under a descending anchor. Per item, \(\theta=\overline{a}_{\mathrm{action}}-\overline{a}_{\mathrm{complement}}\in[-1,1]\). \(\theta\) is a chosen coordinate (no absolute zero or unit). Replication depth: 3–8 per condition for the sampled models (open-weight models: single logit pass; error bars from salt replications, below).

I-2: free choice. The model freely chooses between the two courses of action under twelve framings, with no scale and no yes/no label; verdicts (PRO/ANTI/NONCOMMIT) are judged from the free text by Claude Opus 4.8 (high effort). Verdict extraction is a transcription task—the transcript states a choice—but the judge shares a vendor family with several subjects, a circularity risk we flag; per-form verdicts and raw transcripts ship with the data for audit. I-2 supplies the convergent-validity checks of SI Appendix, section S5: it is the most coherent elicitation we observe (cross-form \(\sigma_{\mathrm{repro}}\approx0\)) and the most saturated (\(|\mathrm{stance}|\approx0.86\) on the \(\pm1\) axis).

I-3: the forced binary, and the projection framework. Verb-flip forms cross the question’s verb (approve/oppose) with the printed answer order (yes-first/no-first); of the 126 forms in the design, Claude ran 49 (Sonnet\(\cdot\)think: 50) and every other model the 12-form baseline, on all 20 items (replication r8 for Sonnet/GPT/Flash-Lite, r10–12 across batches for Gemini-3-Flash, r4 for Haiku—a budget choice fixed in advance—and r1 logit for the open-weight models). Because the corpora differ in breadth, the cross-model comparison was re-run on the matched 12-form baseline for all models: the Claude decompositions move by \(\le0.06\) and away from zero (Sonnet \(-0.32=-0.21-0.12\); Haiku \(-0.92\), on the 17 items whose cells survive refusal exclusion there), so the family contrast is not a wording-corpus artifact (script ref2_matched12.py; SI Appendix, section S4). The label-map instruments below are separate corpora run on every model (verb\(\times\)label\(\times\)order: \(26{,}617\) trials; the 13-pair label\(\times\)order map: \(45{,}092\) trials; full census in SI Appendix, section S4). From the four verb\(\times\)order cells with per-cell \(P(\mathrm{CBR})\), the story’s I-3 stance is the mean over all four, \(z=\langle 2P(\mathrm{CBR})-1\rangle\)—one value per story, independent of which bias is examined—and each bias is a balanced 2-vs-2 split of the same cells sharing that stance: \[\begin{align} b \;&=\; p_{+}-p_{-},\qquad \tfrac12\,(p_{+}+p_{-})=\overline{P(\mathrm{CBR})},\\ b_{\mathrm{apparent}} \;&=\; b_{\mathrm{order}}+b_{\mathrm{lexical}} . \end{align} \label{eq:split}\tag{1}\] Concretely, with \(\mathrm{bias}_o = P(\mathrm{CBR}\,|\,\mathrm{approve},o)-P(\mathrm{CBR}\,|\,\mathrm{oppose},o)\) for each printed order \(o\): the lexical channel is \(\tfrac12(\mathrm{bias}_{\mathrm{yf}}+\mathrm{bias}_{\mathrm{nf}})\) (the order-balanced verb split, equal to \(\langle 2P(\mathrm{yes})-1\rangle\)); the order channel is \(\tfrac12(\mathrm{bias}_{\mathrm{yf}}-\mathrm{bias}_{\mathrm{nf}})\); and the apparent (single-framing, yes-first) bias is \(\mathrm{bias}_{\mathrm{yf}}\), so the identity in Eq. 1 is exact and definitional—its content is that the two summands are separately measurable and separately meaningful: the order part flips with print order, the lexical part survives order-balancing. The independent axis \(\theta\) always comes from I-1, so the bias test never uses the biased format to measure the stance it is biased about.

Identification: lexical versus logical. In yes/no data the word and the verdict are one token, so the verb-flip lexical channel conflates them. The full verb\(\times\)label\(\times\)order design (\(2\times2\times2\)) carries the verdict on labels other than the English words—the fully arbitrary A/B; the valenced mark pair \(+/-\); Chinese yes/no words for the cross-lingual lexical case—adding the logic projection (attachment to the verdict when the verdict is carried by a label without the English word) and the label projection (attachment to the printed label itself, counterbalanced over the label\(\leftrightarrow\)verdict mapping). The identification claim rests on the fully arbitrary family: empirically the A/B logic is \({\approx}0\) for every frontier model (Fig. 5), which licenses the cheaper label\(\times\)order \(2\times2\) used for the 13-pair gradient of Fig. 6.

The two-parameter fit. For each bias source we fit \((s,m)\) jointly to that bias’s two sides by nonlinear least squares, \(p_{+}=\sigma((\theta+m)/s)\) and \(p_{-}=\sigma((\theta-m)/s)\) stacked, with \(\theta\) from I-1; the shared stance \(\tfrac12(p_{+}+p_{-})\) pins the origin, so \(s\) is not inflated by averaging over the offset. Each fit generates the stance sigmoid, the bowtie locus, and the ridge of Fig. 3; because \(z(\theta)\) depends only weakly on \(m\), the order-fit and lexical-fit stance curves must overlap, which they do (the check is weak by construction; goodness of fit is reported separately). Goodness of fit on the stacked two-side data: \(R^2 = 0.77\)–\(0.84\) (RMSE \(0.14\)–\(0.17\)) for the resolvable configurations; Haiku\(\cdot\)direct’s order channel is worst (\(R^2=0.57\)), consistent with its degeneracy flag. Saturated or degenerate configurations read as clip-limited \(s\) (Haiku\(\cdot\)direct large-\(s\) degenerate; the Geminis’ saturated stance reads as small \(s\)); for these only the \(m\)-ranking and within-model differences are interpreted. Identifiability of \(m\) varies with saturation: the number of dilemmas within \(|z|<0.9\) of a model’s tie is 3 (Gemini-3-Flash), 7 (GPT-5.5), 10 (Flash-Lite), and 11–19 (the Claude configurations); a saturated model’s fitted \(m\) is additionally attenuated toward zero. Estimator properties: \(\theta\) enters as a regressor with incoherence of its own (drawn as \(x\) error bars in Fig. 3), and such errors-in-variables attenuate \(|m|\) and inflate \(s\), so the fitted susceptibilities are conservative; \(m\) is the aggregate offset of a wording family—item-level residuals of \(\pm0.5\) exist and cancel (the K01 anchors)—so “portable” means a story-independent estimator, not an item-homogeneous mechanism. CIs are percentile bootstraps over dilemmas (\(\geq\)​2,000–10,000 resamples, fixed seeds), with no multiplicity correction; percentile intervals on \(n=20\) clusters can undercover slightly, so boundary-grazing limits (e.g., \(+0.001\) or \(-0.002\)) are read as marginal (coverage discussion: SI Appendix, section S3). The symbol \(b\) always denotes a signed bias on the \([-1,1]\) axis; each figure’s per-family definition is the same quantity computed in that family’s cells.

Raw versus fitted. The story-averaged biases (Fig. 2) and the fitted parameters (Fig. 4) are two presentations of the same verb\(\times\)order data. The raw mean is story-distribution-dependent (saturation collapses it toward zero far from the tie); the fitted \((s,m)\) are story-independent and diagnose that saturation. Neither replaces the other: raw numbers compare models on a fixed item set, parameters are portable.

Cross-form incoherence (the reported error bar). Every error bar on a stance-like quantity is the format incoherence of that quantity, not the standard error of its mean: the spread of the per-form value across logically equivalent forms, corrected for finite sampling (a measurement-systems “Gauge R&R” correction [104]), \(\sigma_{\mathrm{repro}}=\sqrt{\max(0,\;\mathrm{Var}_{\mathrm{forms}}(x_f)-\overline{\mathrm{Var}_{\mathrm{samp}}(x_f)})}\), where \(x_f\) is the per-form value (\(\mathrm{Var}_{\mathrm{forms}}\) over the format cells—I-1: up to 24 per item; I-3: the family’s forms—and \(\mathrm{Var}_{\mathrm{samp}}\) from the 3–8 replicates per cell). The \(\max(0,\cdot)\) floor means small values are resolution-limited rather than exact zeros. \(\sigma_{\mathrm{repro}}\) is expressed in the units of the \([-1,1]\) axis; cross-model comparison rests on the shared normalization and identical item set. Components combine in quadrature (\(\sigma_{\mathrm{repro}}^2=\sigma_{\mathrm{dir}}^2+\sigma_{\mathrm{scale}}^2\), treated as additive; \(\sigma_{\mathrm{tot}}^2=\sigma_{\mathrm{repro}}^2+\sigma_{\mathrm{repeat}}^2\)), so the decomposition of Fig. 1b is drawn by variance share, never stacked linearly. Two estimators of the per-model incoherence appear: the headline values and the Fig. 1b bars are the frozen per-model aggregate (a pooled variance decomposition across all format cells), while uncertainty statements use the mean per-item \(\sigma_{\mathrm{repro}}\) with a \(10{,}000\)-resample bootstrap over items; the estimator comparison is tabulated in SI Appendix, section S3. The generalizability coefficient [105]–[108] is \(G=\mathrm{Var}_{\mathrm{items}}(\theta)\,/\,(\mathrm{Var}_{\mathrm{items}}(\theta)+\overline{\sigma_{\mathrm{repro}}^{2}})\), an ICC analog for a single randomly chosen form. Two error-bar regimes appear in the paper: incoherence spreads (Figs. 1a and 3; descriptive) and bootstrap CIs (Figs. 2 and 4–6; inferential)—overlap comparisons are licensed only for the latter. For the deterministic open-weight logit readout, genuine replications are created by salting: prepending a meaningless hexadecimal context string. Salt is itself a logically irrelevant perturbation—of the context, not the question form—and the argument is comparative: if the between-form spread were sampling noise, the same spread would appear within form under salt; it does not. Salting leaves \(\theta\) unchanged (\(r=0.993\) to the unsalted pass) while exposing the within-form floor; the between-form SD within the salt experiment’s unipolar subset (\(0.345\); the full-design quadrature total is the \(0.40\) quoted throughout) versus the within-form/salt SD (mean \(0.072\); median \(0.000\)) establishes that its incoherence is dominantly deterministic, not sampling noise (SI Appendix, section S3). This determinism claim is Qwen’s; Nemotron’s salt floor is different in kind—ubiquitous, small logit jitter (mean \(|\Delta p|=0.02\))—and is likewise subtracted by the correction (SI Appendix, section S6).

Reading “\({\approx}0\).” Throughout, “\({\approx}0\)” summarizes a point estimate \(|b|\le0.06\) whose CI spans zero; it is a non-rejection shorthand, not an equivalence test—wide channels (e.g., Haiku’s logic cells, \(\pm0.2\)–\(0.3\)) are underpowered rather than null and are flagged as such, and the same reading is applied on both sides of zero.

The Cheung-verbatim control. One verb-flip family (K01) reproduces Cheung et al.’s question wording verbatim—the one-line answer instruction is our standardized one—and crosses only the printed answer order on top of it, across all 20 items at full depth (r8; run on Sonnet, both reasoning modes). Its yes-first apparent bias (\(-0.12\)) order-balances to \(+0.01\), 95% CI \([-0.12,+0.16]\) (cluster bootstrap over items)—consistent with no systematic residual at this precision, though the interval does not exclude word-level residuals of the size reported for our own wording families; per-item anchor residuals are real (up to \(\pm0.5\)) and cancel across items.

Human data. All human values are re-extracted from Cheung et al.’s published data [9], [49] (Study 1, \(N=285\), within-subjects, action framing; Study 2, \(N=474\), between-subjects): per-dilemma choice rates, averaged over framing conditions, supply the Fig. 1a anchors; the framing shift (\({\approx}0.12\) in choice proportion; Study 2) and the near-50/50 split are quoted in the Discussion (each participant judged each dilemma once). We collected no new human data.

Data availability↩︎

All raw trials (JSON/JSONL), derived quantities, and analysis code regenerate every number and figure deterministically from raw via a single pipeline (run_all.py; one canonical function per quantity); provenance, pre-registration artifacts, and integrity notes are consolidated in SI Appendix, section S9. [Deposition DOI to be added at submission.]

Author contributions. H.H. conceived the project, designed the research, performed the experiments, analyzed the data, audited the code, and wrote the manuscript.

Competing interests. The authors declare no competing interest.

References↩︎