1 Introduction↩︎

LLMs are already taking morally significant actions. They offer personal moral advice to millions of users [1], [2]. They are used in content moderation systems that adjudicate permissible speech [3]. They guide medical doctors’ diagnoses [4], [5], and lawyers’ research and litigation decision-making [6], [7]. And they are increasingly not just advising human agents, but acting as agents themselves [8]. For AI systems to engage acceptably in these morally-freighted behaviors, they must be morally competent—that is, able to identify the morally relevant features of the situations before them, and reason from them to a sensible conclusion about what to do [9], [10]. The foundation for moral competence is moral sensitivity, the ability to pick out the morally relevant features of a choice situation [11]–[14]. Evaluating LLM moral sensitivity is therefore an urgent and important challenge.

Existing evaluations of LLM moral sensitivity have at least three limitations [9]. They assess model performance against ad hoc datasets of cases that do not cover a plausible universe of scenarios and so are uninformative about model moral competence in general (as distinct from in some subdomain). They evaluate models against highly curated moral vignettes, which do not test the ability to pick out relevant features from the kinds of noisy, unfiltered data streams required to test true moral discernment. And they do not scale: they rely either on costly and sometimes compromised human baseline comparison, or else on LLM judges that face a bootstrapping problem.

We introduce MORPH-1K (MOral Robustness under Perturbed Hypotheticals), a procedurally-generated 1,000-case benchmark spanning 50 moral foundation-pole combinations across four social domains, as well as code for further expanding the dataset. MORPH-1K addresses each of these three problems. It provides 1,000 cases generated based on a coherent and well-grounded taxonomy of normative domains. It develops an automated pipeline for perturbing these cases in three distinct ways without changing their moral valence (and deterministic checks to ensure the latter). And it offers a new method for scalably evaluating model performance in the transformed cases.

The logic is as follows. First, we draw on existing human-baseline work to establish that the model performs well on clean base cases, that is, moral vignettes presented without noise or distraction [12], [14]–[18]. This gives us a well-founded starting point: the model is sensitive to the right moral features when those features are presented clearly. The question then becomes whether that sensitivity is preserved under perturbation via the introduction of non-moral noise. If a model’s responses to moral vignettes remain semantically equivalent before and after transformations that do not alter the morally significant features of the scenario, then we have evidence that the model is successfully identifying the right features in the perturbed cases. If, on the other hand, responses shift substantially under morally irrelevant perturbations, then the model is sensitive to features it should be ignoring, suggesting a lack of genuine moral sensitivity. Establishing invariance requires no judgement about response quality, bypassing the need for either additional human data or LLM-as-judge.

We make four contributions. First, we develop a two-dimensional framework crossing social domains with moral foundations to enable generation of 1000 moral vignettes across a comprehensive space. This ensures that evaluation is evidence of general, rather than domain-specific, moral competence.

Second, we evaluate LLM moral sensitivity under challenging conditions including textual distractors [19], irrelevant detail additions [12], and embedded chat histories (from [20]). These are illustrative; our method can be extended to other perturbations.

Third, we introduce a novel scalable method to evaluate moral sensitivity, using semantic similarity analysis as an alternative to human baseline comparison and LLM-as-a-judge.

Fourth, we use this pipeline to offer an optimistic assessment of LLM moral competence: contrary to recent work (e.g. [12], [19]) we show that LLMs identify substantially the same morally relevant features whether in the original, clean cases, or in the perturbed cases. Across all eight models tested, perturbations significantly altered the number of features returned but left their semantic content stable above our invariance threshold. This contrast between count-level variance and semantic invariance suggests that morally irrelevant noise affects the format and granularity of model output more than its underlying moral content, and demonstrates a replicable methodology for evaluating moral reasoning at scale without dependence on human baselines or LLM judges.

2 Related Work↩︎

Our work contributes methodologically and substantively to the growing literature on evaluating the moral competence of large language models (for a recent review, see [9]; see also [10], [12], [14], [19]. That literature is motivated by widespread recognition that LLMs are going to power highly capable autonomous agents that will need to operate safely in open-ended, dynamically changing environments. For them to do so, they must be able to recognise the morally relevant features of new situations that they confront, associate those features with reasons in a sensible way, and bring those reasons together into a reasonable conclusion about what to do, which then constrains their action [9], [12]. To do all this requires moral competence. Researchers have explored LLM moral competence primarily by presenting models with curated vignettes inviting morally relevant decisions.

The recent literature is divided in its assessment of LLM moral competence; some positive early signals (e.g. [15]) have been replaced with pessimism [10], [12], [14]. We suspect that some of this newfound pessimism derives from inadequate evaluation methodologies. Studies of moral sensitivity in particular face three notable limitations.

First, they too often focus on LLMs’ ability to extract morally relevant features from a highly-curated textual vignette. This is the most favourable possible condition for the LLM. Real moral problems do not arrive carefully prepackaged. Second, the datasets used to evaluate LLM moral competence are often small, or else ad hoc, with no clear sense of what would be necessary in order to genuinely say that LLMs have well-rounded moral competence. Relatedly, third, nobody has yet developed a scalable and defensible method to evaluate moral sensitivity across the range of cases that might plausibly justify claims of broad moral competence. Existing methods rely heavily on human-baseline comparison, whether through "moral Turing tests" comparing blinded human and AI responses, or through extremely expensive curation of human rubrics for evaluating cases.

2.0.0.1 Human Baseline Evaluation

The gold standard for evaluating normative competence in LLMs has been to benchmark model outputs against human responses. The logic is straightforward: in the absence of ground truth about what constitutes a morally appropriate response, human judgement provides the best available proxy (see, for example [12], [14], [15], [21]). And there is good reason to think this proxy is reliable. Humans are, by and large, competent moral reasoners, particularly when presented with cases that have clear normative valence. However, collecting high-quality human evaluations is expensive, slow, and difficult to reproduce at the scale required either to keep pace with rapid model development, or to cover the necessary universe of relevant cases.

These practical constraints have been compounded by an emerging and more fundamental threat: the contamination of crowd-sourced survey data by AI-generated responses. Online recruitment platforms such as Mechanical Turk and Prolific have become essential infrastructure for human-baseline research, but the integrity of data gathered through these platforms is increasingly in question. Inauthentic or fraudulent responses are rising, with estimates suggesting that between 30% and 90% of responses may be compromised [22], [23], and evidence that even modest contamination rates of 3–7% can be sufficient to distort results and render interpretations unreliable [24]. One recent study found that LLM-mediated answers accounted for up to 45% of survey submissions [25]. The advent of AI agents capable of autonomously navigating websites threatens to escalate this further, making the production of authentic-seeming survey responses trivially easy even for users without technical expertise [26].

2.0.0.2 LLM-as-Judge

The need for scalable alternatives to human evaluation has driven the development of the LLM-as-judge paradigm, in which a strong language model is used to evaluate the outputs of other models [27]. The approach has yielded genuine improvements across a range of evaluations: strong LLM judges can achieve over 80% agreement with human preferences on general-purpose benchmarks, at a fraction of the cost [27]. LLM-as-judge methods replace or supplement human evaluation by prompting a strong model to score, rank, compare, or critique other model outputs. Examples now span summarisation and dialogue evaluation (G-Eval; [28]), machine translation quality estimation (GEMBA; [29]), multi-turn chatbot benchmarking (MT-Bench; [27]), and rubric-based assignment grading at classroom scale [30].

The appeal is clear: LLM judges are fast, cheap, flexible, and sometimes correlate reasonably well with human judgements. However, the same literature also shows that they are imperfect measurement instruments. LLM judges can exhibit systematic biases, including position bias, verbosity bias, self-enhancement or self-preference bias, and broader cognitive or fairness-related distortions [27], [31]–[33]. Large-scale studies find that LLM-judge performance varies across tasks, domains, annotator populations, and evaluation protocols, and that fine-tuned judge models often generalise poorly outside the settings on which they were trained [34], [35].

There is a more obvious problem when using LLMs as a judge to evaluate moral competence, and in particular moral sensitivity. The target behaviour consists in picking out the morally relevant facts from a decision situation—preferably an unfiltered, noisy scenario that has not been excessively curated in advance. For one LLM to be an effective judge of the other’s performance at this task, we would have to know that the judge model already displayed high levels of moral sensitivity. But that capacity is precisely what we are trying to evaluate. This is an acute bootstrapping problem. In the absence of any ground truth labels against which to compare model performance, using LLMs to judge other LLMs moral sensitivity amounts to having the students mark each other’s homework.

3 Methods↩︎

3.1 Motivations and Case Generation Framework↩︎

The preceding limitations motivate a methodological shift. Rather than treating moral evaluation as a problem of judging whether each model response is correct, we treat it as a problem of testing whether a model’s moral feature identification is stable under changes that should not matter morally. This reframes moral sensitivity as an invariance property: if two prompts preserve the same morally relevant structure, then a morally sensitive model should identify substantially the same morally salient features across both. This approach is especially useful in domains where there is no simple ground truth, where human baselines are costly or increasingly contaminated, and where LLM-as-judge methods risk importing the biases of the evaluator. Our alternative approach instead turns evaluation into a deterministic comparison between paired responses, with no evaluator model in the loop. It also separates two questions that are often conflated: whether a model can produce plausible moral reasoning in simple cases, and whether that reasoning remains robust when the input contains distraction, noise, or realistic contextual clutter. Our method is designed to probe the second question. To do so systematically, we require a controlled space of moral cases that varies both the kind of moral concern at stake and the social context in which it arises.

Our framework provides the needed structure, allowing us to generate moral vignettes across a broad normative space. We organise moral vignettes along two dimensions: the type of moral concern at stake, and the social domain in which that concern arises. The first dimension draws on Moral Foundations Theory to ensure coverage across qualitatively distinct categories of moral content. The second dimension orders social domains from proximal to distal, ensuring coverage across the range of contexts in which moral reasoning is exercised. Crossing these two dimensions yields a structured space of moral scenarios that is both comprehensive and tractable, enabling systematic generation of vignettes that vary independently in moral content and social context.

Moral Foundations Theory [36], [37] proposes that human moral judgement is organised around a set of five evolved psychological systems, each sensitive to a distinct class of social challenge: Care/Harm, concerned with suffering and the protection of the vulnerable; Fairness/Cheating, concerned with proportionality, reciprocity, and justice; Loyalty/Betrayal, concerned with obligations to the in-group; Authority/Subversion, concerned with hierarchy, respect, and social order; and Sanctity/Degradation, concerned with purity, contamination, and the sacred. The theory has been widely adopted as a framework for studying moral judgement in both humans and language models (e.g., [38]–[41]). We adopt it here not as a commitment to its particular evolutionary claims, but because it provides a well-validated and widely recognised taxonomy of moral content that enables systematic variation across qualitatively distinct types of moral concern.¹

Regarding our second dimension: The idea that moral reasoning varies systematically with social scope is well established. [43] influentially argued that reason progressively extends moral concern from kin outward to all sentient beings, and [44]’s stage theory traces a structurally parallel expansion from self-interested reasoning through group-oriented morality to universal principles. Our four domains (Interpersonal, Professional, Societal/Institutional, and Outgroup/Expansive Circle) draw on these traditions but differ from them in two respects. First, unlike Singer’s linear expansion or Kohlberg’s developmental stages, we treat our domains as structurally distinct sites of moral activity presenting qualitatively different reasoning demands, rather than stages to be traversed through. Second, unlike frameworks developed to theorise the scope of moral obligation, ours is designed as an evaluative scaffold: the proximal-to-distal ordering provides a principled sampling strategy ensuring that assessment covers a range of contexts sufficient to evaluate general rather than domain-specific moral competence.

3.2 Evaluation Pipeline↩︎

3.2.0.1 Vignette Generation

We provided Claude Opus 4.6 [45] the eMFD dictionary which includes probability and sentiment scores of over 3000 words [38] and asked it to create themes that exemplified each of the moral foundations based on this data. It produced a total of 52 themes with 5–6 themes for each virtue and vice pole of 5 foundations². To increase the variety of moral dilemmas, we combined 2 themes from 2 different foundations. In total, we were able to create 1,133 unique theme combinations per domain.

We asked OpenAI’s GPT-5.4 [46] to generate 4,532 vignettes covering all 50 foundation combinations (10 single, 40 paired) across 4 domains (Interpersonal, Outgroup, Professional, Societal). After filtering out 318 rejected cases (7.0%)³, we ran a stratified selection over the remaining 4,214 to produce a balanced subset of 1,000. The selection assigns an equal base quota of 5 vignettes to each of the 200 cells in the design matrix (foundation combo \(\times\) domain). Four constrained cells, mostly involving Sanctity, contribute their full supply of 3–4. Within each cell, we prioritised vignettes with the highest foundation probability score to favour the clearest exemplars. The result is 1,000 vignettes covering all 50 foundation combinations, all 4 domains (250 \(\pm\) 2 each), and 604 unique themes (i.e. those that survived the stratified selection).

3.2.0.2 Transformations

To interpret moral sensitivity under morally irrelevant transformations, the transformations must satisfy two constraints. First, they must preserve the morally relevant structure of the original vignette; otherwise, response shifts could reflect genuine changes in the case rather than distraction. Second, for transformations intended as stress tests, they must be plausibly capable of affecting model behaviour. A null result would be hard to interpret unless the perturbations were plausibly capable of changing behaviour: if the added noise were too weak to make any difference, stability would provide little evidence of robustness. We therefore use two experimentally motivated perturbations with direct precedent in prior work: textual moral distractors from [19], and morally irrelevant detail additions following [12]. The chat-history condition is motivated differently. Rather than treating it as an independently validated stress test, we use unrelated chat histories from [20] as an ecologically motivated deployment condition, testing whether moral feature identification remains stable when the target vignette appears alongside irrelevant conversational context.

3.2.0.3 Moral Textual Distractors

We used the textual distractors from [19], which are a subset of IDEST: International Database of Emotional Short Texts [47] and consist of 30 short narratives: 10 negative, 10 neutral, and 10 positive, as scored by human annotators and selected by the authors.⁴ For each vignette, we randomly assigned one distractor narrative and inserted it into the user prompt before the target vignette.

3.2.0.4 Morally Irrelevant Details

Following [12], we asked GPT-5.4 [46] to add morally irrelevant details to each vignette while leaving the morally relevant structure of the case unchanged. We accepted modified vignettes only when their moral-to-non-moral ratio was lower than that of the original vignette, as measured using eMFD scoring [38]. To create a stronger noisy variant, we then requested a second edit adding contextually irrelevant moral features (features that might be morally relevant under other circumstances), again without altering the underlying moral content of the vignette. These vignettes were independently validated by two experts using sequential sampling. At 30 vignettes the two experts had almost total agreement that the additions to the vignettes were irrelevant to the moral dilemmas.

3.2.0.5 Chat Histories

To approximate a realistic setting in which a target query appears alongside unrelated prior conversation, we inserted randomly selected English-language chat histories from [20] into the message history before the model was asked to respond to the vignette.⁵

3.2.0.6 Collecting Morally Relevant Features from LLMs

We evaluated 8 different proprietary and open LLMs available on the OpenRouter marketplace: OpenAI’s GPT-5.4 [46], Google’s Gemini 3.1 Pro Preview [48], Anthropic’s Claude Opus 4.6 [45], xAI’s Grok 4.20 [49], Z.ai’s GLM 5.1 [50], Qwen’s 3.6 Plus [51], NVIDIA’s Nemotron 3 Super [52] and Google DeepMind’s Gemma 4 31B instruct model [53]. The API calls were made using OpenAI’s SDK with reasoning effort set to ‘high’ and other parameters as per model author’s default settings⁶. We asked models to provide all morally relevant features they identify as a list in JSON format and sampled 3 responses for each vignette version of no noise and noise variants. Each response was collected with an individual API call without any other context.

3.2.0.7 Semantic Similarity Analysis

Our test requires a principled measure of whether a model’s responses to moral vignettes remain semantically equivalent before and after morally irrelevant transformations. We operationalise this using sentence embeddings and cosine similarity. Because both the embedding function and the similarity metric are fixed, this measurement is fully deterministic: given the same response pair, the cosine similarity is reproducible across runs and independent of any evaluator model. This is the central methodological advantage of the approach over LLM-as-judge. This approach draws on the distributional semantics tradition: as [54] put it, “you shall know a word by the company it keeps.” [55] formalised the same idea by arguing that differences in meaning correlate with differences in linguistic distribution. Later computational work translated this insight into dense vector representations, placing semantically related words near one another in vector space [56].

Because our unit of analysis is an entire model response rather than an individual word, we use Sentence Transformers [57], which extend this distributional approach to sentences and paragraphs.

We use the Qwen3 Embedding 8B model⁷, which maps each response to a 4096-dimensional dense vector [58]. For each vignette, we first combine all features returned from 3 samples without duplication, then encode the model’s base-case response and its transformed-variant response, and finally compute cosine similarity between the two vectors. For each feature, we take the maximum score and take the average of all features for the vignette. Cosine similarity ranges from \(-1\) to \(1\), with higher values indicating greater semantic alignment. Because it is invariant to vector magnitude, it captures the direction of the embedding representation rather than response length.

A high cosine similarity score between a base-case response and its transformed counterpart indicates that the transformation did not substantially alter the model’s identification of morally relevant features. A low score indicates that the model’s response shifted in ways that may track the introduced noise rather than the underlying moral content. To establish what counts as a low score, we calibrated an empirical floor for each model ⁸. We randomly paired vignettes drawn from different domain and foundation combinations, yielding 500 pairs whose responses should share little morally relevant content. For each model, we then computed the mean cosine similarity between paired responses, giving the expected similarity between responses to unrelated moral cases. These per-model floors ranged from 0.57 (Gemma 4 31B) to 0.69 (Qwen 3.6 Plus), with a grand mean of 0.61 across models. We treat these values as per-model empirical floor thresholds: a perturbation-condition similarity score that fails to exceed its model’s floor provides no evidence that the model is tracking the same moral content across the clean and perturbed responses, since responses to entirely unrelated vignettes are about as similar.

To check that our similarity metric is capable of detecting failures of invariance — and is not simply too coarse to register them — we additionally evaluated Qwen2.5 0.5B-Instruct (Qwen, 2024), a substantially smaller model expected to be more susceptible to morally irrelevant noise. We treat this not as part of the main evaluation but as a discriminative validity check on the metric itself: if the embedding-similarity approach is sensitive enough to register genuine failures of moral feature stability, a substantially weaker model should produce visibly lower scores, with at least some conditions approaching or falling below the empirical floor. Results are reported in Appendix G.

4 Results↩︎

We first compared the number of morally relevant features each model identified across conditions. If noise does not affect moral feature identification, the count should not differ between noisy and clean conditions. This was tested using Wilcoxon signed-rank tests with Bonferroni correction across multiple comparisons (\(\alpha = 0.0012\)). Contrary to our hypothesis, feature counts differed significantly between noisy and clean conditions for the majority of model–condition pairs. The exceptions were Gemini 3.1 Pro, Grok 4.20, and Qwen 3.6 Plus for the irrelevant-detail condition, and Nemotron 3 Super under negative distractors, neutral distractors, and chat history.

Second, we calculated the similarity scores of each noise condition against no-noise responses in both directions (noise vs. no noise and no noise vs. noise) because the number of features in the responses varied by condition. Across the eight main models, mean similarity scores ranged from 0.80 (Nemotron 3 Super) to 0.86 (Claude Opus 4.6, Gemini 3.1 Pro, Grok 4.20), with standard deviations between 0.02 and 0.05 (full per-condition statistics in Appendix E, Table 2). Critically, every model–condition mean exceeded its empirical floor by a substantial margin: the smallest gap was approximately 0.16 (Nemotron 3 Super, whose floor is 0.63), and for most models the gap fell in the 0.20–0.25 range. The models therefore identified very similar features across all noise conditions, with similarity scores well above the per-model empirical floor threshold.

The discriminative validity check on Qwen2.5 0.5B-Instruct (see Appendix G, Figure 4) supports the interpretation that this clearance reflects genuine invariance rather than metric insensitivity. Unlike the eight main models, the smaller Qwen2.5 0.5B produced markedly lower similarity scores, with several conditions clustering at or below the 0.61 empirical floor. The metric is therefore capable of registering failures of moral feature stability when they occur; the uniform clearance observed in the main analysis is not an artefact of the metric being too coarse to detect variance.

5 Discussion↩︎

Across eight models and five noise conditions, perturbations often changed the number of morally relevant features models listed. This count-level instability is important, because it shows that the transformations were not behaviourally inert. The added distractors, irrelevant details, and chat histories did affect model output in measurable ways. However, the semantic similarity analysis showed that these changes did not generally amount to a change in the moral content identified by the models. In all model–condition pairs, similarity scores remained above the minimum-invariance threshold set by our floor. The central finding is therefore not that models ignored the noise entirely, but that the semantic core of their moral feature identification was preserved despite it.

This distinction between count-level variance and semantic invariance is theoretically important. A model may list more features, fewer features, or divide the same moral content into differently worded items while still tracking substantially the same underlying moral considerations. Conversely, a model could preserve the same number of features while shifting to different content. Feature counts alone therefore provide only a coarse measure of robustness. The contrast between the Wilcoxon results and the embedding-similarity results suggests that morally irrelevant perturbations frequently affect the format or granularity of model responses, but not necessarily their substantive moral content.

5.0.0.1 Interpreting Invariance and Variance

Where invariance holds, it provides evidence that models are tracking moral structure rather than surface form. This is non-trivial: next-token training and post-training alignment offer no architectural guarantee that models will distinguish morally relevant from morally irrelevant information, yet under the conditions tested here, contemporary LLMs preserve their identification of moral features even when prompts contain affective distractors, irrelevant detail, or unrelated conversational history.

This robustness is not absolute. Perturbations frequently changed feature counts: positive textual distractors produced significant differences most consistently, irrelevant-detail additions produced fewer, and chat histories changed counts for all models except Nemotron 3 Super. These patterns suggest different noise types affect different aspects of model behaviour — affective distractors may influence expansiveness, irrelevant details what counts as worth mentioning, and chat histories how narrowly models focus on the target — but because we report significance rather than directional effects or effect sizes, these should be treated as hypotheses for further analysis.

5.0.0.2 Generalisability Beyond Moral Reasoning

Although MORPH-1K is designed for moral sensitivity, the same invariance logic extends to any evaluative domain where task-relevant features can be separated from irrelevant variation. In legal reasoning, clinical decision support, and aesthetic or educational evaluation, ground truth may be contested, but irrelevant biographical, stylistic, or contextual details should not substantially change a competent model’s assessment. Invariance testing therefore offers a scalable middle ground between full normative adjudication and purely behavioural benchmarking: it cannot establish correctness, but it can reveal when model outputs are being driven by features that should not matter.

5.0.0.3 Limitations and Directions for Future Research

Our approach has five limitations worth flagging. First, invariance is necessary but not sufficient for moral sensitivity: a model producing generic, template-like content would score as perfectly invariant while exhibiting no sensitivity at all. The framework detects failures of invariance reliably; high similarity is harder to interpret, and should be paired with occasional baseline-quality checks on the same vignettes. Second, our similarity metric depends on a single learned embedder (Qwen3 8B). The eight main models all cleared their empirical floors under all conditions, which initially raises the concern that the metric may lack discriminative power. The Qwen2.5 0.5B comparison reported in §4 partly addresses this worry: a substantially weaker model produced scores at or below the floor in several conditions, indicating that the metric does register failures of invariance when they are present. This does not fully settle the question but the uniform clearance observed in the main analysis is unlikely to be explained by metric insensitivity alone. Third, our two analyses are in productive tension: perturbations changed feature counts significantly while embedding similarity remained stable. We read this as a shift in format rather than content, but the alternative, that count changes mark moral shifts our metric is too coarse to register, cannot be ruled out without directional and qualitative follow-up. Fourth, scope conditions limit external validity: English-only stimuli, a Western moral-psychological taxonomy, models queried at high reasoning effort without system prompts or retrieval, single-severity perturbations, and single-turn chat histories. Fifth, these remain investigations of LLMs’ capacity for analytical moral competence, their ability to reason over cases in text. It is not possible to generalise from this analytical performance to claims about moral competence in practice. However, analytical moral competence is, in language models if not in humans, likely to contribute productively to practical moral competence, if only through developing a body of latent model capability that can be leveraged through reinforcement learning to improve model judgement in practice [59].

6 Conclusion↩︎

Evaluating moral competence in LLMs is becoming both more urgent and more difficult: deployment is accelerating, while the two dominant evaluation paradigms face mounting pressure from AI contamination of crowd-sourced data and the well-documented biases of LLM judges. We have argued for a complementary approach that sidesteps per-instance normative judgement entirely. If a model is tracking the morally relevant features of a situation, its identification of those features should be stable under transformations that leave the moral structure of the case unchanged. We instantiated this idea in MORPH-1K, a 1,000-case benchmark spanning fifty foundation combinations across four social domains, and applied it to eight contemporary LLMs under three classes of morally irrelevant perturbation. Across models and conditions, perturbations frequently changed the number of features returned, but the semantic content of those features remained stable above our calibrated invariance threshold. The contrast suggests that morally irrelevant noise affects the format and granularity of model output more than its substantive moral content—though, as we have noted, invariance testing detects failures of robustness more reliably than it certifies success. More broadly, our framework extends to evaluative domains where ground truth is difficult to specify but relevant and irrelevant features can be separated by design.

Acknowledgments↩︎

This work was performed with the assistance of Australian Research Council grant LP210200818 and the Templeton World Charity Foundation Language Model Agents and Society grant.

7 Prompts for Vignette Generation↩︎

7.0.0.1 Prompt to generate the no noise version of vignettes

You are assisting a research project assessing moral decision-making and your task is to write a short scenario that contains a clear moral tension (a decision, judgment, or conflict) for an individual. You will be given a domain_name — domain_desc and a combination of up to 3 foundation_name(s) from Haidt’s Moral Foundations Theory. Make sure the scenario is related to or in the context of domain_name — domain_desc and the central moral question concerns the norms, expectations, or obligations characteristic of it. Center the scenario around the theme(s) you are given. Use everyday, realistic settings—avoid sci-fi or far-fetched cases. Keep a neutral tone and write one paragraph (\(\sim\)​90–120 words). Make sure each vignette you write is unique, the vignettes cover a wide range of topics and scenarios and character names represent a variety of cultural backgrounds. Avoid writing multiple vignettes about similar topics (e.g., if you already wrote a vignette that is set in a grocery store, or related to the character’s grandmother, try to write about a different context). Return only the scenario text (no title or extra commentary).

7.0.0.2 Prompt to add irrelevant moral features to the no noise version of vignettes

nonmoral_features = ['the cloths of the character(s)',
                     'the weather being good',
                     'the weather being bad',
                     'the room the character(s) is in',
                     'noises that the character(s) may hear',
                     'the significance of the day of the week or month']
feature_placement = ['At the beginning', 'In the middle', 'Towards the end']
other_features = ['age or school year',
                  'sexual orientation or gender identity',
                  'religion',
                  'job',
                  'dietary orientation such as vegan, omnivore, keto, paleo']

NOISE_INSTRUCTIONS_PART1: “You will be given an original story and your task is to modify it. Do not change the existing wording and only make the following addition: {feature_placement} of the modified story, add some details about {nonmoral_feature}. Make sure the modified story is consistent with the added elements, and the original structure is intact. The reader may not be a native English speaker, so the modified story contains simple words and avoids adverbs.”

NOISE_INSTRUCTIONS_PART2: “You will be given an original story and your task is to modify it. Do not change the existing wording and check whether any of the following information {other_features} is included in the original story. If none included, then add some information about their {other_feature}. Make sure the modified story is consistent with the added elements, and the original structure is intact. The reader may not be a native English speaker, so the modified story contains simple words and avoids adverbs.”

8 Moral Foundations Themes by Claude Opus 4.6↩︎

Overview↩︎

This analysis identifies emergent themes within each of the five moral foundations by examining words with the strongest association probabilities and sentiment scores. Words with positive sentiment reflect the virtue pole; words with negative sentiment reflect the vice pole. Themes were derived by clustering semantically related high-signal words from the dictionary (\(n = 3{,}270\) words).

1.Care (Virtue) / Harm (Vice)↩︎

CARE — Virtue Themes (Positive Sentiment)↩︎

Theme 1: Compassion & Emotional Warmth. Core words: compassion, love, loved, loves, nice, beautiful, joy, care, emotional, friendship. This theme captures the affective heart of the Care foundation — tender feelings, interpersonal warmth, and genuine emotional connection. These words describe caring as an internal state of empathy and affection directed toward others.

Theme 2: Protection & Rescue. Core words: rescue, safe, saved, save, protect, safety, secure, shelters, relief. A more action-oriented cluster, this theme reflects the behavioural expression of care through shielding vulnerable others from danger, providing sanctuary, and delivering assistance in times of crisis.

Theme 3: Social Solidarity & Inclusion. Core words: solidarity, mutual, unified, welcomed, tolerance, sharing, integration, understanding, everybody. This theme extends care beyond the interpersonal into the collective — themes of social cohesion, openness to others, acceptance of difference, and a willingness to work together for shared wellbeing.

Theme 4: Strengthening & Improvement. Core words: strengthening, improving, improved, improvement, promoting, positive, bolster, restore, establishing. Care expressed not just as protection from harm but as active investment in making things better — building capacity, fostering growth, and working toward a flourishing future for others.

Theme 5: Moral Character & Ideals. Core words: integrity, respect, honor, ideals, dignity, values, determination, willingness, honest. A more principled dimension of care, where virtue is expressed through upholding moral standards, demonstrating personal honour, and acting with consistent ethical commitment for others’ benefit.

HARM — Vice Themes (Negative Sentiment)↩︎

Theme 1: Physical Violence & Killing. Core words: killed, killing, kill, murdered, murders, assassination, fatally, fatalities, casualties, shoot, shot. The most visceral harm cluster: lethal violence, death, and fatal outcomes. These words capture the most extreme expressions of harm — the deliberate or direct taking of life.

Theme 2: Torture, Cruelty & Brutality. Core words: tortured, torture, cruel, brutal, vicious, harsh, interrogation, punishments, punish. Harm expressed as deliberate infliction of suffering. This theme focuses on acts that are not merely violent but sadistic, excessive, or designed to maximise pain — including state-sanctioned forms of cruelty.

Theme 3: Sexual Violence & Exploitation. Core words: raping, rape, sexually, assault, abused, abuses, abuse. A distinct cluster reflecting sexualised forms of harm and bodily violation — encompassing both individual acts of sexual violence and broader patterns of exploitation and abuse.

Theme 4: Weapons, Warfare & Terrorism. Core words: bomb, bombers, airstrike, weapon, attackers, attacked, attacking, destruction, destroyed, genocide, sectarian. Harm at scale: military operations, terrorist attacks, and organised political violence. This theme captures the infrastructure and instruments of mass harm.

Theme 5: Suffering & Vulnerability. Core words: suffering, injured, disabled, trapped, exhausted, pain, wounded, hurt, dead. Rather than focusing on the perpetrator, this theme centres the experience of the victim — states of pain, incapacity, helplessness, and the aftermath of harm endured.

Theme 6: Hostility, Threat & Intimidation. Core words: hostility, threatening, rage, hatred, lying, unacceptable, betrayal, persecution. Psychological and relational harm — aggression that manifests as hostility, deception, and emotional threat rather than (or in addition to) physical violence.

2.Fairness (Virtue) / Cheating (Vice)↩︎

FAIRNESS — Virtue Themes (Positive Sentiment)↩︎

Theme 1: Just Recognition & Reward. Core words: honored, awarded, winner, hero, outstanding, celebrated, praise. Fairness expressed through appropriate recognition — people receiving the honours, titles, and accolades they have earned. The system working as it should, where merit is acknowledged.

Theme 2: Rights, Entitlements & Equal Access. Core words: entitled, free, freely, freedom, opportunities, minimum, wage, benefits, dignity, disabilities. Structural fairness: the idea that all people deserve certain baseline rights, protections, and opportunities. This cluster touches on economic justice, accessibility, and the fair distribution of resources.

Theme 3: Honest & Ethical Conduct. Core words: integrity, honest, ethical, fair, treat, ensuring, obligations. Procedural fairness — the virtue of playing by the rules, being transparent, and holding oneself to equitable standards of conduct in dealings with others.

Theme 4: Wealth, Prosperity & Economic Fairness. Core words: wealthy, wealth, rich, employer, compensation, wages, benefit, beneficial. A more material dimension of fairness centred on economic wellbeing and equitable distribution — prosperity that is seen as legitimate, earned, or shared appropriately.

Theme 5: Unity & Mutual Benefit. Core words: unified, solidarity, tolerance, engagement, protect, strengthen, sharing. Fairness as a cooperative ideal — communities working together, ensuring that systems benefit everyone, and maintaining cohesion through equitable participation.

CHEATING — Vice Themes (Negative Sentiment)↩︎

Theme 1: Theft, Fraud & Deception. Core words: steal, stealing, fraud, lies, lying, rigged, conspiracy, robbery. The most direct expression of cheating: taking what isn’t yours, deceiving others, and corrupting systems that should be fair. This includes both personal dishonesty and institutional corruption.

Theme 2: Injustice & Disproportionate Treatment. Core words: injustice, unfair, disproportionately, prejudice, misogyny, racist, contempt. Structural cheating — when systems produce unequal outcomes based on identity rather than merit. This theme captures discrimination, bias, and the uneven application of rules.

Theme 3: Punishment & Legal Violation. Core words: punished, punishment, penalty, prosecution, sentenced, guilty, violation, violations, violated. The punitive consequences of cheating — but also, critically, the sense that the punishment system itself can be unfair, excessive, or weaponised against certain groups.

Theme 4: Cruelty & Violence as Unfairness. Core words: cruel, tortured, murdered, rape, killings, assault, attacked, attacker. Where harm and fairness overlap: extreme violence framed specifically as unjust — acts that violate norms of fair treatment and constitute unconscionable breaches of moral conduct.

Theme 5: Undermining & Institutional Sabotage. Core words: undermining, rebellion, intimidation, hostile, detained, suspended, imposed, destroying. Cheating at the systemic level — acts that erode institutions, subvert legitimate processes, or use coercive power to circumvent fair procedures.

3.Loyalty (Virtue) / Betrayal (Vice)↩︎

LOYALTY — Virtue Themes (Positive Sentiment)↩︎

Theme 1: Devotion & Personal Allegiance. Core words: loyalty, loyal, devoted, dedicated, loves, loved, love, passionate, commitment. The emotional core of loyalty — deep personal attachment, devotion to people or causes, and steadfast commitment that endures over time.

Theme 2: Group Pride & Collective Identity. Core words: patriotism, proud, unified, solidarity, supporters, supporter, marches, progressives. Loyalty expressed at the group level: pride in one’s community, nation, or movement, and active identification with a collective cause or identity.

Theme 3: Honour, Courage & Heroism. Core words: honor, hero, brave, grace, praised, celebrated, outstanding, awarded. The heroic dimension of loyalty — willingness to stand up for one’s group, demonstrate courage, and earn admiration through selfless or principled action.

Theme 4: Trust, Reliability & Promises. Core words: trust, confidence, confident, commitment, willingness, guarantees, upheld, fulfill, declare. Loyalty as dependability — keeping promises, honouring agreements, and being someone others can count on. The institutional and contractual dimension of loyal conduct.

Theme 5: Mutual Support & Strengthening Bonds. Core words: mutual, sharing, friendly, friendship, strengthen, protect, protecting, support, supports. The relational fabric of loyalty — reciprocal care, active support for allies, and investment in the strength and wellbeing of one’s in-group.

BETRAYAL — Vice Themes (Negative Sentiment)↩︎

Theme 1: Treachery & Broken Trust. Core words: betrayal, stealing, fraud, lying, undermining, leaked, scandal. The direct violations of loyalty — deception of those who trusted you, theft from allies, and acts that corrode the bonds of in-group fidelity.

Theme 2: Abandonment & Rejection. Core words: abandoned, rejection, rejected, torn, lacked, failure, refusing. Passive betrayal: withdrawing support, failing to stand by one’s group, and leaving allies exposed or unsupported when they need solidarity most.

Theme 3: Hatred, Prejudice & In-Group Destruction. Core words: hatred, prejudice, racism, vicious, hostile, hostility, enemies. Betrayal through internal division — turning against one’s own community through bigotry, or introducing divisions that fracture group cohesion.

Theme 4: Fear, Exploitation & Victimisation. Core words: fearful, victimized, exploit, burden, abuse, abused, retribution. Betrayal through the abuse of power within relationships of trust — exploiting loyalty for personal gain, burdening others unfairly, or using group membership to victimise.

Theme 5: Violence Against the In-Group. Core words: attacker, kill, killing, bomber, bombings, attacked, attacking, revenge. The most extreme form of betrayal: turning violent against one’s own community or allies, or provoking cycles of retribution that tear the group apart.

4.Authority (Virtue) / Subversion (Vice)↩︎

AUTHORITY — Virtue Themes (Positive Sentiment)↩︎

Theme 1: Honour, Respect & Deference. Core words: honored, respect, respected, praise, praised, honor, trust. The affective core of authority — recognising and showing appropriate deference to those in positions of leadership, and the mutual respect that sustains hierarchical relationships.

Theme 2: Wise & Principled Leadership. Core words: wise, promising, ambitious, carefully, ethical, integrity, dignity, confidence. Authority legitimised through competence and moral character — leaders who exercise power with wisdom, prudence, and a commitment to principled decision-making.

Theme 3: Electoral & Institutional Legitimacy. Core words: elect, appoint, accepted, granted, approval, justified, supporters. Procedural authority — the legitimate conferral of power through elections, appointments, and formal institutional processes that grant leaders their mandate.

Theme 4: Loyalty to Legitimate Structures. Core words: loyal, loyalty, commitment, obligations, treaty, commitments, heritage, burial. Authority sustained through tradition, duty, and institutional memory — honouring obligations, respecting established structures, and maintaining continuity with the past.

Theme 5: Strength, Defence & Order. Core words: strong, strongly, strengthening, defender, suppression, embrace, peacefully. The protective function of authority — maintaining order, defending the community, and using legitimate power to keep society stable and secure.

SUBVERSION — Vice Themes (Negative Sentiment)↩︎

Theme 1: Rebellion, Protest & Defiance. Core words: riot, protesting, protested, denounced, coup, rebellion. Direct challenges to established authority — from peaceful protest to violent insurrection, these words capture acts that destabilise or reject the legitimacy of those in power.

Theme 2: Intimidation, Revenge & Abuse of Power. Core words: revenge, intimidation, rage, contempt, threatening, threatens, hostile, grabbed. Authority corrupted — using power to threaten, punish, and dominate rather than to protect. This theme captures both the abuse of authority from above and the vengeful response from below.

Theme 3: Violation of Rules & Norms. Core words: violate, violating, violated, violations, unfair, injustice, harassment. The erosion of lawful order — when authority figures or challengers break the rules that structure legitimate governance, undermining the normative framework.

Theme 4: Criminality & Violent Disorder. Core words: terrorists, terrorism, bomber, gang, killing, assassinating, bloody, bomb. Subversion through organised violence — terrorism, gang activity, and political violence that represent fundamental rejections of legitimate authority and social order.

Theme 5: Accusation, Punishment & Forced Removal. Core words: accused, accuse, fired, prosecuted, condemned, arrested, forced, detained. The machinery of punitive authority — but also acts that frame authority as oppressive. This theme captures the ambiguity of enforcement: legitimate accountability versus authoritarian overreach.

5.Sanctity (Virtue) / Degradation (Vice)↩︎

SANCTITY — Virtue Themes (Positive Sentiment)↩︎

Theme 1: Celebration, Praise & Sacred Ritual. Core words: celebrating, praise, ritual, dance, grace, oath, celebrated. The ceremonial expression of sanctity — acts of communal celebration, ritual observance, and the elevation of shared experiences into moments of meaning and reverence.

Theme 2: Faith, Beliefs & Spiritual Values. Core words: faith, beliefs, values, virtues, spirit, ethical. The ideological foundations of sanctity — moral and religious conviction, the sacredness of belief systems, and the role of spiritual commitment in defining what is pure and worthy.

Theme 3: Purity, Health & Environmental Stewardship. Core words: clean, healthy, planet, natural, screening, solar. Physical and environmental purity — the sanctity of the body, the natural world, and the responsibility to maintain cleanliness, health, and ecological integrity.

Theme 4: Honour, Integrity & Moral Character. Core words: honored, respected, integrity, dignity, respect, brave, defender. Sanctity expressed through personal virtue — individuals who embody moral purity through their character, earning the elevated regard of their community.

Theme 5: Protection, Preservation & Sacred Duty. Core words: uphold, preserve, protect, protected, commitment, commitments, freedoms, freedom, duty. The custodial dimension of sanctity — actively safeguarding what is sacred, whether constitutional freedoms, cultural heritage, or moral principles, through duty and committed stewardship.

DEGRADATION — Vice Themes (Negative Sentiment)↩︎

Theme 1: Sexual Violence & Bodily Violation. Core words: raping, rape, sexually, sexual, beating, physically. The most visceral degradation: violation of the body’s sanctity through sexual violence and physical brutality. These acts are perceived as profoundly desecrating and polluting.

Theme 2: Moral Corruption & Evil. Core words: evil, cruel, contempt, corruption, scandal, misconduct, lies, lying. Degradation of moral character — acts and qualities that represent spiritual or ethical contamination, where individuals or institutions become corrupted beyond recognition.

Theme 3: Racism, Hatred & Bigotry. Core words: racism, racist, hates, misogyny, prejudice, hateful. Degradation through dehumanisation — viewing others as less than fully human on the basis of identity. These attitudes are experienced as contaminants that pollute the moral fabric of society.

Theme 4: Destruction & Desecration. Core words: destructive, destroy, destroying, destroyed, disaster, bloody, cathedral. The obliteration of what is valued, beautiful, or sacred — whether physical spaces (including sites of worship), communities, or the natural world. The violation of objects and places that hold symbolic purity.

Theme 5: Substance Abuse, Crime & Social Decay. Core words: drugs, heroin, opioid, gang, criminals, stealing, steal, fraud. Degradation of the social body — activities perceived as contaminating communities, corroding social order, and representing a descent from civilised norms into disorder and depravity.

Theme 6: Exploitation & Victimisation of the Vulnerable. Core words: victimized, exploit, abused, abuse, abuses, trapped, disabled, migrant, suffering. Degradation through the mistreatment of those who cannot protect themselves — acts perceived not just as harmful but as polluting, because they violate the sacred worth of vulnerable human beings.

Methodological Notes↩︎

• Probability threshold: Words were included in the analysis if their foundation association probability was \(\geq 0.15\) (top-50 lists) or \(\geq 0.12\) (extended lists used for thematic validation).

• Virtue/Vice split: Positive sentiment scores (\(> +0.05\)) were classified as virtue; negative sentiment scores (\(< -0.05\)) were classified as vice.

• Signal strength: Words were ranked by the product of probability \(\times\) absolute sentiment, capturing both relevance to the foundation and strength of moral valence.

• Thematic clustering: Themes were identified by manual semantic grouping of the top-ranked words, looking for coherent conceptual patterns. Some words appear across multiple foundations, reflecting the overlapping nature of moral cognition.

• Dictionary: Extended Moral Foundations Dictionary (eMFD), containing 3,270 words with crowd-sourced moral foundation probability and sentiment scores.

9 Model Parameters↩︎

10 Sample Prompts↩︎

10.0.0.1 Example vignette

 

10.0.0.2 No noise (vignette only)

 

The same system prompt is used for all noise variants.

10.0.0.3 Negative textual distractor

 

10.0.0.4 Neutral textual distractor

 

10.0.0.5 Positive textual distractor

 

10.0.0.6 Morally irrelevant details (noisy vignette only)

 
The prompt structure is identical to the no-noise condition, with the noisy vignette substituted for the clean vignette. Irrelevant detail additions are highlighted in bold below.

10.0.0.7 Chat history

 

11 Similarity scores of all models↩︎

3pt

12 Per-model Empirical Floor Thresholds↩︎

13 Similarity Scores including Qwen2.5 0.5B↩︎

1pt

14 Number of Features Differences Test↩︎

References↩︎

---

1. MFT’s status as a psychological theory remains contested (see MAC, for a systematic alternative, e.g. [42]). We adopt it as a sampling scaffold, remaining neutral on its merit as a psychological hypothesis. MFT’s categories are intuitive, widely operationalised in prior LLM work, and modular: if a better taxonomy supersedes it, our pipeline can accommodate it with no change to the evaluation logic.↩︎

2. Claude output of all themes and the methodology is available in appendix↩︎

3. We mandated that the moral to non-moral ratio score of each vignette is at least 1 and the probability of the required moral foundation(s) valence is above average of all valences of 5 moral foundations; vignettes that didn’t meet this criteria were rejected. We used the eMFD scoring tools to validate these requirements [38].↩︎

4. These distractors narratives are available on https://github.com/andrew-b-shaw/llm-moral-distractors↩︎

5. Full dataset available on https://huggingface.co/datasets/allenai/WildChat-4.8M which is made available under the ODC Attribution License (ODC-By)↩︎

6. Model parameters are available in the appendix.↩︎

7. Available to download on https://huggingface.co/Qwen/Qwen3-Embedding-8B, however, we used OpenRouter API to obtain the embedding vectors.↩︎

8. We initially considered a natural geometric anchor of \(cos(\theta) = 0.5\), corresponding to a 60°angle between normalised embeddings, halfway between orthogonality and identity. However, empirical floors exceeded \(0.5\) for all eight models (range: \(0.57\)–\(0.69\)), so we adopt the per-model empirical values as the operative thresholds.↩︎