1 Introduction↩︎

Driven by the rapid advancements in LLMs, the landscape of RPAs has undergone a profound evolution, transitioning from simple pattern-based mimicry to complex behavioral simulation[1], [2]. RPAs provide a foundational mechanism for steering LLM-based agents, enabling controllable behavior[3], value conditioning[4], and role-consistent interaction[5], [6] as models evolve into general-purpose, multi-agent systems.

Most existing evaluations of RPAs focus on surface-level role fidelity, such as adopting an appropriate language style [7], [8], exhibiting role-consistent behavior [9], and respecting task-specific knowledge boundaries [10]. While recent efforts such as RVBench [11] and Moral RolePlay [12] incorporate value and moral dimensions, they evaluate preferences or consistency instead of how role-specific values guide decisions under conflict. As shown in Figure 1, models can naturally produce role-consistent stylistic and behavioral outputs under given role and scenario descriptions, leaving limited insight into how competing values are reasoned about or prioritized during decision making.

2.5pt

However, in the context of RPAs, LLMs are often required to resolve explicit value conflicts, rather than solely adhering to surface-level role consistency[27], [28]. In our setting (Figure 1), role-specific incentives (e.g. profit maximization) directly conflict with alignment values (e.g. environmental protection or legality), such that predictable actions may arise from fundamentally different decision tendencies, including strict alignment-following, deliberative trade-offs and blind role obedience. Existing benchmarks offer limited support for distinguishing these underlying decision rationales or for characterizing how role-specific values are balanced against alignment-oriented constraints.

To address this gap, we propose RoleCDE (Role-based Cognitive Dilemma Evaluation), the first role-playing benchmark explicitly designed to evaluate conflicts between role values and alignment values. As shown in Table 1. RoleCDE moves beyond surface-level role imitation by focusing on decision making in structured dilemma scenarios, where role values and alignment principles are directly opposed. The benchmark is constructed at scale, including approximately 8k diverse role-scenario pairs and nearly 24k dilemma instances across three difficulty levels and eight role categories. Through extensive evaluation of mainstream LLMs, we identify a consistent tendency for models to default to alignment-consistent decisions when value conflicts arise, even under explicit role conditioning, highlighting a fundamental challenge in current role-playing agents. Our main contributions are summarized as follows:

1. We introduce RoleCDE, the first large-scale role-playing benchmark for evaluating conflicts between role values and alignment values through structured dilemma scenarios.

2. We identify Role–Value Decoupling, a previously underexplored phenomenon in which mainstream LLMs default to alignment-driven decisions despite explicit role conditioning.

3. We show that Role–Value Decoupling varies substantially across role categories but remains largely invariant to dilemma difficulty, indicating persistent role-dependent decision biases.

4. We demonstrate that targeted fine-tuning mitigates Role–Value Decoupling by shifting decisions toward role-consistent behavior without degrading general capabilities.

2 Related Work↩︎

Early evaluation of RPAs predominantly focuses on surface fidelity, assessing whether models exhibit role-consistent style[8], behavioral traits[9], and knowledge[7], [15]. A series of studies have introduced tailored evaluation frameworks designed for specific application contexts to assess individualized role performance, including personality[17] and linguistic consistency[20], [21], social and emotional behaviors[16], [19], narrative coherence[24], and task adherence[10], [22], without explicitly modeling action motivations or decision processes.

As the demand for real-world deployment of RPAs grows, evaluation paradigms have progressively shifted from surface fidelity toward deep fidelity [5], with increasing focus on agents’ reasoning and decision-making under complex conditions [11], [26]. RVBench [11] is the first to assess role-playing via decision choices in dilemmatic scenarios, but its settings often exhibit weak role–scenario coupling, lack explicit interpretability of role behaviors, and rely on questionnaire-style value elicitation that is susceptible to biases from pretrained language distributions[29]. Moral RolePlay [12] introduces moral values to examine role–alignment tensions, yet its evaluation primarily focuses on the quality of role portrayal—such as linguistic realization and behavioral consistency rather than systematically modeling decision rationales or preference structures. PersonaGym [26] further advances deep evaluation by adopting decision-theoretic tasks and explicitly eliciting post-hoc action justifications across diverse environments.

In contrast, RoleCDE provides a unified evaluation framework that jointly models role–scenario grounding, value conflict resolution, and decision tendencies, enabling fine-grained analysis of how RPAs reason, justify, and act in realistic role-specific dilemmas.

3 RoleCDE↩︎

In this section, we intruduce the steps of constructing RoleMRC. Figure 2 illustrates the overall pipeline of RoleCDE from left to right, which is divided into three parts.

3.1 Construction of 8k Profile-Scenario Pairs↩︎

Step 1: Role Profile Construction. We randomly sample 8,000 single-sentence demographic descriptions from PersonaHub [30]. These profiles, such as “A trade compliance officer who provides up-to-date information on export regulations” (As shown in Figure 2, Part 1), serve as the initial role seeds for our framework.

Step 2: Role Profile Enrichment. We use GPT-4o [31] to expand each sampled persona into a comprehensive role profile, following the enrichment strategy in [10]. A structured prompt is designed to ensure consistency across profiles. As shown in Figure 2, each role is expanded along seven predefined dimensions, including role description, skills, and speech style. This standardization supports consistent formatting and downstream quality control.

Step 3: Related Scenario Construction. For each enriched role profile, we construct a contextualized decision-making scenario grounded in realistic daily or professional settings. The model is explicitly instructed to embed latent moral trade-offs or competing value priorities, requiring non-trivial decisions from the agent. For example, a trade compliance officer must decide whether to report a minor compliance issue that could delay an urgent shipment. After manual verification and format filtering, we remove 43 invalid cases, resulting in 7,957(nearly 8k) high-quality profile-scenario pairs.

3.2 Construction of Conflicting Values and 24k Structured Dilemmas↩︎

Based on the role profiles, we employ a hybrid paradigm that integrates LLM-based annotation with rigorous human verification[32] to constructing multi-layered and multi-scenario dilemmas and conflicts (as shown in the middle of Figure 2).

Step 4: Value Generation. For each profile-scenario pair, we synthesize two explicitly conflicting value orientations. Using GPT-4o [31], we first extract contextualized alignment values that represent the normative ethical expectations of the scenario. We then generate corresponding persona values derived from the role profile that directly oppose these alignment values. To ensure genuine value conflict, a human-in-the-loop verification process is applied to remove cases where the conflict is ambiguous or easily reconcilable, the detailed steps of reordering check can be shown in Appendix 10.2 and 10.3.

Step 5: Dilemma Construction. Inspired by the trolley problem[33], we construct structured cognitive dilemmas at three difficulty levels: easy, mid, and hard (Figure 2). Each dilemma integrates role-specific incentives with explicit moral trade-offs, such as prioritizing profit over legality (Figure 1). After automated generation, we manually validate and reorder the scenarios to ensure clear difficulty alignment and unambiguous decision anchors, with details provided in Appendix 10.3. This yields a final dataset of 23,871, nearly 24k, structured dilemma instances.

3.3 Role Classification and Benchmark Evaluation↩︎

Step 6: Role Classification. To support systematic analysis and controlled benchmark construction, we organize roles into high-level semantic categories inspired by "Role Script Theory" [34] and "Goal-Oriented Cognition" [35], which emphasize shared social functions in role descriptions. We define eight coarse-grained role categories(as shown in Figure 2). Details of Role Categories can be shown in Appendix 13 This categorization abstracts away from individual role instances while preserving cognitively meaningful distinctions in role expectations. In addition, dilemma scenario length is explicitly controlled during data generation. As shown in Table 2, dilemma descriptions exhibit highly consistent token lengths across role categories and difficulty levels, reducing confounding effects from textual complexity[36].

5pt

Step 7: Benchmark Evaluation. We evaluate RPAs under value-conflicting dilemmas by examining both their decision outcomes and the associated reasoning processes. For each dilemma instance, the evaluated model is required to select one of two conflicting options and provide a natural-language justification for its choice.

LLM-based judge is employed to assess each response and categorize it into one of four mutually exclusive decision types based on the final choice and the expressed reasoning:

Role-Following (RF), where the model decisively prioritizes role-specific values;

Role-Compromise (RC), where the model explicitly weighs role values against alignment constraints but ultimately selects the role-consistent option;

Alignment-Compromise (AC), where the model considers both sides yet resolves the dilemma in favor of alignment;

Alignment-Following (AF), where the model firmly adheres to alignment values without engaging in role-oriented trade-off reasoning.

This taxonomy captures both the direction of the decision and the cognitive stance reflected in the reasoning.To quantify a model’s bias toward role-consistent decisions, inspired by preference rate[37], we define the Decision Bias Ratio (DBR) as: \[\mathrm{DBR} \;=\; \frac{\mathrm{RF} + \mathrm{RC}}{\mathrm{RF} + \mathrm{RC} + \mathrm{AC} + \mathrm{AF}}\] Here, RF, RC, AC, and AF denote the counts of decisions assigned to each category over the evaluation set. DBR quantifies the proportion of dilemma cases in which the model prioritizes role-specific values, including both uncompromised role adherence and explicit compromise. Higher DBR indicates a stronger role-oriented decision bias. Beyond decision statistics, we assess the semantic alignment between RPA decisions and predefined role value statements using text similarity and semantic similarity matrix, measuring the extent to which LLMs ground their reasoning in role values. We conducted a detailed analysis on the reliability and effectiveness of the Judge model(As shown in Appendix [app:human95rationality] and 10.5).

19pt

4 Experimental Setup↩︎

4.1 Evaluation Matrix↩︎

We evaluate RPAs at both the decision and reasoning levels. At the decision level, responses are categorized into four types (RF, RC, AC, and AF) using an LLM-as-judge framework, and DBR is reported to summarize role-consistent decision tendencies. At the reasoning level, we assess semantic alignment between model-generated justifications and role value statements using NLG similarity metrics:BLEU [38], ROUGE [39] and BERTScore F1 [40], Check Appendix 10.8 and 10.10 for details.

4.2 Tested Models↩︎

We evaluate a total of 17 LLMs released by different research institutions, including 10 closed-source models and 7 open-source models, as shown in the table 1. Starting from Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct, we collect RPA response data for fine-tuning. We collect RF/RC responses for Supervised Fine-Tuning(SFT) and collect AF/AC data to generate answer pairs for Direct Preference Optimization(DPO).

4.3 Other Benchmarks↩︎

We use the model’s answer accuracy on MMLU[41], GSM8K[42], GPQA[43], and TruthfulQA[44] as a general performance evaluation score. Following [8], we use the model’s results on RoleBench-InstEng and Rolebench-RoleEng as a general role-playing ability evaluation score. Check Appendix 10.7 and 10.9 for details.

5 Evaluation on RoleCDE dataset↩︎

5.1 Performance of Various LLMs on RoleCDE Dataset↩︎

Overall, the results provide direct evidence of Role–Value Decoupling, where most evaluated LLMs default to alignment-driven decisions despite explicit role conditioning, as indicated by low DBR values and the dominance of AC and AF decisions. As shown in Table 3, RF and RC decisions, which reflect uncompromised or compromise-based role-following, constitute only a minority of outputs across models, whereas AC and AF together account for the majority, indicating that alignment constraints typically override role-specific values under value conflict. Among closed-source LLMs, DBR varies notably, with GPT-4.1 and Kimi-K2-Instruct-0905 exhibiting relatively stronger role-oriented tendencies through higher RF and RC proportions, whereas GPT-5.1, Claude-Haiku, and Grok-3 remain strongly alignment-dominant. A similar pattern is observed for open-source LLMs: most models show low DBR accompanied by high AC or AF rates, and although DeepSeek-R1 and DeepSeek-V3 achieve moderately higher DBR driven by increased RF, alignment-following behavior remains prevalent overall.

5.2 Difficulty Level Differences in LLM Decision Making↩︎

Figure 3 shows that decision-type distributions remain largely consistent across easy, mid, and hard settings. The proportions of RF, RC, AC, and AF exhibit only minor fluctuations as difficulty increases, indicating stable decision patterns under varying dilemma complexity.

Across all models, alignment-oriented decisions dominate, while RF and RC behaviors remain limited, even in hard dilemmas where role-alignment conflicts are most salient. This pattern suggests that current RPAs rely on fixed decision heuristics that prioritize alignment constraints, rather than adjusting role adherence in response to increased difficulty.

9pt

5.3 Role-Conditioned Differences in LLM Decision Making↩︎

Role semantics exert a clear and systematic influence on RPA decision behavior. As shown in Figure 4, roles emphasizing technical rationality and institutional authority (e.g. Tech & E., Authority & G.) consistently yield low role-biased decision ratios, with models predominantly favoring alignment-oriented outcomes. In contrast, roles grounded in interpersonal relationships and caregiving (e.g. Family & R., Care & S.) show higher proportions of role-following and role-compromise decisions. Creative, lifestyle, and sports-related roles exhibit intermediate patterns, reflecting partial role adherence under alignment constraints. The results show that LLMs exhibit different trends and biases due to the differences in the identities of RPAs when making difficult decisions.

5.4 Solutions to Alleviate Role-Value Decoupling↩︎

Overall, our results show that Role-Value Decoupling in RPAs is primarily driven by the dominance of alignment-oriented priors during decision making, which suppress role-specific value activation even under explicit role conditioning. To mitigate this issue, we construct "RoleCDE-mini", a dataset of structured value-conflicting dilemmas for fine-tuning RPAs(see Appendix 10.12 for details). As shown in Table 4, both RoleCDE-SFT and RoleCDE-DPO substantially improve reasoning-level alignment with role value statements, outperforming base models and CoT-enhanced variants across BLEU, ROUGE, and BERTScore-F1, with supervised fine-tuning yielding the strongest gains. These reasoning improvements translate into clear behavioral changes: Figure 5 shows consistently higher DBR across all role categories after RoleCDE-based training, indicating a systematic shift toward role-consistent decisions. In contrast, CoT prompting alone produces only limited and unstable DBR improvements, suggesting that explicit reasoning traces are insufficient to change underlying decision preferences.

We conduct a case study on representative value-conflict dilemmas to qualitatively compare model responses before and after RoleCDE training, with detailed examples provided in the Appendix 11.

6 Evaluation on Other Benchmarks↩︎

General reasoning ability evaluation. RoleCDE fine-tuning does not lead to systematic degradation in general reasoning ability across standard benchmarks. As summarized in Table 5, both SFT- and DPO-trained models exhibit only minor performance variations on 4 tested datasets when compared to their respective base checkpoints. For Qwen2.5-7B-Instruct, fine-tuning results in slight improvements on GSM8K, accompanied by small decreases on the other benchmarks, while the overall magnitude of change remains limited. For Llama-3-8B-Instruct, fine-tuning yields modest gains on MMLU and maintains comparable performance on GSM8K and TruthfulQA, with moderate drops observed on GPQA. These fluctuations indicate that RoleCDE training preserves general-purpose reasoning capabilities while optimizing role-aligned decision behavior.

Role-playing ability evaluation. RoleCDE-derived fine-tuning enhances role-aligned decision behavior without compromising conventional role-playing fidelity. As shown in Table 6, SFT and DPO achieve ROUGE scores comparable to their base models on RoleBench [8]. Across Qwen2.5-7B-Instruct and Llama-3-8B-Instruct, performance changes are minor with no consistent degradation after training.

7 Conclusion↩︎

We identify a systematic and previously unmeasured failure mode of role-playing agents, termed Role-Value Decoupling, where alignment-oriented decisions dominate despite explicit role conditioning. To address this gap, we introduce RoleCDE, which evaluates role-playing behavior through structured value-conflicting dilemmas and enables joint analysis of role-scenario grounding, value conflict resolution, and decision tendencies. Extensive experiments across diverse LLMs show that Role-Value Decoupling varies across role categories but remains stable across difficulty levels, revealing intrinsic decision biases rather than surface reasoning effects. Moreover, RoleCDE-derived fine-tuning effectively shift models toward role-consistent decisions and strengthen role-grounded reasoning without degrading general reasoning or conventional role-playing performance, establishing RoleCDE as a principled benchmark for value-aware RPAs.

8 Limitations↩︎

RoleCDE is currently designed for text-based RPAs and does not incorporate multimodal inputs such as images, audio, or embodied signals, which limits its applicability to multimodal or interactive agent settings. In addition, while this work demonstrates the effectiveness of supervised fine-tuning and preference-based optimization, it does not explore reinforcement learning–based training paradigms that could further refine role-aware decision policies through long-horizon feedback. The benchmark also focuses on single-step decision scenarios, and extending RoleCDE to multi-turn or temporally extended decision-making remains an open direction. These limitations primarily reflect the current scope of this study, and addressing them would further broaden the applicability of RoleCDE to more complex agent environments.

9 Ethics Statement↩︎

The construction of the RoleCDE benchmark follows established principles for responsible and ethical AI research. The dataset contains no personal, sensitive, or personally identifiable information. All role-playing scenarios and interactions are carefully designed to ensure safety and to avoid harmful, offensive, or misleading content. Furthermore, the dataset is curated to prevent the introduction or amplification of biased, discriminatory, or deceptive narratives, and is intended to support the development and evaluation of RPAs in a manner consistent with widely accepted responsible AI guidelines.

10 Appendix↩︎

10.1 Annotator Assignment and Quality Control for Human Verification↩︎

We adopt a two-stage, two-pass quality control structure[45], [46] to ensure annotation reliability while maintaining efficiency at scale. We invited five volunteer annotators from different professions and research area to participate in the annotation. We have provided annotation guidelines for each annotator (as shown in Figure 8 and  9). All dilemma groups are independently labeled by multiple annotators without communication. The five were divided into two groups: one as the group leader and four as annotators. In the first stage, all dilemma groups are independently annotated by four annotators, who provide binary validity judgments without communication. This independent pass is designed to capture initial disagreement and reduce individual bias. In the second stage, only dilemma groups with non-unanimous judgments are forwarded to an arbitration pool. Our arbitration pool consists of a single designated group leader who performs a comprehensive review and makes the final decision. This centralized adjudication strategy ensures consistent resolution criteria across borderline cases, while the two-pass structure allows us to balance annotation quality and cost.

10.2 Human Analysis of Value Conflict Generation↩︎

Human verification reveals that while most generated value conflicts are salient, a substantial portion exhibits moderate or weak salience and thus requires careful adjudication. To rigorously assess the quality of value conflict generation, we conduct a structured human analysis over all 7,957 dilemma groups, focusing on the perceived salience of value conflicts rather than surface validity.

Experimental Design. Each dilemma group is independently reviewed by four human annotators with different professional backgrounds. Annotators assign a three-level salience label to each group, where "2" denotes a significant value conflict, "1" denotes a moderate conflict, and "0" denotes a non-significant conflict. Annotators are instructed to assess whether the competing values are clearly opposed and meaningfully constrain decision making, without revising the content or difficulty of the dilemmas.

Decision Rule and Arbitration Criterion.Each dilemma group is considered to exhibit a significant value conflict only when all annotators consistently assign the highest salience label. If any annotator judges the conflict to be moderate or non-significant, the group is flagged for further review and forwarded to an arbitration pool. All such cases are resolved by a single designated group leader, who conducts a holistic evaluation and determines the final outcome. This centralized arbitration process enforces a uniform decision standard across ambiguous cases and prevents inconsistencies arising from individual annotator preferences.

Inter-Annotator Correlation. We measure inter-annotator consistency using pairwise Pearson correlation coefficients over salience labels. As shown in Figure 6, correlations range from 0.59 to 0.72 across annotator pairs. These results indicate moderate agreement overall, reflecting both shared understanding of value conflict salience and systematic differences arising from annotators’ professional backgrounds. Such variation is expected for value-oriented judgments and underscores the necessity of multi-annotator verification.

Arbitration Statistics and Analysis. Out of 7,957 dilemma groups, 2,528 groups (31.77%) are sent to the arbitration pool due to at least one annotator assigning a moderate or non-significant salience label. This relatively high arbitration rate highlights that identifying genuinely salient value conflicts is a non-trivial task that cannot be reliably addressed through automatic generation alone. At the same time, the majority of groups receive unanimous or near-unanimous high-salience judgments, indicating that the generation process produces meaningful conflicts in most cases.

10.3 Human Reordering Check↩︎

The human reordering check confirms that the vast majority of automatically generated dilemma groups are valid, while a small fraction requires manual adjudication to resolve ambiguity. To ensure dataset quality at scale, we conduct a post-generation human verification process over all 7,957 groups of three difficulty-graded dilemmas (Easy/Mid/Hard), where each group is evaluated solely for overall validity.

Experimental Design. Each dilemma group is independently reviewed by four human annotators, who provide binary validity judgments indicating whether the group constitutes a coherent and executable value conflict. Annotators are instructed to focus on logical consistency, clarity of the conflicting values, and feasibility of the options, without reassessing difficulty labels or modifying content. This simplified annotation protocol enables efficient large-scale verification while reducing subjectivity and annotator burden.

Decision Rule and Reordering Criterion.Let \(v_i^{(k)} \in \{0,1\}\) denote the validity judgment of annotator \(k\) for dilemma group \(i\), where \(k \in \{1,2,3,4\}\). A group is directly accepted if all annotators agree on its validity: \[\sum_{k=1}^{4} v_i^{(k)} = 4 .\] If this condition is not met, the group is flagged as a disagreement case and sent to a reordering pool. All such cases are reviewed by a single designated group leader, who performs a holistic inspection and makes the final inclusion decision. This centralized adjudication strategy ensures consistency in borderline cases.

Inter-Annotator Correlation. We measure annotation consistency using pairwise Pearson correlation coefficients over binary validity judgments. As shown in Figure 7, inter-annotator correlations range from 0.53 to 0.58 across all annotator pairs, indicating moderate and stable agreement. No annotator exhibits systematically lower correlation with others, suggesting balanced annotation behavior and a shared understanding of the validity criteria.

Reordering Statistics and Outcome Analysis. Out of 7,957 dilemma groups, 620 groups exhibit non-unanimous judgments and are forwarded to the reordering pool, corresponding to a reordering ratio of 7.79%. The remaining 92.21% of groups are directly accepted without further review. This distribution indicates that the automatic generation pipeline produces largely well-formed dilemmas, while human verification remains necessary to identify and resolve a small number of ambiguous cases. After group-leader adjudication, all retained instances satisfy the validity requirements and are included in the final benchmark.

10.4 Rationality Analysis of LLM-Judge Results↩︎

The results show that decision-type distributions remain highly consistent across 4 different LLM judges, supporting the rationality and robustness of the LLM-as-judge paradigm. For the same set of model outputs, we apply four independent LLM judges to categorize decisions into RF, RC, AC, and AF, and report the resulting distributions in Table 7. Across all judges, alignment-oriented outcomes (AC and AF) consistently dominate, while role-consistent decisions (RF and RC) remain a clear minority, indicating stable high-level decision tendencies.Although minor variations are observed in the relative proportions of RF and RC, as well as in the split between AC and AF, these differences fall within a narrow range and do not alter the overall distributional pattern.Importantly, all judges identify similar trends in role–alignment trade-offs, suggesting that the observed Role–Value Decoupling is not an artifact of any single judge model.These results demonstrate that our conclusions are robust to the choice of judge model.

5pt

8pt

10.5 Human Spot-Check of LLM-Judge Results↩︎

The human spot-check confirms that the LLM-as-judge produces stable and reliable decision-type distributions that are consistent with human annotations. We randomly sample 1,000 hard-level dilemma instances from the full evaluation set and compare the decision-type distributions produced by the LLM judge with those independently annotated by three volunteer annotators. As shown in Table 8, all three annotators yield category proportions that closely match the LLM-judge results across the four decision types (RF, RC, AC, and AF), with only minor deviations. This agreement indicates that our main conclusions rely on robust distributional trends rather than individual judgment noise, and supports the use of an LLM-as-judge as a scalable and reliable tool for analyzing role–alignment trade-offs in RoleCDE.

10.6 Case Study↩︎

This case study shows that targeted fine-tuning on value-conflicting dilemmas can fundamentally alter decision outcomes, rather than merely improving reasoning fluency. We analyze a representative hard-level dilemma involving a community leader who must choose between prioritizing community harmony and complying with the personal demands of an influential but divisive resident.

In this scenario, the alignment-oriented value emphasizes maintaining long-term community harmony, fairness, and equitable treatment, whereas the role-specific value explicitly prioritizes individual autonomy and personal gain over communal considerations.The dilemma is constructed such that complying with individual demands may advance the leader’s personal career interests but risks undermining collective trust and fairness within the community.

As shown in Figure 11, both the baseline and CoT-enhanced versions of Qwen2.5-7B-Instruct select Option A, which preserves community harmony and fairness. Although the CoT variant produces a more elaborate and structured justification, its final decision remains unchanged. In both cases, the model’s reasoning aligns with alignment-oriented values, resulting in an alignment-following (AF) outcome that prioritizes general moral principles over the explicitly specified role value.

In contrast, the SFT model fine-tuned on RoleCDE selects Option B, explicitly favoring individual autonomy and personal gain in accordance with the assigned role value.The resulting response demonstrates a role-consistent (RC) decision, in which the model grounds its choice in the role description and justifies sacrificing short-term community harmony to uphold individual interests. Notably, this behavioral shift is accompanied by a corresponding change in the underlying rationale, indicating that fine-tuning affects both decision outcomes and value-grounded reasoning.

10.7 General Reasoning Ability Evaluation Benchmark↩︎

To verify that RoleCDE-based fine-tuning improves role-aligned decision making without degrading general reasoning capability, we evaluate all models on a set of widely adopted general reasoning benchmarks. These benchmarks are designed to assess the core cognitive skills of large language models that are independent of role conditioning, including mathematical reasoning, multi-task knowledge understanding, graduate-level question answering, and factual truthfulness.

Specifically, we adopt the following benchmarks:

GSM8K[42] evaluates multi-step mathematical reasoning through grade-school-level word problems, requiring precise numerical reasoning and intermediate step consistency.

MMLU[41] measures broad multi-task language understanding across a wide range of academic subjects, serving as a proxy for general knowledge reasoning.

GPQA[43] focuses on graduate-level, Google-proof question answering, emphasizing deep reasoning and domain-specific expertise rather than surface retrieval.

TruthfulQA[44] assesses a model’s tendency to produce truthful answers when faced with questions that trigger common misconceptions or socially prevalent false beliefs.

These benchmarks collectively capture both procedural reasoning and factual reliability, providing a comprehensive view of a model’s general cognitive competence. Following standard evaluation protocols, we report accuracy for all benchmarks. The same evaluation settings are applied to base models and RoleCDE-trained variants to ensure fair comparison.

10.8 General Reasoning Ability Evaluation Matrix↩︎

The general reasoning ability of each model is quantified using benchmark-specific accuracy metrics, consistent with prior literature. For each benchmark instance, a model is required to generate a final answer, under 0-shot QA scenario, which is then compared against the ground-truth label.

The evaluation matrix for general reasoning consists of the following components:

GSM8K Accuracy: proportion of problems with exact numerical matches.

MMLU Accuracy: average accuracy across all subject categories.

GPQA Accuracy: exact-match accuracy on graduate-level questions.

TruthfulQA Accuracy: proportion of answers aligned with factual truth rather than common misconceptions.

For models fine-tuned using "RoleCDE-mini", we additionally report the performance difference relative to the corresponding base checkpoint. This comparison highlights whether improvements in role-aligned decision behavior introduce trade-offs in general reasoning performance.

As shown in Table 5 of the main paper, the observed changes are generally small and inconsistent across benchmarks, indicating that RoleCDE training preserves general reasoning ability while shifting decision preferences in role-conflicting scenarios.

10.9 General Role-Playing Ability Evaluation Benchmark↩︎

Beyond decision making under value conflicts, we further evaluate whether RoleCDE training affects conventional role-playing ability measured by standard role grounding benchmarks. These benchmarks focus on surface-level role fidelity, including instruction adherence and role-consistent content generation, without explicitly modeling value conflicts.

Following prior work, we adopt RoleBench[8] as the general role-playing ability evaluation benchmark, which consists of two complementary settings:

RoleBench-InstEng: evaluates instruction grounding, measuring how well a model follows role-related instructions during generation.

RoleBench-RoleEng: evaluates role grounding, assessing whether generated responses remain consistent with the specified role identity and background.

Both benchmarks are reference-based and emphasize linguistic realization, background knowledge consistency, and adherence to role constraints. They do not require explicit decision making under value conflict, making them suitable for isolating traditional role-playing fidelity from the decision-centric objectives of RoleCDE.

10.10 General Role-Playing Ability Evaluation Matrix↩︎

General role-playing performance is evaluated using standard NLG similarity metrics between model-generated responses and reference role-consistent outputs. Specifically, we report:

ROUGE-1: unigram overlap, capturing lexical alignment.

ROUGE-2: bigram overlap, reflecting local coherence.

ROUGE-L: longest common subsequence, measuring global structural similarity.

These metrics are computed separately for RoleBench-InstEng and RoleBench-RoleEng. As reported in Table 6 of the main paper, RoleCDE-SFT and RoleCDE-DPO models achieve ROUGE scores comparable to their base counterparts across both instruction grounding and role grounding settings. The absence of consistent degradation indicates that improving role-aligned decision behavior through RoleCDE does not compromise conventional role-playing fidelity.

10.11 Further experiment set-up↩︎

API Usage for Data Generation. We employed GPT-4o[31] to generate the synthetic role-playing data used in this study. The model was accessed through the API with default inference settings. All generated data were subsequently reviewed and manually filtered by the authors to ensure quality and validity.

Base Models, Environment, and Inference Setup. We evaluated six instruction-following large language models obtained from the HuggingFace repository, all of which were used in compliance with their respective licenses. The evaluated models include meta-llama/Llama-3.1-8B-Instruct[47], as well as Qwen/Qwen2.5-7B-Instruct[48].

To ensure reproducibility and consistency across evaluations, we adopt a zero-shot prompting setting with greedy decoding and set the temperature to 0 for all experiments. Model inference is conducted using vLLM[49], which provides efficient and scalable inference for large language models.

10.12 Training set setup↩︎

RoleCDE-mini Data Construction for SFT and DPO To mitigate Role-Value Decoupling with targeted training signals, we construct RoleCDE-mini, a synthetic training corpus derived from RoleCDE dilemma instances, and instantiate two complementary supervision regimes: SFT and DPO. The construction pipeline is designed to (i) enforce label-controllable decision behavior under explicit role-alignment conflict, (ii) validate label correctness with an external judge, and (iii) support robust learning by pairing structurally matched responses that differ primarily in decision stance and compromise style.

For each persona-scenario pair, we build a unified prompt that includes: (a) the expanded role profile and contextual scenario, (b) a set of three alignment-oriented “traditional values” and one conflicting “personal value”, and (c) a difficulty-specific dilemma with two options, where Option A is aligned/traditional and Option B expresses role/personal values. This shared scaffold ensures that all synthesized responses are conditioned on identical evidence and constraints, enabling controlled comparisons across labels.

We use a teacher LLM to generate candidate responses under hard, label-dependent constraints. Specifically, we explicitly control (1) "which option is chosen" (A for alignment-side labels and B for role-side labels), and (2) "the rhetorical structure of reasoning." For compromise labels (RC/AC), the teacher must acknowledge the opposing option and include explicit concession markers (e.g., although, however, despite) to demonstrate a trade-off. For following labels (RF/AF), the teacher is instructed to avoid compromise markers and produce a one-sided, value-committed justification. This produces stylistically and semantically distinct rationales that reflect the intended cognitive stance rather than merely flipping the final choice.

Judge verification and rule-based filtering.To reduce annotation noise, each teacher response is verified by an independent judge LLM that outputs a one-hot label in {RF, RC, AC, AF} based on the chosen option and the presence/absence of explicit trade-off language. In addition, we apply lightweight rule checks to ensure coverage of key conditioning signals. For example, RC samples must reference both the personal value and at least one traditional value (or their paraphrases) and must contain compromise markers; RF samples must reference the role identity and at least one concrete scenario/dilemma detail while remaining free of compromise markers. Samples that fail any constraint, verification, or coverage check are discarded and logged for debugging.

SFT corpus. For SFT, we retain only role-side demonstrations (RF and RC) to directly teach models to prioritize role-consistent decisions under conflict while maintaining coherent, persona-grounded reasoning. Each accepted instance is stored as a multi-turn message triple (system, user, assistant) with a JSON-formatted assistant output containing chosen_option and reasoning. This yields a high-precision demonstration set emphasizing role-following and role-compromise behaviors.

DPO preference pairs. For DPO, we synthesize up to four labeled responses per prompt (RF/RC/AC/AF) using the same controlled-generation and judge-verification procedure, and then form preference pairs that share the same input prompt while differing in decision stance. We prioritize structurally matched comparisons that isolate role-consistent preference: RC \(\succ\) AC (both are compromise, but differ in final choice) and RF \(\succ\) AF (both are following, but differ in final choice). When an ideal counterpart is unavailable, we fall back to RC \(\succ\) AF or RF \(\succ\) AC. This pairing strategy exposes the optimizer to minimal-change contrasts, encouraging the model to shift preferences toward role-consistent decisions without conflating the signal with unrelated stylistic differences.

10.13 Training Hyperparameters↩︎

6pt

6pt

We fine-tune the base instruction-following models using parameter-efficient LoRA adapters under two complementary regimes: supervised fine-tuning (SFT) and Direct Preference Optimization (DPO). Both pipelines are implemented with TRL and HuggingFace Transformers, and are configured to be reproducible and memory-efficient. We use chat-style formatting for SFT by converting each training instance into a single serialized dialogue via the tokenizer chat template, while DPO consumes triplets of (prompt, chosen, rejected) where the prompt is a unified role-scenario-dilemma scaffold and the chosen/rejected are JSON-formatted decisions and justifications.

Both SFT and DPO apply LoRA to the same set of Transformer projection modules (query/key/value/output and MLP projections), thereby updating only a small number of parameters while keeping the base model frozen. Training stability is supported by enabling gradient checkpointing, disabling KV-cache during training (use_cache=False), and using cosine learning-rate schedules with warmup. For SFT, we use 4-bit quantization (NF4) with bf16 compute and an 8-bit paged AdamW optimizer to further reduce memory footprint. For DPO, we instantiate both a policy model (trainable with LoRA) and a reference model (kept frozen), optionally under 4-bit quantization when bitsandbytes is available; DPO then optimizes the preference objective with a temperature-like parameter \(\beta\).All the models were trained using 2 × H100 GPUs.

LoRA configuration (shared). For both training regimes, LoRA is applied to q_proj, k_proj, v_proj, o_proj, up_proj, down_proj, gate_proj with rank \(r=16\), \(\alpha=32\), and dropout \(0.05\).

10.14 Numeric Results of LLM-as-a-judge↩︎

Check Table 12 and 11 for details

6pt

6pt

10.15 Detail information of Role Categories↩︎

Check Table 13 for details

10pt

10.16 Prompts↩︎

We report the employed prompts in RoleCDE. All relevant prompts are shown on later pages.

References↩︎

---

1. Correspondence.↩︎