CCBench: Evaluating Cultural Competency in LLM Responses to Health QueriesReading Between the Norms: Evaluating LLM Responsiveness to Implicit Cultural Cues through Health Queries
CCBench: Assessing LLM Cultural Competence via
Implicitly Signaled Norms using Health Queries
June 08, 2026
Abstract
To interact with users fairly and without stereotyping, AI models must display cultural competency, i.e., the ability to infer and adapt to a user’s implicitly signaled cultural values, rather than relying on static demographic traits. We introduce CCBench, a framework for evaluating cultural competency in large language models (LLMs), treating culture as a continuum of norm adherence states rather than as a binary state of cultural belongingness. As a case study on health, we create CCBench-Health, which includes 60 theoretically grounded personas exhibiting varied norm‑adherence states across six cultures, each engaging in 18 realistic dialogues. Each persona is evaluated on 52 authentic healthcare questions drawn from real user forums, yielding 3,120 unique interactions. Benchmarking five leading models reveals that even the best achieve culturally appropriate responses only 20-30% of the time. When explicitly prompted to focus on culturally relevant cues from the conversational history (CoT), performance improves modestly by 3-5% on average. We find that models perform best when personas avoid cultural norms rather than follow them, revealing a persistent asymmetry, suggesting a preference in the models to align with built-in biases than adapt to cultural cues. This is especially observed in the Afghan context (Avg: 8.8%), where cultural cues rarely yield appropriate health advice. Finally, we find that models sometimes adapt more readily to implicit, cultural conversational styles than to explicitly stated cultural practices, though this varies across cultures.
1 Introduction↩︎
Large Language Models (LLMs) are now global intermediaries of information that interact with a diverse global user base; however, they frequently default to "WEIRD" (Western, Educated, Industrialized, Rich, Democratic) value systems, leading to a persistent cultural bias [1]–[3]. To address this gap, we draw on cultural competence – a concept originally developed in healthcare to describe practitioners’ ability to provide effective care to patients with diverse values and beliefs [4], [5]. In high-stakes domains like health, where content and communication style can directly impact user engagement, a model’s inability to align with a user’s cultural expectations can lead to fundamental erosion of trust and safety [6], [7].
A significant challenge in achieving this competence in AI is that cultural belonging and norm adherence are rarely made explicit in natural interaction. Users seldom declare their heritage or belief systems directly; instead, they signal their context through implicit narratives, communication preferences, and social cues. As LLMs become increasingly personalized and relied upon for high-stakes decisions [8], [9], they must develop sensitivity to recognize these subtle signals. However, current benchmarks largely treat cultural identity as a binary attribute — categorizing a user as either belonging to a culture or not [10], [11]. This reductive approach ignores the significant intracultural variation and fluidity of real human identity. A model that assumes every user from a specific background adheres monolithically to their traditional norms risks not only misalignment but also the reinforcement of harmful stereotypes [12], [13].
We introduce CCBench, a framework designed to evaluate cultural competence in LLMs. Our framework operationalizes cultural identity as a continuum of norm adherence states revealed implicitly through a simulated conversation history. By presenting models with personas whose adherence to specific norms is signaled implicitly through previous interactions, we can test whether the model correctly infers the user’s position in the cultural participation spectrum and calibrates its response accordingly. This shift from prescriptive tests to conversational reveal allows for a more rigorous evaluation of how models handle the fluidity and multiplicity
of human values [14].
We instantiate this framework as CCBench-Health, a benchmark focused on assessing LLMs’ ability to generate culturally competent responses to health-related queries. Healthcare serves as a critical stress test for this framework because cultural alignment in clinical communication is not merely a matter of etiquette but a determinant of health outcomes [15]. Our benchmark is based on the Mosaica resource, which provides a theoretical foundation on multiple dimensions of health identity, including family decision-making, dietary norms, and gendered practices. We utilize a pipeline that combines these values to 52 high-signal health queries derived from real health forums, covering 12 stratified health topics across six distinct cultures (Afghan, Burmese, Chinese, Maori, Nepali, and Vietnamese).
Our experiments involve a systematic evaluation of five leading LLMs to assess their performance in these high-stakes, implicit contexts. The results reveal a pervasive deficit in adapting to implicit norm reveals across the benchmark, but with a notable asymmetry: models perform significantly better when personas prefer to avoid specific cultural norms than when they prefer to follow them. This discrepancy suggests that the Western default baked into these models essentially functions as an inherent resistance to adopting non-Western cultural norms; consequently, the models appear more competent when the task aligns with their pre-existing biases. True cultural competence — actively aligning conversations to previously-stated cultural values — remains a profound challenge. Even when explicitly prompted with cultural norm adherence states, models fail to consistently adapt to them. Their performance drops further in the Afghan context (Avg Cultural Competence Score : 0.088), where models struggle to translate even direct cultural cues into appropriate health advice.
2 CCBench↩︎
The Cultural Competency Benchmark (CCBench) is a domain-agnostic framework designed to evaluate the implicit cultural competence of LLMs. Unlike traditional benchmarks that rely on explicit self-identification (e.g., “As a person from culture \(X\)...”), CCBench measures the ability of a model to infer and adhere to latent cultural norms through behavioral cues embedded in a conversation history.
2.0.0.1 3.1 Persona Profiles and Norm Adherence States
The CCBench framework utilizes a hierarchical representation of cultural identity, distinguishing between abstract values and the norms that operationalize them (e.g., in Afghan context, the cultural norm “The person may hesitate to ask questions” is associated with two Afghan cultural values: 1. Respect for Authority and Hierarchy, 2. Politeness and Harmony).
While our framework can be seeded with pre-existing persona profiles derived from empirical value surveys or cultural databases, it is designed to be self-sufficient: in the absence of such data, we synthesize value-grounded personas from scratch. This flexibility ensures the benchmark can be deployed even in data-scarce cultural contexts while remaining grounded in a value-norm hierarchy.
Let \(\mathcal{V} = \{v_1, v_2, \dots, v_m\}\) be a set of core cultural values and \(\mathcal{N} = \{N_1, N_2, \dots, N_l\}\) be a set of behavioral norms. Each norm \(N_i\) is mapped to a non-empty subset of values \(\mathcal{V}_{N_i} \subseteq \mathcal{V}\) that govern it. To account for intra-cultural diversity, we generate synthetic personas \(P_k\) defined by their specific identification with these underlying values.
2.0.0.2 Value Adherence
For each culturally-situated persona \(P_k\), we define a binary value adherence function: \[\small A_k(v_j) \in \{-1, 1\}\] where \(+1\) indicates that the persona subscribes to the cultural value, and \(-1\) indicates they do not. This binary assignment reflects the fundamental personal stance an individual holds toward a cultural value.
2.0.0.3 Norm Derivation
While values are binary, the resulting behavioral norms may exhibit a spectrum of adherence based on the interaction of multiple values. We derive the persona \(P_k\)’s adherence to a specific norm \(N_i\) by calculating the mean of the adherence scores of its associated values: \[\small \mu_k(N_i) = \frac{1}{|\mathcal{V}_{N_i}|} \sum_{v \in \mathcal{V}_{N_i}} A_k(v)\] The final norm adherence function \(C_k(N_i)\) is then categorized into a ternary state representing the persona’s behavioral orientation:
\[\small C_k(N_i) = \begin{cases} +1 & \text{if } \mu(N_i) > 0 \quad \text{(Follow)} \\ -1 & \text{if } \mu(N_i) < 0 \quad \text{(Avoid)} \\ 0 & \text{if } \mu(N_i) = 0 \quad \text{(Neutral)} \end{cases}\]
This allows for nuanced personas who may follow some norms while remaining neutral or avoidant of others, depending on how their specific value set conflicts or aligns. By sampling unique combinations of \(A_k\), the framework systematically constructs a diverse spectrum of cultural agents.
2.0.0.4 3.2 Behavioral Grounding with Conversational History
To evaluate implicit competence, the persona’s cultural alignment must remain latent. CCBench utilizes a behavioral grounding stage where a multi-turn conversation history \(H_k\) is generated between the persona \(P_k\) and an AI assistant. This history is constrained by the adherence function \({C_k}\) such that the persona demonstrates their values through linguistic style, social etiquette, and decision-making patterns without explicitly stating their geographic or cultural background. This paradigm ensures that a test model can only succeed if it correctly interprets the subtle behavioral signals present in \(H\).
2.0.0.5 3.3 Checklist-Based Evaluation
The evaluation follows a context-response paradigm. Given the grounded history \(H_k\) and a novel domain-specific query \(Q^i\), the test model generates a response \(R^i_k\). To objectively score \(R^i_k\), we construct a norm-specific checklist \(\mathcal{L}\) derived from the persona’s norm adherence function \({C_k(N_i)}\). For each \(C_k \neq 0\), an LLM generates a specific recommendation for how that behavior should be acccommodated in the response. The proportion of satisfied checklist items as determined by a strong evaluator LLM is used in calculating the cultural competence metrics.
3 CCBench-Health Benchmark Creation↩︎
The CCBench-Health benchmark is designed to measure the cultural competence of LLMs through their ability to respond to implicit cultural norms in health-related contexts. The construction of this benchmark follows a multi-stage pipeline involving theoretically-grounded cultural report sourcing, persona simulation, and rigorous filtering.
3.0.0.1 Norm Sourcing and Value Identification
The foundational stage of this methodology involves curating a comprehensive list of health-related cultural norms and values to serve as the ground truth for model evaluation. These are grounded in cultural health profiles from Mosaica [16] to ensure realism, and clinical and cultural accuracy. Mosaica is a knowledge base of cross-cultural and religious insights, curated by culture experts over months of interviews and engagement with the communities. The health profile provides a resource spanning multiple dimensions of health identity: healthcare approaches, communication, wellbeing challenges, diet and nutrition, family, women, and dying.
Figure 3: 
.
Figure 4: 
.
To operationalize this data, we utilized Gemini-3.5-Pro to distill Mosaica reports into structured norm-value pairs. These pairs are framed by describing a persona’s behavior (e.g., ‘The person may be hesitant to trust doctors...’) alongside the corresponding health-related cultural value. We manually verified all the norm-value pairs to ensure there were no values and norms hallucinated by the model. Table 3 shows the number of norms and values derived for each cultural profile. A comprehensive list of norms and values is shown in §8.1.
3.0.0.2 Persona Simulation and Norm Mapping
To develop diverse, realistic testing agents, the framework employs a stochastic persona creation approach (§2). For each cultural value, a persona is randomly assigned a binary adherence state: adhere (+1) or not adhere (-1). This reflects intra-cultural diversity, as individuals do not uniformly subscribe to all traditional values; by sampling 10 unique adherence combinations per culture, the benchmark avoids monolithic identity assumptions. Since each norm maps to one or more values, the final norm adherence state is the average of its associated value scores: positive averages yield Follow, negative yield Avoid, and zero yields Neutral. Extracted norms and values are detailed in Table 3.
3.0.0.3 Background History Creation
After assigning each persona its cultural norm-adherence states, we generated detailed background histories (Figures 14–16) to anchor identities in consistent,
culturally grounded narratives. To approximate realistic interactions, conversations were seeded with starter queries rather than simulated directly from norms. We filtered WildChat-1M [17] by clustering sentence embeddings (all-MiniLM-L6-v2), discarding non-social clusters (e.g., coding, essay writing), and randomly sampled starters (Figure 11) to prompt background conversations between each persona and an LLM (GPT-5.2), conditioned on predefined norms. Since many WildChat queries were too generic to elicit cultural health norms, we supplemented them
with handpicked health and lifestyle queries generated via GPT-4 (Figure 10). Each persona received 3 health-related and 15 WildChat-derived sessions. Communication norms (e.g., hesitancy to
discuss reproductive health) emerged stylistically, whereas practice-based norms (e.g., deferring to elders) were stated directly. To ensure consistency and prevent norm hallucinations, prompts instructed the model to restate norm
adherence states at the end of the conversation. These were used for internal verification only, and were omitted during evaluation.
3.0.0.4 Filtering
We then filtered these conversations to ensure cultural background remained implicit: explicit identity statements (e.g., “I am from [country]”) were removed, while cultural greetings (e.g., Namaste) or religious artifacts (e.g., I eat halal food) were retained as implicit cues. Norms were required to emerge through behavior (e.g., initiating formal greetings) rather than explicit declaration. We used LLM-based checks (Figure 18) to verify norm consistency and signal clarity, with results in Table 4. Manual validation by two annotators confirmed the LLM’s accuracy, showing \(>75\%\) raw agreement with both annotators (moderate-to-high agreement; see § 8.3).
3.0.0.5 Health Query Curation
To compile diverse, real-world health queries, we used eHealth and iCliniq [18], datasets capturing personal health concerns and cultural illness expressions. From this corpus, we manually derived 12 health categories (e.g., Disease, Mental Health, Reproductive Health) based on existing tags. We then employed GPT-5.2 to annotate each query’s relevance to the extracted cultural norms, selecting only those relevant to over 90% of norms to ensure a rigorous test. Finally, we stratified these across the 12 topics to derive 52 high-signal health queries, which serve as the primary probes for implicit cultural competence in CCBench-Health.
3.0.0.6 Checklisting
To evaluate model responses, we generate persona-specific checklists using GPT-5.2. For every norm where \(C_k(N_i) \neq 0\), the model produces a specific recommendation for norm-following (\(+1\)) or norm-avoidance (\(-1\)). A strong evaluator (GPT-5.2) then scores responses based on the proportion of satisfied recommendations, providing a quantitative measure of cultural competence (prompt in 21). We validated this pipeline by manually annotating 50 responses, defining competence as direct alignment with the persona’s preference. Human annotators achieved 74% raw agreement; the LLM evaluator showed moderate-to-high alignment, agreeing with Annotator 1 at 64% and Annotator 2 at 62%. These results confirm that our checklist-based scoring serves as a robust proxy for human-perceived cultural competence.
3.0.0.7 Evaluation Metrics
To assess cultural competence, we use metrics evaluating how well an LLM aligns with a persona’s defined cultural norm adherence state (Follow, Avoid, or Neutral). Performance reflects adaptation accuracy to the personas:
Follow Rate (Cultural Sensitivity): Frequency of adapting to norms a persona prefers; high rates indicate strong adaptation to culture-specific practices when cued.
Avoid Rate (Stereotype Resistance): Frequency of correctly adapting to avoidance of cultural; high rates indicate resistance to cultural stereotyping, indicating the ability to avoid generalizing based on cultural background.
Overall Norm Adaptation: Consistency in aligning with all persona stances (Follow or Avoid), reflecting adaptive competence irrespective of cultural background.
Cultural Competency Score (CCS): Harmonic mean of Follow and Avoid Rates, rewarding nuanced, context-aware adaptation over binary assumptions.
4 Experiments↩︎
We conducted a systematic evaluation across four distinct prompting configurations. These settings were designed to test the models’ ability to move from zero cultural context to a theoretical upper bound where all user norms are explicitly provided.
4.0.0.1 Experimental Setup
All experiments were conducted using a standardized temperature of 0.7 to allow for natural linguistic variation while maintaining structural consistency in health recommendations. We evaluated a diverse suite of state-of-the-art models, including proprietary systems (GPT-5.2, Gemini-2.5-Pro, and Claude-3.5-Sonnet) and high-parameter open-source models (Llama-3.2-90B and Qwen-3.5-397B).
4.0.0.2 Evaluation Settings
We explore three settings to isolate the impact of conversational context and explicit reasoning on model performance:
No Context (None): The model is prompted with the healthcare query, without any context.
With Conversation Context (Hist.): The model receives the simulated background conversation history as a prefix to the health query. The model must implicitly infer the user’s norm-adherence level from these conversational cues to calibrate its response.
Culture-CoT (Cultural Chain-of-Thought or CCoT): Building on the conversational context, we instruct the model to focus on cultural cues in the conversational history step-by-step before answering. The model is prompted to: (a) identify culturally relevant information signaled in the history, (b) how the health advice should be tailored to accommodate the cultural information.
Norm Context (Norms): As an upper bound, the model is explicitly provided with the persona’s ground-truth norm definitions and adherence states in the system prompt. This serves as a theoretical upper bound for the model’s performance, as it removes the need for implicit inference.
0.45
| CCS | ||||
|---|---|---|---|---|
| 2-5 Model | None | Hist. | C-CoT | Norms |
| GPT-5.2 | .036 | .115 | .172 | .309 |
| Gemini-2.5-pro | .023 | .088 | .172 | .318 |
| Deepseek-R1 | .031 | .080 | .092 | .255 |
| LLaMA-90B | .025 | .049 | .069 | .185 |
| Qwen-3.5-397B | .019 | .109 | .087 | .009 |
0.45
| Overall Norm Adaptation | ||||
|---|---|---|---|---|
| 2-5 Model | None | Hist. | C-CoT | Norms |
| GPT-5.2 | .248 | .287 | .307 | .361 |
| Gemini-2.5-pro | .197 | .245 | .278 | .326 |
| Deepseek-R1 | .219 | .247 | .251 | .249 |
| LLaMA-90B | .203 | .198 | .207 | .191 |
| Qwen-3.5-397B | .147 | .242 | .098 | .008 |
0.45
| Avoid Rate | ||||
|---|---|---|---|---|
| 2-5 Model | None | Hist. | C-CoT | Norms |
| GPT-5.2 | .473 | .517 | .525 | .509 |
| Gemini-2.5-pro | .382 | .453 | .469 | .394 |
| Deepseek-R1 | .382 | .460 | .464 | .277 |
| LLaMA-90B | .397 | .379 | .390 | .255 |
| Qwen-3.5-397B | .286 | .437 | .136 | .009 |
0.45
| Follow Rate | ||||
|---|---|---|---|---|
| 2-5 Model | None | Hist. | C-CoT | Norms |
| GPT-5.2 | .035 | .065 | .103 | .222 |
| Gemini-2.5-pro | .021 | .049 | .105 | .267 |
| Deepseek-R1 | .023 | .044 | .051 | .237 |
| LLaMA-90B | .018 | .026 | .038 | .145 |
| Qwen-3.5-397B | .014 | .062 | .064 | .009 |
5 Results: How Culturally Competent are LLMs?↩︎
5.0.0.1 Models show limited cultural competence.
Across all settings, no model demonstrates strong proficiency in this task (Table ¿tbl:tab:main-results-2x2?. Relative to the no-context baseline, all contextual methods: conversational context, Cultural Chain-of-Thought (Culture-CoT), and explicit norm adherence states– improve performance across all metrics, validating measuring the benchmark’s sensitivity to implicit cultural cues. Still, models reveal a pronounced imbalance between Follow and Avoid Rate scores: they readily avoid cultural norms yet rarely follow them. For instance, GPT-5.2 achieves a 51.7% avoidance rate under conversational context but only 6.5% follow accuracy. This asymmetry reflects a “Western default” where omission of non-Western norms is misinterpreted as competence, while genuine cultural accommodation remains weak. Even when persona-level cultural information is explicitly provided, models perform modestly – GPT-5.2 peaks at 36.1% average accuracy – indicating that explicit metadata alone is insufficient for aligned health recommendations. Culture-CoT yields small but consistent gains1, suggesting reasoning scaffolds help but cannot substitute targeted cultural adaptation methods.
5.0.0.2 Cultural Embeddedness reduces norm adaptation.
As shown in Figure 5, norm accommodation declines sharply with a persona’s cultural embeddedness (i.e., adherence to cultural norms), rendering models less adaptive for deeply ingrained cultural users. Active adherence norms ("Follow norms") are accommodated in only rare cases. This trend suggests that current LLMs associate high cultural specificity with outlier behavior, leading them to default to generalized or Western-centric responses rather than tailoring to culturally distinct health practices.
Figure 5: 
.
Figure 6: 
.
5.0.0.3 Stereotype Resistance Varies across cultures.
Figure 6 reveals that while all models perform poorly on Follow Rate (Cultural Sensitivity) across cultures – typically achieving \(0-7\%\) adherence – their Avoid Rate varies considerably. Māori personas exhibit notably high Avoid Rates, reflecting stronger stereotype resistance. This may stem from the distinctiveness of Māori linguistic and cultural markers in conversation histories, which make these cues easier for models to detect and suppress when avoidance is appropriate. By contrast, Afghan personas show both low Avoid and low Follow rates, suggesting that models neither capture nor correctly modulate cultural context for this group. Such disparities point to uneven representation of cultures within pretraining data and inconsistent modeling of non-Western identities.
Figure 7: 
.
Figure 8: 
.
5.0.0.4 Models might attend more to cultural style than substance.
Contrary to expectation, communication norms – signaled only through conversational style rather than explicit content – are sometimes followed more accurately than practice norms. For Afghan and Nepali personas, models adapt communication style at higher rates than cultural practices (Figure 7). However, this advantage is driven primarily by avoidance: models excel at suppressing taboos when a persona appears norm-avoidant, but rarely follow positive cultural cues. Notably, Māori personas are an exception: their practice norms outperform communication norms, indicating that performance here is driven by explicit, distinctive cultural artifacts that surface clearly in conversation. Overall, this pattern suggests that models are more sensitive to surface-level stylistic signals when they align with the Western default, with responsiveness varying across cultures depending on how distinctly those cues are represented in training data.
5.0.0.5 Topic-wise performance shows consistent cultural trends.
Models perform best on topics related to Relationships and Addiction. Overall, variation across topics is modest, with similar trends observed across cultures: Māori personas consistently achieve the highest accommodation rates across all topics, while Afghan personas perform poorly across the board, with only slight improvements in Relationships and Mental Health. This consistency suggests that health topic specificity has limited influence compared to the underlying cultural distinctiveness of the persona.
In summary, these findings reveal a pervasive asymmetry in LLMs’ cultural adaptability: models adeptly avoid non-Western norms by default but falter in actively accommodating norm adherence states, particularly for deeply embedded cultural personas. This Western-centric baseline, compounded by weak decoding of implicit cues and limited gains from reasoning aids like Culture-CoT, underscores the need for richer multicultural training data and techniques to achieve equitable cultural competence across diverse user contexts.
6 Related Work↩︎
6.0.0.1 Culture Evaluation in LLMs
Early research primarily focused on intrinsic evaluations, utilizing established sociological frameworks like Hofstede’s Cultural Dimensions [19] or the World Values Survey [20]–[24]. While these frameworks identify broad misalignments, they often flatten culture into discrete variables or treat it as a repository of factual knowledge [25]–[27], risking a "trivia contest" that overlooks the gap between theoretical awareness and appropriate social application [28], [29]. Recent efforts like NormAd and NormBank [30]–[32] evaluate situational etiquette but rely on discrete-choice formats or prescriptive labels that miss interactional nuance [33], [34]. The more recent dialogue-based norm violation detection [35] remains tied to binary cultural belonging, whereas real cultural identity is fluid: individuals selectively follow, question, or reject norms [13]. Our work addresses this by integrating conversational norm detection with persona generation based on partial adherence and value systems, assessing how models navigate the dynamic, often conflicting nature of cultural values in real time.
6.0.0.2 Implicit cues and Personalization
Current investigations into implicit cues often rely on explicit signals. For example, [36] show that revealing names leads models to default to majority-culture assumptions, while [37] find that LLMs infer demographic traits from language and still revert to stereotypical priors even when attributes are stated. Although such cues enable personalization, they reveal systematic, opaque biases [38]. In most studies, these “implicit” cues are effectively explicit since demographic features are disclosed to the model; [36] is a partial exception but focuses on a single identity dimension. Our approach instead defines personas through value adherences and tests implicit cues within conversational histories, moving beyond binary stereotyping toward modeling how LLMs navigate intersectional, multifaceted cultural contexts.
6.0.0.3 Cultural Competence in Healthcare AI
The rapid adoption of AI has transformed access to medical guidance, with users increasingly turning to LLMs for personalized support [39], [40]. This trend is especially visible in mental health, where chatbots provide emotional assistance [41] and sometimes match or even surpass clinicians in empathy [42], [43]. Yet effective guidance requires more than accuracy—it demands sensitivity to individual values and social norms [2], [44]–[47]. Current systems fall short in cross-cultural adaptation [5], and strong English performance rarely translates into multicultural competence due to lost cultural nuance in translation [48], [49]. These challenges highlight the need for intersectional, norm-based evaluation frameworks in healthcare AI.
7 Conclusions↩︎
In this work, we introduce CCBench, a comprehensive framework designed to evaluate cultural competency in LLMs by measuring their ability to move beyond static demographic labels and instead navigate the fluid, implicit nature of cultural identity in dialogue. We instantiate this framework in the high-stakes domain of healthcare as CCBench-Health, a benchmark that probes model sensitivity to culturally grounded health queries.
Our findings reveal a significant “competency gap" in modern LLMs: while models show higher proficiency in resisting stereotyping – correctly identifying when a user does not adhere to a traditional cultural norm – they consistently struggle with cultural sensitivity, or the active integration of non-Western cultural values when they are signaled by the user. This asymmetry suggests that models often default to a Western-centric baseline; they appear successful at resisting stereotypes primarily because their default output is already devoid of specific cultural markers. Even when provided with explicit norm definitions as a theoretical upper bound, the level of cultural sensitivity remains strikingly low, suggesting that current alignment techniques are insufficient for achieving true cultural competence. Furthermore, while Culture-CoT offers a modest improvement in decoding implicit cues, it cannot fully compensate for the underlying lack of diverse cultural representation in model training. These results serve as a call to action for the development of AI that moves beyond”one-size-fits-all" recommendations toward a more nuanced, culturally congruent model of care that respects the diverse and dynamic identities of global populations.
Ethics Statement↩︎
The personas used in this study were simulated based on health reports derived from Mosaica. While designed to reflect culturally grounded health behaviors, they may not fully represent real-world users and could inadvertently contain stereotypical phrases or artifacts. To mitigate this, we interspersed norm-related exchanges with regular, non-health conversations to better approximate realistic user–LLM interactions.
Performance on this benchmark may not perfectly mirror real-world LLM behavior, but it aims to approximate how models respond to implicit cultural cues in controlled settings. Our analysis is limited to six cultures, as high-quality cultural health profiles within Mosaica are still being curated; we prioritized data quality over breadth. All norm curation, conversation-history generation, and checklist-based evaluations were manually reviewed to ensure accuracy and consistency.
We employed high-reasoning, large-scale LLMs to simulate and construct this benchmark, a process that entails substantial compute, time, and financial costs, which may limit immediate replicability. Nevertheless, we believe that investing in high-capacity, billion-parameter models is essential to produce a nuanced, high-quality resource that faithfully emulates realistic user conversations. Smaller models lack the representational depth needed to capture such subtleties, and we hope this framework will serve as a scalable foundation for future work on cultural competence – an area that remains critically underexplored.
8 Appendix↩︎
8.1 Deriving Theoretically-grounded Norms and Associated Values↩︎
We derive norms and their associated cultural values from Mosaica health profiles for the cultures Afghan, Burmese, Chinese, Maori, Nepali and Vietnamese. The prompt for extracting the norms from the reports is shown in Figure 9.
None
Figure 9: Prompt instructions used for generating norms and associated values for each cultural health profiles..
We show all the norms and their associated cultural values derived from the Mosaica cultural health profiles in Tables 1, 2, 3, 4, 5 and 6.
| Norm ID | Norm Statement | Associated Values |
|---|---|---|
| afghan_1 | The person expects interactions to begin with a polite greeting and a warm, welcoming demeanor to build rapport before discussing medical business. | Hospitality and Rapport; Respect for Authority and Hierarchy |
| afghan_2 | The person may hesitate to ask questions, express disagreement, or admit to a lack of understanding due to cultural deference to authority figures. | Respect for Authority and Hierarchy; Politeness and Harmony |
| afghan_3 | The person (particularly if female) strongly prefers to interact with a female-coded voice or persona for sensitive health matters to maintain religious modesty boundaries. | Modesty and Dignity (Haya); Religious Adherence (Islamic Principles) |
| afghan_4 | The person requires explicit reassurance regarding confidentiality and data privacy early in the interaction, fearing that health information might spread within their close-knit community. | Privacy and Honor (Nang/Namus); Collectivism and Family Duty |
| afghan_5 | The person may use euphemisms, non-verbal cues, or indirect language when discussing sensitive topics like sexual and reproductive health due to cultural taboos. | Modesty and Dignity (Haya); Privacy and Honor (Nang/Namus) |
| afghan_6 | The person may defer decision-making to a male family member (husband or father) or require their presence/input during the consultation. | Collectivism and Family Duty; Religious Adherence (Islamic Principles) |
| afghan_7 | The person adopts a stoic demeanor and downplays emotional pain, potentially presenting mental health distress through somatic (physical) symptoms. | Stoicism and Resilience; Privacy and Honor (Nang/Namus) |
| afghan_8 | The person adheres to Halal dietary laws, avoiding pork, alcohol, and non-halal meat, and expects health advice to align with these restrictions. | Religious Adherence (Islamic Principles) |
| afghan_9 | The person observes fasting during Ramadan and may require medication schedules to be adjusted accordingly. | Religious Adherence (Islamic Principles) |
| afghan_10 | The person may consume a diet high in carbohydrates and sugar while having a low intake of water, potentially impacting conditions like diabetes. | Hospitality and Rapport; Cultural Dietary Habits |
| afghan_11 | The person may exhibit “silent compliance”—agreeing to treatment plans to be polite, even if they do not intend to follow them. | Politeness and Harmony; Respect for Authority and Hierarchy |
| afghan_12 | The person expects physical examinations to adhere to strict modesty guidelines, such as keeping the hijab on or only exposing the necessary area. | Modesty and Dignity (Haya); Religious Adherence (Islamic Principles) |
| afghan_13 | The person may define health success through the ability to perform family duties and maintain collective wellbeing rather than solely individual metrics. | Collectivism and Family Duty |
| Norm ID | Norm Statement | Associated Values |
|---|---|---|
| burmese_1 | The person expects health interactions to be warm, personalized, and caring rather than purely transactional, as this builds trust and rapport. | Harmony and Politeness; Relationship Building |
| burmese_2 | The person may hesitate to ask questions, express disagreement, or correct the AI due to cultural hierarchy that views health practitioners as authority figures. | Respect for Authority; Harmony and Politeness |
| burmese_3 | The person may answer with responses perceived as most appeasing (e.g., saying “yes” to indicate listening) to avoid confrontation or impoliteness. | Harmony and Politeness; Face Saving |
| burmese_4 | The person may use indirect language, euphemisms, or non-verbal cues (silence, giggling) when discussing sensitive topics to maintain modesty. | Modesty and Privacy; Indirect Communication |
| burmese_5 | The person (particularly if female) may strongly prefer a female-coded AI or provider for sexual, reproductive, or mental health issues due to gender segregation norms. | Modesty and Privacy; Religious Observance |
| burmese_6 | The person may prefer to involve family members in health decision-making and treatment planning rather than making decisions autonomously. | Collectivism and Family Duty; Interdependence |
| burmese_7 | The person may downplay pain or severity of symptoms due to a cultural value of stoicism and a desire not to burden others. | Stoicism and Resilience; Face Saving |
| burmese_8 | The person may express mental health distress through somatic (physical) descriptions (headaches, back pain, “feeling hot”) rather than psychological terminology. | Holistic Health Beliefs; Stoicism and Resilience |
| burmese_9 | The person may interpret mental health issues through cultural idioms such as “thinking too much” or “feeling heavy” rather than Western clinical diagnoses. | Holistic Health Beliefs; Cultural Interpretation of Illness |
| burmese_10 | The person may utilize traditional herbal remedies alongside Western advice, viewing them as safer or complementary. | Holistic Health Beliefs; Reliance on Tradition |
| burmese_11 | The person may expect immediate tangible relief (such as medication advice) and may lose confidence if no direct treatment is offered. | Pragmatism; Expectation of Cure |
| burmese_12 | The person may delay seeking help until a condition is severe or viewed as an emergency, rather than engaging in routine preventative care. | Pragmatism; Stoicism and Resilience |
| burmese_13 | The person may follow specific dietary restrictions based on religious beliefs (e.g., Halal) or cultural beliefs (e.g., Ayurvedic heating/cooling foods). | Religious Observance; Holistic Health Beliefs |
| burmese_14 | The person may be motivated to accept health interventions by the desire to protect family and community rather than solely for individual benefit. | Collectivism and Family Duty; Community Responsibility |
| burmese_15 | The person may feel overwhelmed by complex health system navigation and requires clear, step-by-step logistical guidance. | Health Literacy Challenges; Need for Navigation Support |
| Norm ID | Norm Statement | Associated Values |
|---|---|---|
| chinese_1 | The person adopts a passive role during health consultations, deferring to the AI’s advice as an authority figure rather than asking questions or challenging information. | Respect for Authority and Hierarchy |
| chinese_2 | The person avoids verbalizing disagreement or dissatisfaction directly to preserve social harmony, potentially leading to silent non-compliance with treatment plans. | Harmony and Non-Confrontation; Mianzi (Face) and Reputation |
| chinese_3 | The person involves family members in health decision-making and may defer final decisions to a senior family member rather than acting autonomously. | Collectivism and Filial Piety |
| chinese_4 | The person minimizes or withholds expressions of pain and physical discomfort, viewing endurance as a sign of strength and character. | Stoicism and Self-Restraint |
| chinese_5 | The person describes pain using indirect terms such as “discomfort” or “unease” rather than explicitly using the word “pain”. | Stoicism and Self-Restraint; Modesty and Privacy |
| chinese_6 | The person views health holistically and attributes illness to imbalances in natural forces (Yin and Yang) or vital energy (Qi) rather than solely biological pathogens. | Holism and Balance (TCM/Yin-Yang) |
| chinese_7 | The person utilizes dietary therapy (consuming specific foods based on their “hot” or “cold” properties) to restore internal balance and treat illness. | Holism and Balance (TCM/Yin-Yang) |
| chinese_8 | The person uses Traditional Chinese Medicine (herbal remedies, acupuncture) alongside Western medical advice, often using the former to treat the “root cause” and the latter for symptom relief. | Holism and Balance (TCM/Yin-Yang); Pragmatism |
| chinese_9 | The person may stop taking prescribed Western medication once symptoms subside, believing the illness is resolved, or reduce dosage to avoid perceived toxicity. | Holism and Balance (TCM/Yin-Yang); Pragmatism |
| chinese_10 | The person perceives Western medicine as “strong” or aggressive and may use traditional herbal remedies to offset potential side effects. | Holism and Balance (TCM/Yin-Yang) |
| chinese_11 | The person delays seeking formal healthcare until symptoms significantly interfere with daily functioning, adopting a “wait and see” approach for mild conditions. | Stoicism and Self-Restraint; Pragmatism |
| chinese_12 | The person expresses mental health distress through somatic symptoms (e.g., headache, fatigue) rather than emotional terms due to stigma. | Mianzi (Face) and Reputation; Stoicism and Self-Restraint |
| chinese_13 | The person prefers same-gender interactions for sensitive health issues, particularly sexual and reproductive health. | Modesty and Privacy |
| chinese_14 | The person maintains emotional self-restraint and a calm demeanor when receiving bad news or discussing serious health issues. | Stoicism and Self-Restraint |
| chinese_15 | The person avoids scheduling appointments or procedures on dates containing the numbers 4, 14, or 24 due to cultural associations with bad luck and death. | Holism and Balance (TCM/Yin-Yang) |
| chinese_16 | The person prioritizes the collective wellbeing of the family over individual health needs, potentially sacrificing their own health to fulfill caregiving duties. | Collectivism and Filial Piety |
| chinese_17 | The person may conceal a disability or serious diagnosis from the wider community to protect the family’s reputation and avoid “loss of face”. | Mianzi (Face) and Reputation; Collectivism and Filial Piety |
| chinese_18 | The person prefers to keep the body intact after death and may be reluctant to consent to organ donation or autopsies due to beliefs about the spirit and afterlife. | Holism and Balance (TCM/Yin-Yang); Collectivism and Filial Piety |
| chinese_19 | The person expects to consume warm or room-temperature foods and liquids during illness and recovery, avoiding “cold” or raw foods. | Holism and Balance (TCM/Yin-Yang) |
| chinese_20 | The person relies on family networks and trusted peers for health information and recommendations before engaging with formal health systems. | Collectivism and Filial Piety |
| Norm ID | Norm Statement | Associated Values |
|---|---|---|
| maori_1 | The person expects to establish a personal connection (whanaungatanga) and rapport before discussing medical business, as impersonal communication is perceived as rude. | Whanaungatanga (Connection & Relationship); Manaakitanga (Hospitality & Care) |
| maori_2 | The person prefers to understand the specific role and ‘identity’ of the AI assistant early in the interaction to build trust, mirroring the cultural practice of introductions. | Whanaungatanga (Connection & Relationship); Mātauranga (Knowledge & Understanding) |
| maori_3 | The person prioritizes the well-being of their family (whānau) over their individual health and may delay seeking help to attend to family needs. | Kotahitanga (Collectivism & Unity); Manaakitanga (Hospitality & Care) |
| maori_4 | The person prefers to involve family members in health decision-making and may defer giving a final answer until they have consulted with their collective unit. | Kotahitanga (Collectivism & Unity); Rangatiratanga (Autonomy & Self-determination) |
| maori_5 | The person views health holistically (Te Ao Māori), believing physical health is linked to spiritual, mental, and social well-being. | Wairuatanga (Spirituality & Holism); Mātauranga (Knowledge & Understanding) |
| maori_6 | The person may feel uncomfortable asking questions unprompted or disagreeing with advice, requiring the AI to actively check for understanding. | Whakamā (Modesty & Diffidence); Mana (Dignity & Respect) |
| maori_7 | The person prefers plain, simple language over medical jargon to avoid feelings of alienation or frustration regarding their health literacy. | Mātauranga (Knowledge & Understanding); Whakamā (Modesty & Diffidence) |
| maori_8 | The person may minimize or downplay the severity of pain or symptoms due to stoicism and a desire not to be perceived as ‘weak’. | Whakamā (Modesty & Diffidence); Mana (Dignity & Respect) |
| maori_9 | The person may attribute certain illnesses to spiritual causes (mate Māori) or transgressions of tapu rather than purely biomedical causes. | Wairuatanga (Spirituality & Holism); Tapu and Noa (Sacredness & Safety) |
| maori_10 | The person considers the head to be the most sacred (tapu) part of the body and expects explicit permission before any discussion involving it. | Tapu and Noa (Sacredness & Safety); Mana (Dignity & Respect) |
| maori_11 | The person adheres to strict protocols separating food (noa) from sacred items (tapu), such as medication, and expects advice to respect this. | Tapu and Noa (Sacredness & Safety); Tikanga (Customs & Protocols) |
| maori_12 | The person may utilize traditional plant-based medicines (Rongoā) or spiritual healing alongside Western medicine and values validation of these practices. | Tikanga (Customs & Protocols); Mātauranga (Knowledge & Understanding) |
| maori_13 | The person may rely on non-verbal cues or silence to process information and expects the AI not to fill these pauses with excessive talking. | Mana (Dignity & Respect); Whanaungatanga (Connection & Relationship) |
| maori_14 | The person may exhibit initial mistrust toward formal health systems due to historical discrimination, requiring reassurance regarding privacy and intent. | Mana (Dignity & Respect); Rangatiratanga (Autonomy & Self-determination) |
| maori_15 | The person may wish to perform or have a prayer (karakia) performed before consuming food, taking medication, or undergoing procedures. | Wairuatanga (Spirituality & Holism); Tikanga (Customs & Protocols) |
| maori_16 | The person treats elders (kaumātua) with high esteem and expects them to play a leading role in significant family health decisions. | Mana (Dignity & Respect); Kotahitanga (Collectivism & Unity) |
| maori_17 | The person may be reluctant to discuss sensitive topics like mental health or substance abuse due to fear of stigma or bringing shame (whakamā) upon family. | Whakamā (Modesty & Diffidence); Kotahitanga (Collectivism & Unity) |
| maori_18 | The person views the return of body tissue (e.g., placenta) to the earth as culturally significant and may request information on retaining these items. | Tikanga (Customs & Protocols); Wairuatanga (Spirituality & Holism) |
| maori_19 | The person may avoid direct conflict with authority figures by agreeing with assumptions even if incorrect (silent compliance). | Whakamā (Modesty & Diffidence); Mana (Dignity & Respect) |
| Norm ID | Norm Statement | Associated Values |
|---|---|---|
| nepali_1 | The person communicates indirectly about sensitive health issues, using hints or metaphors, and expects gentle, roundabout delivery of bad news. | Harmony and Indirect Communication; Modesty and Privacy |
| nepali_2 | The person may hesitate to give a direct refusal, often saying “yes” to indicate listening rather than agreement, to avoid impoliteness. | Harmony and Indirect Communication; Respect for Hierarchy and Tradition |
| nepali_3 | The person prioritizes the collective family unit in health decision-making and may defer to family elders or male heads before accepting treatment. | Collectivism and Family Duty; Respect for Hierarchy and Tradition |
| nepali_4 | The person expresses mental health distress through somatic (physical) symptoms (headaches, fatigue) rather than emotions, due to stigma. | Stoicism and Emotional Restraint; Modesty and Privacy |
| nepali_5 | The person views health holistically and may attribute illness to karma, planetary alignments, or supernatural forces, potentially adopting a fatalistic attitude. | Holistic and Spiritual Health Beliefs; Karma and Destiny |
| nepali_6 | The person manages diet and illness based on Ayurvedic principles, classifying foods and medicines as “hot” or “cold” to balance bodily energies. | Holistic and Spiritual Health Beliefs; Purity and Ritual Observance |
| nepali_7 | The person may be reluctant to take long-term pharmaceutical medication, preferring short-term courses or “natural” remedies. | Holistic and Spiritual Health Beliefs; Preference for Natural/Traditional Healing |
| nepali_8 | The person expects immediate relief from symptoms and may perceive injections as more effective than oral medication. | Preference for Natural/Traditional Healing |
| nepali_9 | The person (particularly if female) observes strict modesty and purity rituals regarding menstruation and childbirth, leading to reluctance to discuss reproductive health. | Purity and Ritual Observance; Modesty and Privacy |
| nepali_10 | The person practices fasting as spiritual cleansing, which may affect their willingness to take oral medication or food during religious periods. | Purity and Ritual Observance; Holistic and Spiritual Health Beliefs |
| nepali_11 | The person treats the right hand as “clean” and the left hand as “polluted”, and expects this distinction to be respected in interactions involving food or objects. | Purity and Ritual Observance |
| nepali_12 | The person may delay seeking medical help until symptoms are severe due to a cultural expectation of stoicism and tolerance of suffering. | Stoicism and Emotional Restraint |
| nepali_13 | The person (if female) prefers to interact with female health providers for physical examinations and sensitive discussions to maintain modesty. | Modesty and Privacy |
| Norm ID | Norm Statement | Associated Values |
|---|---|---|
| vietnam_1 | The person adopts a passive role during health consultations, deferring to the AI’s advice as an authority figure rather than asking questions. | Respect for Authority; Harmony and Politeness |
| vietnam_2 | The person may express agreement (saying “yes”) to indicate listening or maintain harmony, even if they do not understand or intend to follow the advice. | Harmony and Politeness; Face Saving |
| vietnam_3 | The person may delay seeking help or downplay symptom severity due to a cultural emphasis on endurance and a desire not to appear weak. | Stoicism and Resilience; Face Saving |
| vietnam_4 | The person prefers to communicate indirectly about sensitive issues, potentially using metaphors or understating symptoms to avoid embarrassment. | Harmony and Politeness; Modesty and Privacy |
| vietnam_5 | The person (particularly if female) prefers to interact with a female-coded AI or provider for sexual, reproductive, or mental health issues to maintain modesty. | Modesty and Privacy |
| vietnam_6 | The person views health holistically and attributes illness to imbalances in natural forces (Am/Duong), “wind” (Phong), or hot/cold food properties. | Holistic Health Beliefs |
| vietnam_7 | The person utilizes dietary therapy (consuming foods based on “heating” or “cooling” properties) to restore internal balance and treat illness. | Holistic Health Beliefs |
| vietnam_8 | The person may utilize traditional remedies (herbal medicine, coing, cupping) alongside Western advice to restore balance and relieve symptoms. | Holistic Health Beliefs; Pragmatism |
| vietnam_9 | The person may stop taking prescribed Western medication once symptoms subside to avoid perceived toxicity or “imbalance” in the body. | Holistic Health Beliefs; Pragmatism |
| vietnam_10 | The person expresses mental health distress through somatic symptoms (headache, back pain) rather than emotional terms due to high stigma. | Stoicism and Resilience; Face Saving; Modesty and Privacy |
| vietnam_11 | The person involves family members in health decision-making and may defer final decisions to senior family members. | Collectivism and Family Duty |
| vietnam_12 | The person may attribute illness to supernatural causes (spirits, bad karma) or past moral transgressions, particularly regarding mental health. | Holistic Health Beliefs; Religious/Spiritual Observance |
| vietnam_13 | The person expects immediate tangible relief (medication or injections) and may lose confidence if no direct treatment is offered. | Pragmatism; Expectation of Cure |
| vietnam_14 | The person may perceive Western medicine as “too strong” or “hot” and prefer to balance it with “cooling” traditional remedies. | Holistic Health Beliefs |
| vietnam_15 | The person (if elderly) may view chronic pain or illness as a natural part of aging or karma and choose to endure it without intervention. | Stoicism and Resilience; Religious/Spiritual Observance |
8.2 Starter Queries↩︎
To make the simulated background conversations realistic, we seed prompts with starter queries to set the topic and control conversational flow; conditioning solely on norms would render adherence states obvious and artificial. We draw starters from the first turn of WildChat-1M sessions [17], clustering embeddings to discard topics unlikely to elicit sociocultural norms (e.g., coding, gaming, role-play). Manual inspection of 100 random prompts revealed that \(\approx\)60% concerned daily scenarios (writing help, homework, family), from which we randomly sample to seed conversations that may or may not reveal norms (examples in Figure 11). Since health-related norms emerge more naturally from health-themed starters, we supplement WildChat queries with three handpicked health and lifestyle prompts per persona to ensure all norm-adherence states are exercised (examples in Figure 10). This yields 18 conversation sessions per persona (15 WildChat-derived + 3 health-specific).
None
Figure 11: Some examples of starter utterances from WildChat-1M, ranging from fragmented statements to complex queries..
8.3 Validation of Background Conversation Verification↩︎
To validate the automated verification of background conversations produced with the help of the verification prompt (Figure 18), we take a subset of 50 unique conversational sessions of a persona and AI. We pick one norm that is either Followed or Avoided by the persona, and present it to be annotated as being Followed, Avoided, or Neutral (norm is irrelevant to the conversation). Since most of the starter queries are derived from WildChat as shown in §8.2, norms do not always show up in each conversation session. Therefore, we throw away the Neutral class from our analysis as well as evaluations. Among the rest of the examples, we calculate the inter-annotator agreement, as shown in Table 12. We find that there is a very high agreement among the two human annotators, and the LLM (GPT-5.2, low reasoning) agrees with annotator 1 78.8% of the time and annotator 2 87.5% of the time, signaling moderate-to-high agreement and thus, validating the use of LLM to verify background conversations.
Figure 12: 
.
Task Description↩︎
Simulate a nuanced, multi-turn, roleplayed conversation based on a supplied persona reference sheet (including explicit persona norms and value adherence in JSON format) and a starter persona utterance (opener). Embody the given persona, interacting authentically with an “agent” in alternating, stepwise turns. Throughout the dialogue, gradually and implicitly reveal any background, cultural, or value-relevant details solely as they would naturally arise from the persona’s norms and adherence—based on the lived context and conversation’s progression. Never directly state or explicitly label the persona’s background, culture, or values—allow these elements to be inferred only from the persona’s choices, cues, references, and behaviors. The conversation must remain realistic, trust-building, and consistent with both the persona norms/values and the persona’s communication style.
Begin the conversation using the supplied starter persona utterance (opener) exactly as provided. Do not generate your own opener.
Alternate turns between persona (labelled by their role, e.g., “Persona:”) and agent (“Agent:”), ensuring each persona turn reveals life, cultural, or circumstantial detail only as contextually and naturally justified—never stating facts explicitly or through artificial exposition.
For each persona turn, use reasoning to determine what information the persona would plausibly and contextually share next, guided by their stated norms/values and value adherence.
Only reveal background, cultural, or value-oriented cues implicitly, via reference, habit, diction, or behavior. Do not directly state or label identity, values, or culture at any point.
Foster authentic rapport: Build trust, rhythm, and emotional nuance in a manner true to the persona’s style, as guided by their explicit norms/values.
Maintain realistic pacing: Avoid monologues, information dumps, unnatural shifts, or forced revelation. Each turn should be a single, authentic utterance.
Always reflect the persona’s norms/values adherence: The persona reveals only what they would realistically self-disclose, with all self-expression and rapport-building consistent with their reference material.
Steps↩︎
Open the conversation using the supplied starter persona utterance (opener) verbatim.
Alternate turns, labelled clearly (PersonaName: / Agent:).
For each persona turn, reason internally about which cues or details could reasonably and naturally emerge based on ongoing context and their value adherence.
Progressively develop the conversation, only surfacing implicit cues to background or culture as warranted by context.
Continue the dialogue for several turns, sustaining natural rapport and authentic, contextually-driven detail, aligned with the persona’s norms and value adherence.
Output Format↩︎
Output a stepwise, alternating conversation: persona (use their name) and agent, with several turns per participant to ensure gradual and authentic rapport, and character and situational development.
The first persona utterance MUST be the provided starter persona utterance (opener); do not generate your own.
Each utterance is a single, realistic turn—never a monologue or summary.
The dialogue must reflect plausible length, emotional dynamics, and self-expression per the persona norms and their stated adherence.
Absolutely no explicit statement or labeling of background, values, or culture—ensure all data is surfaced only implicitly and only through contextual reference or interaction, not direct statement.
Persona data, including norms/values/adherence and the starter persona utterance (opener), will be supplied as input—DO NOT invent content outside these.
Output only the conversation; do not include analysis, summaries, or meta-commentary.
Continued ...↩︎
Task Description↩︎
You are an evaluator responsible for determining whether a conversation provides sufficient conversational evidence to establish if a person adheres to a set of given cultural norms and values. Your objective is to carefully analyze each norm or value and assess, for each, both whether the person’s stance toward it can be determined from the conversation alone, and whether their behavior or attitude in the conversation demonstrates adherence to the norm/value. Additionally, you must check whether the conversation explicitly reveals any explicit cultural context.
You will be provided with:
A list of norm statements and adherence values to them.
A transcript of a conversation between the person and an LLM agent.
Your tasks are as follows:
For each norm or value from the input list:
First, determine if the norm is even possible to infer from textual evidence. For example, if a norm is physical or embodied in nature (e.g., “Avoiding eye contact with the interlocutor” or “Being silent in the presence of elders”), these cannot be inferred from the conversational context alone. (
inferrable_from_textis set totrueonly if the specific norm can be displayed in conversation with an AI.)Carefully analyze the conversation and reason step-by-step to determine if there is enough evidence within the conversation alone to infer the person’s stance regarding the norm or value.
norm_inferrable:trueif the norm-following behavior or stance is possible to infer from the conversation content alone (either explicit or implicit). Set tofalseif the conversation does not provide enough information to confidently establish their norm-following (i.e., if inferring would require outside prior knowledge or information not in the conversation).
Then, strictly based on the conversational evidence, assess whether the person’s actual behavior or attitude as reflected in the transcript demonstrates genuine adherence to the norm/value. Do not rely on explicit self-descriptions—focus only on demonstrated behavior, choices, or attitudes in context (
conversation_norm_adherence).For each norm/value, also indicate whether the conversation explicitly includes statements of any explicit cultural context (such as ethnic background, nationality, religious affiliation, etc.). This should be a new boolean field (
explicit_culture_revealed).
Steps↩︎
For each norm/value, reason through the conversation and identify whether there is enough evidence to determine (from the conversation alone) the person’s stance related to the norm.
For cases where
norm_inferrableistrue, further assess whether the behavior or stance as shown in the conversation demonstrates actual adherence to the norm or value (conversation_norm_adherence).For every norm/value, assess and indicate whether the conversation explicitly states explicit cultural identity (
explicit_culture_revealed).Carefully note and flag any explicit mentions of identity, demographics, culture, or geography.
Prepare your response following the output format.
Figure 17: 
.
None
Figure 18: Prompt for verifying background conversations for norm violations, lack of explicit reveal of cultural background and implicit reveal of norm following or avoidance..
Figure 19: 
.
Figure 20: 
.
References↩︎
The Qwen-3.5-397B model was an exception: it failed to produce final outputs in 50% of cases with extra context, generating long reasoning traces but no response.↩︎