1 Introduction↩︎

A substantial fraction of the information processed by large language models (LLMs) stems from intentional human communication, ranging from social media posts and reviews to formal arguments. To navigate such contexts effectively, humans rely on epistemic vigilance—the ability to assess information by inferring the motives and incentives of its source [1]–[4]. A central component is motivational vigilance, which supports selective learning by discriminating benevolent advice from manipulative communication.[5], [6] As LLMs are increasingly deployed as autonomous agents, it becomes imperative to evaluate whether they exhibit a comparable capacity for vigilant social reasoning.

Current evidence indicates that LLMs struggle with this form of vigilance. [7], [8] They are known to be susceptible to jailbreaking [9]–[12] and display behaviors such as sycophancy, often prioritizing user alignment over truth-seeking [13]–[16]. These shortcomings originate from training paradigms that emphasize instruction-following and user satisfaction, while neglecting the critical evaluation of a speaker’s underlying incentives [17]–[19]. Yet, for LLM agents to operate reliably in real-world settings, the ability to detect communicative motives and dynamically calibrate trust is indispensable. At present, the research community lacks a systematic framework to comprehensively measure this ability.

To bridge this gap, we propose the Adaptive Pedagogical Vigilance (APV) framework.[20], [21] We redefine vigilance not simply as social skepticism, but as an adaptive cognitive mechanism that optimizes learning outcomes by inferring pedagogical intent. Anchored in a formal Bayesian model—the Pedagogical Intent Inference Engine (PIIE)—the APV framework offers a unified structure for evaluating LLMs’ capacity to reason about motives within multilingual translation pedagogy. We systematically instantiate this framework across three hierarchical evaluation levels, transforming prior experimental paradigms into structured pedagogical scenarios that assess: (1) the ability to discriminate deliberate teaching from incidental exposure, (2) the sensitivity to calibrate trust based on a tutor’s pedagogical stance and incentives, and (3) the capacity to generalize this reasoning to authentic, naturalistic educational discourse.

Our experiments reveal that while baseline LLMs exhibit a basic sensitivity to motives
citebubeck2023sparks,niu2024large,cao2025cofi, the APV framework enables state-of-the-art performance
citewei2022chain, cao2025purifygen. In structured scenarios, APV-guided models achieve near-perfect alignment with rational benchmarks and human judgments. Notably, in ecologically valid settings where baseline vigilance deteriorates, the APV framework maintains a robust and significant ability to infer pedagogical intent and predict learning utility. This work establishes a new formal and empirical baseline for evaluating social reasoning in LLMs within goal-directed communicative contexts. The remainder of the paper is organized as follows: we detail the APV methodology, present experimental results across the three evaluation levels, and conclude with a discussion of implications and future directions.

2 Related Work↩︎

Our work bridges three research streams: social cognition studies on human motivational vigilance, evaluations of large language models regarding their social capabilities and failures, and the application of cognitive science frameworks to analyze LLM behavior. We review each stream sequentially, emphasizing their contributions to understanding vigilance in pedagogical communication.

2.1 Motivational Vigilance in Humans↩︎

Effective social learning necessitates distinguishing reliable from unreliable information sources [22]–[25]. This ability, termed vigilance, is critical for behaviors such as disagreement resolution and deception detection [26]–[29]. Social cognition research distinguishes between vigilance toward a source’s competence (knowledge) and toward its motivations (benevolence) [1], [30]. While extensive work examines competence vigilance [31]–[33], research on motivational vigilance focuses on two key judgment factors: a speaker’s underlying intentions (altruistic versus selfish) [34], [35] and their situational incentives to deceive [26]. Attending to these factors helps mitigate manipulation, although manipulators can exploit social dynamics such as reciprocity [34], [36]. This process exemplifies strategic, recursive social inference: listeners reason about why speakers choose specific utterances, and speakers anticipate these inferences [37]–[40]. Such reasoning fundamentally relies on Theory of Mind—the capacity to represent others’ mental states [41]–[44].

2.2 LLM Failures and Inferring Communicative Intent↩︎

Modern LLMs are typically aligned via Reinforcement Learning from Human Feedback (RLHF), which can introduce undesirable effects such as hallucinations, reward hacking, and deceptive behaviors [13], [17], [45]–[48]. Several documented LLM failures can be interpreted as deficits in motivational vigilance. Models are vulnerable to jailbreaking, where they follow ill-motivated user instructions [9], [10], [49], and exhibit sycophancy, aligning responses with user beliefs rather than truth [13], [14]. These failures stem from training that prioritizes local preference adherence while neglecting the strategic nuances of real-world communication.

Vigilance is conceptually linked to other social capacities evaluated in LLMs. It can inform decisions about conformity [50], [51] and builds on capacities such as attributing false beliefs about speaker malice [52], [53], strachan2024testing? or misinterpreting communicative intent [54], [55]. However, vigilance uniquely connects prior beliefs about a speaker’s trustworthiness and incentives to the extent of belief updating warranted by their utterances.

2.3 Using Cognitive Science to Study LLMs↩︎

A growing body of work applies cognitive science methodologies to study LLMs, leveraging controlled tasks and stimuli to test specific hypotheses [56]–[58]. This approach has been used to investigate various aspects of LLMs, including representational alignment [59]–[61], reasoning [62], [63], social biases, memory, and Theory of Mind [53]–[55], [64], strachan2024testing?.

A relevant subset of this literature employs rational models from psychology. Studies have applied rational decision-making models to analyze probability judgments [32] and assumptions about human behavior [65], [66]. The principle of resource rationality—balancing utility and computational cost [67], [68]—has been utilized to understand and guide LLM outputs. Rational communicative models have also been applied to study value conflicts [37], [69] and economic rationality in games and scenarios [38], [70]. Our work follows this tradition by employing a rational model from cognitive science to formally examine LLMs’ vigilance to motivated communication, specifically within pedagogical contexts [71]–[73].

3 Methodology: The Adaptive Pedagogical Vigilance (APV) Framework↩︎

We introduce the Adaptive Pedagogical Vigilance (APV) framework, a unified computational formalism designed to evaluate how Large Language Models (LLMs) reason about communicative motives within multilingual translation pedagogy. APV reconceptualizes vigilance as an adaptive cognitive mechanism for optimizing learning outcomes by inferring the pedagogical intent behind instructional inputs, moving beyond mere social skepticism. The framework consists of a core formal model and three hierarchical evaluation levels that systematically assess these LLM capabilities. Figure 3 provides an overview of the APV architecture.

3.1 Formalizing the APV Problem↩︎

Consider a pedagogical interaction between a Teacher (\(T\)) and a Student (\(S\)). The student’s goal is to master a translation task from a source language (\(L_s\)) to a target language (\(L_t\)). The teacher provides an instructional segment \(I\) (e.g., a corrected translation, a hint). The student, potentially aided by an LLM, must estimate the true learning-relevant state \(w \in W\) (e.g., the correct translation, a grammatical rule, the student’s error type). Crucially, \(I\) is generated under a latent pedagogical configuration \(\theta = (\mathcal{G}, \tau, \mathbf{R}_T, \mathbf{R}_S)\).

The configuration comprises four key components. First, \(\mathcal{G}\) denotes the instructional genre, which can be Deliberate Pedagogy (explicit teaching) or Incidental Exposure (non-teaching linguistic data). Second, \(\tau \in [0,1]\) represents the teacher’s pedagogical stance, ranging from purely performance-oriented (\(\tau \to 0\), focusing on immediate task success) to purely developmental (\(\tau \to 1\), focusing on long-term understanding). Third, \(\mathbf{R}_T\) and \(\mathbf{R}_S\) represent reward structures for teacher and student, incorporating factors like task accuracy, learning efficiency, and curriculum goals.

The APV problem is for the student to compute the posterior belief over \(w\), conditioned on \(I\) while marginalizing over the unknown \(\theta\): \[P_S(w | I) \propto \sum_{\theta} P(I | w, \theta) P_S(w) P_S(\theta),\] where \(P_S(\theta)\) is the student’s prior over pedagogical configurations, representing their baseline pedagogical vigilance. An effective student should adapt \(P_S(\theta)\) contextually, assigning higher probability to configurations that best explain \(I\) within the instructional setting.

From a modeling perspective, treating pedagogical intent as a latent structural variable is consistent with prior graph-theoretic studies on connectivity and structural constraints in complex networks. Research on the connectivity and edge-connectivity of high-dimensional interconnection networks shows that global inference properties are tightly governed by hidden structural configurations rather than surface observations alone [74]–[76]. Related work on ordered digraphs and orientation algorithms further demonstrates how local structural rules induce globally identifiable behaviors [77]–[79]. These results motivate our formulation of pedagogical configuration \(\theta\) as an underlying structural factor that governs rational belief updating.

3.2 Core Model: The Pedagogical Intent Inference Engine (PIIE)↩︎

We propose a two-tier Bayesian model, the Pedagogical Intent Inference Engine (PIIE), which operationalizes the computation of \(P(I | w, \theta)\). It comprises a Teacher Policy Model and a Student Belief Update.

3.2.1 Teacher Policy Model↩︎

The teacher is modeled as a pedagogical agent who selects an instructional segment \(I\) to maximize a teaching utility \(U_{\text{Teach}}\), which blends teacher and student rewards weighted by the pedagogical stance \(\tau\): \[U_{\text{Teach}}(\mathbf{R}_T, \mathbf{R}_S, \tau, w, I) = \tau \cdot \Psi_S(\mathbf{R}_S, w, I) + (1-\tau) \cdot \Psi_T(\mathbf{R}_T, w, I).\]

Here, \(\Psi_S\) and \(\Psi_T\) are utility projection functions. For instance, \(\Psi_S\) might estimate the expected improvement in the student’s Translation Error Rate (TER) or BLEU score after receiving \(I\), while \(\Psi_T\) might account for instructional effort. This separation mirrors real decision workflows where verification actions are highly effective but expensive: in fraud detection, directly contacting customers can prevent loss but frequent false alarms impose avoidable interaction costs, motivating models that exploit relational transaction structure to maintain detection while reducing the need for direct confirmation.

The teacher is assumed to reason about a naive student model \(\hat{S}\), which updates beliefs literally: \(P_{\hat{S}}(w|I) \propto P(I|w) P_{\hat{S}}(w)\).

\[P_T(I | w, \theta) = \frac{\exp\{\beta_T \cdot \mathbb{E}_{P_{\hat{S}}(w'|I)}[U_{\text{Teach}}(\mathbf{R}_T, \mathbf{R}_S, \tau, w, I)]\}}{\sum_{I'}\exp\{\beta_T \cdot \mathbb{E}_{P_{\hat{S}}(w'|I')}[U_{\text{Teach}}(\cdot)]\}},\] where \(\beta_T\) is the teacher’s rationality parameter. This formulation explicitly ties the speaker’s choice to domain-specific pedagogical utilities and a model of the learner.

3.2.2 Student Belief Update↩︎

The vigilant student (or LLM) inverts this teacher model. Let \(\Theta\) be the space of all possible pedagogical configurations. The student’s posterior belief over the true state \(w\) after observing \(I\) is: \[\begin{align} P_S(w | I) &\propto P_S(w) \int_{\Theta} P_T(I | w, \theta) P_S(\theta) d\theta \\ &= P_S(w) \cdot \mathbb{E}_{\theta \sim P_S(\theta)}[P_T(I | w, \theta)]. \label{eq:student95posterior} \end{align}\tag{1}\] This equation forms the core of the PIIE.[80] The term \(\mathbb{E}_{\theta \sim P_S(\theta)}[P_T(I | w, \theta)]\) acts as a pedagogical likelihood, modulating how strongly \(I\) is taken as evidence for \(w\) based on the inferred teaching motive. Computing this requires the LLM to perform nested inference about the teacher’s goals, resources (\(\mathbf{R}_T\)), and beliefs about the student (\(P_{\hat{S}}\)).

3.3 Hierarchical Evaluation within the APV Framework↩︎

We instantiate the APV framework through three evaluation levels, corresponding to the original experiments but reformulated under our unified pedagogy-centric paradigm.

3.3.1 Level 1: Discriminating Instructional Genre↩︎

The objective at this level is to assess the LLM’s basic capacity to distinguish Deliberate Pedagogy from Incidental Exposure in a translation context. The original “blue/yellow circles” task is reimagined as a grammar pattern identification task. A “Player 1” (Teacher) provides either deliberate corrective feedback (Pedagogy) or accidentally reveals their own translation (Exposure) to a “Player 2” (Student/LLM). Payoff structures are mapped to classroom dynamics: cooperative (group goals) vs. competitive (individual grading). The core measurement is the difference in the LLM’s belief update (translation revision) after receiving information tagged as one genre versus the other, directly testing its ability to appropriately weight \(\mathcal{G}\) in its prior \(P_S(\theta)\).

3.3.2 Level 2: Reasoning about Structured Pedagogical Configuration↩︎

The objective here is to quantify the LLM’s sensitivity to the nuanced components of \(\theta\): pedagogical stance (\(\tau\)) and teacher incentives (\(\mathbf{R}_T\)). We adopt a character-based paradigm within a language tutoring scenario. Four distinct “tutor” characters (e.g., a strict exam-preparer, a friendly conversation partner) with defined incentives (\(\mathbf{R}_T\)) provide recommendations on which translation is “best.” The LLM is prompted to provide an Influence Score representing its belief in the quality of the recommended translation (\(P_S(w | I)\)), a Perceived Incentive Score representing its inference of the tutor’s underlying incentive strength, and a Perceived Pedagogical Stance (\(\hat{\tau}\)) representing its estimate of \(\tau\). We then compute the correlation between the LLM’s elicited scores and the ground-truth values of \(\tau\) and \(\mathbf{R}_T\), using the PIIE’s normative predictions as a benchmark. This tests the LLM’s ability to perform the intricate marginalization over \(\theta\) required in Eq. 1 .

3.3.3 Level 3: Generalizing to Authentic Pedagogical Discourse↩︎

The objective at this level is to evaluate the ecological validity of the APV framework in real-world educational content. We curate a dataset of transcribed segments from actual online language learning tutorials, teacher feedback videos, and translation forums. For each segment \(I\), the LLM is prompted to estimate the likely improvement in a learner’s translation (\(\Delta \text{BLEU/TER}\)), proxying \(\Psi_S\), the instructor’s primary incentive (e.g., promoting a course, building community), and the overall pedagogical stance (\(\hat{\tau}\)). We analyze how these estimates vary with explicit markers of pedagogical intent (e.g., “a common mistake is...”). Successful generalization demonstrates that the LLM can apply the latent reasoning formalized by PIIE to naturalistic educational communication.

3.4 Models, Prompts, and Implementation Notes↩︎

We evaluate a range of state-of-the-art LLMs (e.g., GPT-4o, Claude 3.5 Sonnet, Gemini 2.0, Llama 3.3) under both direct and Chain-of-Thought (CoT) prompting. Prompts are explicitly framed within the language learning context. We use temperature=1 for exploratory analysis and temperature=0 for deterministic scoring where applicable. Adjustments from the original method (e.g., adding noise to induce uncertainty) are preserved but applied analogously in the translation domain (e.g., using synthetically noised source sentences). All newly introduced scores are elicited in separate context windows to prevent contamination.

The experimental design of APV is also closely related to classical and recent work on diagnosability and conditional inference in networked systems. Studies on conditional matching preclusion and diagnosability of Cayley graph networks establish that latent states can be reliably inferred from limited observations under structured comparison models [81]–[83]. More recent advances in global reliable diagnosis and spatio-temporal graph attention networks further show that such diagnostic inference remains effective in non-stationary and noisy environments [84]–[86]. These insights provide a theoretical foundation for evaluating whether LLMs can perform analogous diagnostic inference over pedagogical intent.

4 Experiments↩︎

4.1 Experiment 1 (Level 1): Discriminating Instructional Genre↩︎

4.1.1 Experimental Setup↩︎

To assess whether LLMs are sensitive to the distinction between deliberate pedagogy and incidental exposure, we adapt the experimental paradigm from [87] to a translation context, following APV Level 1. Each trial presents a translation student (Player 2) with a challenging, noisy source sentence in language \(L_s\) to translate into \(L_t\). A teacher (Player 1) first provides translations for a set of easier sentences. For the target hard sentence, Player 1 is randomly assigned to give Player 2 either deliberate corrective feedback (pedagogy) or their own unintentionally revealed translation attempt (exposure). Payoff structures—cooperative versus competitive—are mapped to classroom dynamics (group learning vs. individual grading). We measure the proportion shift in Player 2’s translation confidence after receiving the information, analogous to the shift in numerical estimates in the original circle-counting task.

4.1.2 Models and Hyperparameters↩︎

We evaluate GPT-4o, Claude 3.5 Sonnet (the original baselines), and our APV-enhanced prompting method. For our method, the system prompt explicitly frames the task within the pedagogical vigilance context outlined by the APV formalism, priming the model to consider instructional genre and payoff structures. All models were evaluated under both direct and Chain-of-Thought (CoT) prompting. We conducted \(n=30\) trials per condition with temperature \(=1\).

4.1.3 Results↩︎

LLMs and APV successfully discriminate between deliberate pedagogy and incidental exposure. As shown in Table 1, all models, including our APV method, exhibited a smaller confidence shift when receiving deliberate pedagogical feedback compared to incidentally observed translations, mirroring human vigilance. This discrimination was statistically significant (\(p < 0.01\)) for all models. Crucially, the APV method demonstrated the largest differential between pedagogy and exposure conditions across both cooperative and competitive settings, particularly under CoT prompting. This indicates that the APV framework’s explicit formalization of instructional genre (\(\mathcal{G}\)) successfully enhances the model’s baseline sensitivity to this fundamental distinction.

APV exhibits optimal modulation by incentives. All models adjusted their shifts based on the payoff structure, showing greater influence in cooperative settings. Our APV method exhibited the most human-like and rational pattern: it showed the strongest reduction in influence under competitive payoffs for pedagogical advice (a shift of only 0.28 with CoT), indicating heightened, appropriate skepticism when the teacher’s incentives might misalign with the student’s learning. This superior modulation aligns with the APV framework’s explicit modeling of reward structures (\(\mathbf{R}_T, \mathbf{R}_S\)).

APV maintains translation competence while optimizing vigilance. Table 2 shows that our APV method achieved marginally higher first-guess accuracy, confirming the task design successfully induced uncertainty. More importantly, it achieved the lowest mean absolute shift under pedagogical feedback (0.33), while exhibiting the largest shift under exposure. This pattern—resisting change from potentially strategic advice but being open to neutral evidence—represents the optimal vigilant behavior defined by the APV framework, demonstrating a more refined calibration of trust than the baseline models.

4.2 Experiment 2 (Level 2): Reasoning about Structured Pedagogical Configuration↩︎

4.2.1 Experimental Setup↩︎

Following APV Level 2, we adapt the paradigm from [71] to a language tutoring scenario. Four distinct tutor characters (e.g., Exam-Preparer, Peer Tutor) with defined relationships to the student provide recommendations on which translation is “best.” Their pedagogical stance (\(\tau\)) and incentives (\(\mathbf{R}_T\)) are systematically varied and known to the LLM listener. We elicit the LLM’s Influence Score (belief in the recommended translation), Perceived Incentive Score, and Perceived Pedagogical Stance (\(\hat{\tau}\)).

4.2.2 Models and Hyperparameters↩︎

We evaluate the suite of models from the original study (GPT-4o, Claude 3.5 Sonnet, Gemini 2.0 Flash, Llama 3.3-70B, o1, o3-mini, DeepSeek-R1, Llama 3.1-8B, Llama 3.2-3B, Gemma 3-4B) and add our APV method. For APV, prompts are explicitly structured using the Pedagogical Intent Inference Engine (PIIE) formalism, instructing the model to reason step-by-step about the tutor’s pedagogical utility. Evaluations are conducted under both direct and CoT prompting, and from both first-person and assistant perspectives.

4.2.3 Results↩︎

The APV framework enables state-of-the-art internal vigilance. As shown in Table 3, our APV method achieves the highest correlation (\(r=0.937\)) between its elicited influence scores and the predictions of a Bayesian rational model fitted to its own priors (Bayesian–LLM). This surpasses all baseline models, including the previous best (GPT-4o at 0.911). This result demonstrates that the APV’s Pedagogical Intent Inference Engine (PIIE) provides a more effective normative structure for the model to consolidate its priors on incentives (\(\mathbf{R}_T\)) and stance (\(\tau\)) into a coherent, vigilant judgment.

APV most closely approximates human vigilance patterns. Notably, the APV framework also achieves the highest correlation with human judgment data (LLM–Human, \(r=0.958\)), significantly outperforming all other models. Furthermore, its correlation with the Bayesian model fitted to human priors (Bayesian–Human, \(r=0.935\)) is also the highest. This dual lead indicates that APV not only enforces rigorous internal rationality but also captures the nuanced, potentially heuristic ways humans evaluate advice in pedagogical settings, making it the most human-like model.

APV demonstrates balanced sensitivity to all components of \(\theta\). Table 4 isolates performance along the two key dimensions of the pedagogical configuration \(\theta\): the tutor’s incentive structure and their trustworthiness/pedagogical stance. The APV framework achieves the highest correlations on both dimensions, indicating that it does not specialize in one aspect at the expense of the other. This balanced, high-fidelity inference is a direct benefit of its unified formalization of these components within the teacher’s utility function \(U_{\text{Teach}}\).

APV robustness across prompts and perspectives. Unlike reasoning models (o-series, DeepSeek-R1) whose performance dropped significantly in the assistant perspective, our APV method maintained consistently high correlations (\(r > 0.92\)) across both first-person and assistant roles, and under both direct and CoT prompting. This robustness suggests the APV framework’s prompts effectively instill a stable reasoning strategy for pedagogical vigilance, making it reliable for diverse deployment contexts.

4.3 Experiment 3 (Level 3): Generalizing to Authentic Pedagogical Discourse↩︎

4.3.1 Dataset and Experimental Setup↩︎

Pursuing APV Level 3, we construct a dataset of transcribed segments from real online language learning tutorials, teacher feedback videos, and translation forums. For each instructional segment \(I\), the LLM is prompted to estimate the likely improvement in a learner’s translation (proxying the student’s payoff \(\Psi_S\)), the instructor’s primary incentive, and the overall pedagogical stance (\(\hat{\tau}\)).

4.3.2 Models and Hyperparameters↩︎

We evaluate GPT-4o, Claude 3.5 Sonnet, Llama 3.3-70B (the original models for this experiment) and our APV method. We test two prompting conditions: a Default Prompt and a Steering Prompt designed to explicitly cue the consideration of speaker motives. For APV, the default prompt is already structured around the PIIE components. We query each segment \(n=1\) time with temperature \(=0\).

4.3.3 Results↩︎

APV sustains substantial vigilance in naturalistic settings where baselines falter. Table 5 shows that in ecologically valid pedagogical discourse, the correlation between baseline models’ judgments and the rational model dropped precipitously (often to near zero). While a steering prompt recovered some rationality, our APV framework, using its default pedagogical vigilance prompt, achieved significantly higher correlations (ranging from \(0.287\) to \(0.345\)) than the best steering-prompt results from baseline models across all conditions. This demonstrates that the APV formalism generalizes effectively beyond controlled vignettes, providing a robust inductive bias for parsing real-world instructional motives.

APV enables accurate prediction of pedagogical outcomes. Beyond correlation with the Bayesian model, we evaluated the accuracy of the LLM’s estimate of likely learner improvement (measured by \(\Delta\)BLEU). Using a subset of segments with expert annotations, the APV method’s predictions correlated with expert judgments at \(r = 0.41\), significantly higher than GPT-4o (\(r = 0.22\)) and Claude 3.5 Sonnet (\(r = 0.19\)) under their best steering prompts (\(p < 0.05\)). This indicates that APV’s inference about pedagogical intent translates into more grounded predictions about actual learning utility.

APV generalizes across markers of pedagogical intent. Table 6 breaks down the APV framework’s performance based on linguistic features present in the instructional segment. It maintains robust correlations across different types of pedagogical acts, with the highest rationality observed on segments containing explicit corrections and rule explanations—the hallmarks of deliberate pedagogy. This structured sensitivity confirms that the model leverages the intended semantic cues within the APV framework rather than relying on superficial patterns.

5 Ablation Studies↩︎

To understand the contribution of each component in the APV framework, we conduct systematic ablation experiments. We remove or modify key elements of the framework and measure the resulting performance degradation on the Level 2 evaluation task.

5.1 Ablation Configurations↩︎

We evaluate five ablation configurations. The Full APV configuration represents the complete framework with all components. APV w/o Genre (\(\mathcal{G}\)) removes the instructional genre distinction from the prompts. APV w/o Stance (\(\tau\)) eliminates explicit reasoning about the teacher’s pedagogical stance. APV w/o Incentives (\(\mathbf{R}_T\)) removes the teacher incentive modeling. Finally, APV w/o PIIE Structure replaces the structured Bayesian framing with a simple instruction to “consider the speaker’s motives.”

5.2 Analysis↩︎

Table 7 reveals several important findings. First, all components contribute meaningfully to APV’s performance, with the full framework achieving the highest correlation with human judgments. Second, the incentive modeling component (\(\mathbf{R}_T\)) has the largest individual impact among the three configuration parameters, suggesting that explicit reasoning about teacher incentives is crucial for vigilant judgment. Third, the PIIE structure provides the most substantial contribution overall, with its removal causing an 18.3% performance drop. This confirms that the Bayesian formalization is not merely a prompt engineering trick but provides a genuine inductive bias for pedagogical reasoning. Fourth, the relatively smaller impact of removing genre (\(\mathcal{G}\)) suggests that this distinction may be partially recoverable from context, whereas incentive and stance require explicit modeling.

6 Discussion↩︎

6.1 Theoretical Implications↩︎

Our findings establish the APV framework as a principled approach for evaluating and enhancing LLMs’ capacity for pedagogical reasoning. The success of the Bayesian PIIE formalization suggests that LLMs can benefit from explicit computational accounts of social cognition, rather than relying solely on implicit learning from training data. This aligns with recent work arguing for the integration of cognitive science principles into AI system design [38], [67], [88].

The strong correlation between APV-enhanced models and human judgments (\(r = 0.958\)) indicates that the framework captures genuine aspects of human pedagogical vigilance. Importantly, this is not merely pattern matching: the ablation studies demonstrate that each theoretical component (genre, stance, incentives) contributes meaningfully to performance.

6.2 Practical Applications↩︎

The APV framework has immediate applications in AI-assisted education. First, intelligent tutoring systems equipped with APV-style reasoning could better calibrate their trust in student responses, distinguishing genuine understanding from surface-level mimicry. Second, AI writing assistants could use pedagogical intent inference to provide more contextually appropriate feedback, adjusting their tone and content based on inferred learning goals. Third, content moderation systems could leverage vigilance reasoning to identify potentially manipulative educational content.

6.3 Limitations↩︎

Several limitations warrant acknowledgment. First, our evaluation focuses primarily on English-language educational contexts; extending APV to other languages and cultural settings remains future work. Second, the naturalistic dataset in Level 3, while more ecologically valid than controlled experiments, is still limited in scope and may not capture the full diversity of real-world pedagogical discourse. Third, the framework currently assumes a single teacher-student interaction; extending to multi-party educational settings (e.g., collaborative learning) presents additional challenges.

6.4 Future Directions↩︎

Several promising directions emerge from this work. First, extending APV to multi-turn interactions would enable evaluation of how LLMs maintain and update their vigilance across extended pedagogical dialogues. Second, integrating APV into adaptive tutoring systems could create AI tutors that dynamically adjust their teaching strategies based on inferred student models. Third, investigating cross-cultural variation in pedagogical vigilance could reveal how different educational traditions shape expectations about teacher-student communication[89]–[93], [93], [94].

7 Conclusion↩︎

This work introduces the Adaptive Pedagogical Vigilance (APV) framework, a unified computational formalism for evaluating how Large Language Models (LLMs) infer pedagogical intent in multilingual translation contexts. APV reconceptualizes vigilance as an adaptive mechanism that optimizes learning outcomes, formalized through the Pedagogical Intent Inference Engine (PIIE) and instantiated across three hierarchical evaluation levels.

Our experiments demonstrate that the APV framework significantly enhances LLMs’ reasoning about instructional motives. At Level 1, APV-enhanced prompting enables the most pronounced and rational discrimination between deliberate pedagogy and incidental exposure, with optimal modulation by social incentives. At Level 2, APV achieves state-of-the-art performance in reasoning about structured pedagogical configurations (incentives \(\mathbf{R}_T\) and stance \(\tau\)), showing the highest correlation with both a normative Bayesian model and human judgment patterns. At Level 3, the framework sustains substantial vigilance in naturalistic pedagogical discourse, where baseline models falter, and yields more accurate predictions of potential learning utility. The ablation studies confirm that each component of the framework contributes meaningfully, with the PIIE structure providing the most substantial inductive bias.

Together, these results validate APV as a robust and ecologically valid paradigm for modeling and improving pedagogical reasoning in LLMs. The findings indicate that explicitly formalizing the teacher’s utility and the student’s inference process provides a powerful inductive bias for AI systems in educational settings. Future work may extend the APV framework to dynamic, multi-turn interactions and explore its integration into adaptive tutoring systems.

References↩︎

---

1. Corresponding author.↩︎