Keywords: theory of mind \(|\) mentalizing \(|\) mental model \(|\) world model \(|\) belief inference \(|\) bounded rationality

Funding: Research was sponsored by the Army Research Office and was accomplished under Cooperative Agreement Number W911NF-25-2-0040. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Office or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.

1 Introduction↩︎

Inferring the mental states of others, or mentalizing, is fundamental to social life. Contemporary psychology recognizes this ability as having a theory of mind (ToM), a construct that has its modern foundations in Premack and Woodruff’s seminal work on chimpanzees’ ability to reason about each other’s mental states [1]. Importantly, the ToM construct addresses more than just the wants and intentions of others. The canonical empirical test of ToM, for example, is focused on reasoning about beliefs and why people hold them [2]. The crux of inferring beliefs, or any similar mental state, is an epistemic provenance problem: If Sam sees a bag labeled “chocolate” and has no further information, she will conclude it contains chocolate; if before she saw the label a friend said, “that bag contains popcorn,” Sam would likely believe it contained popcorn [3]. In either case, having ToM for Sam’s belief about the bag requires answering questions related to what Sam was exposed to, in what order, from whom, and how credibly. In other words, the ToM problem is one of epistemic state inference based on information access history. The Theory of Mind Utility (ToM-U) is a social reasoning theory presented as a formal framework describing how minds solve this problem.

The ToM-U approach differs significantly from past formalizations. The popular Bayesian Theory of Mind (BToM) approach formalizes goal and belief inference as rational inverse planning, typically over Markov decision processes [4], [5]. BToM, unlike the ToM-U approach, does not derive belief states from information access history and source credibility. BToM instead presupposes belief states as inputs into the formalization. Moreover, BToM handles false beliefs by manipulating perceptual access, whereas the ToM-U approach addresses the more general problem of belief formation based on information exposure, testimony, trust, et cetera. BToM also relies on planning models as the representational object, whereas ToM-U implements an epistemic graph. Similarly, adjacent computer science research relies on action (behavior) and utility as the representational foundation of the formalization rather than the more general problem of how beliefs are formed given epistemic states [6], [7].

ToM-U also differs in important ways from existing theoretical accounts of how humans accomplish mentalizing. In Goldman’s simulationism, the self is the generative engine: we understand others’ mental states by simulating them in our own minds, rather than using a theoretical, rule-based approach [8], [9]. This yields candidate mental states constrained by the perspective of the person doing the reasoning. In contrast, ToM-U generates candidates that are constrained by the information access history of the target of mentalizing. The popular alternative to simulationism, theory-theory, argues that people understand minds by forming “naïve” or “folk” theories, much like scientists form hypotheses [10], [11]. Although theory-theory posits tacit rules about agent behavior, it does not formalize the representational substrate for epistemic inference. In either case, the theories have not produced a formal apparatus operating over the epistemic state inference problem, something that ToM-U accomplishes. More generally, there is not an existing formal mechanism that takes ordered information access history, source credibility, and observability constraints as inputs and produces a belief state estimate as output like ToM-U.

The Theory of Mind Utility is formalized at the computational level of analysis [12]. As such, ToM-U aims to define the overall function, inputs, outputs, and rationale behind the transformation of information during mentalizing while being independent from questions related to the how (algorithmic) and where (neural) aspects of mentalizing. ToM-U relies on the generation of Local Epistemic World Models (LEWMs) to achieve mentalizing. Conceptually, LEWMs are similar to mental models in that they are a representations constructed on the fly to support specific inferential tasks [13]. They differ, however, in their formalism. A LEWM is a directed typed graph constructed on the fly, evaluated for coherence against observed behavior, and accepted or rejected as a working account of another agent’s epistemic situation. This construction, evaluation, and judgment process combined with a means of handling lingering effects of past LEWMs constitutes ToM-U.

The foundational theoretical positions of ToM-U that give rise to its formal structure include:

• Belief states are inferred from ordered exposure history and source credibility, where the sequence and provenance of information access, not merely its content, determines the epistemic state attributed to the target.

• Inference proceeds by generating and evaluating discrete candidate world models. ¹

• Recursive mentalizing is represented as a bounded branching tree in which depth is limited by the sophistication of the mentalizing agent [15] and the focal agent at each depth is treated as a distinct constructed entity rather than as the focal agent with adjusted inputs, preventing infinite regress without requiring additional theoretical commitments.

• Failed mentalizing attempts leave a structured trace that impacts future inference (a “residue”).

These architectural commitments provide foundational, hypothesis-generating bounds to ToM-U. They also ensure the utility is a general mentalizing mechanism, meaning that ToM-U is domain agnostic and not tied to a specific social context (e.g., conflict, cooperation, coordination, etc.) A further theoretical consequence of these commitments and the fact that epistemic states are the output of ToM-U is that it can serve as an input to downstream processes. For example, belief states output from it could feed into a BToM-style goal inference process.

Important contributions of the Theory of Mind Utility are:

1. Formally specifying the epistemic state inference problem that is inherent to mentalizing such that it lends itself to empirical validation.

2. Introducing local epistemic world models as typed graphs that capture ordered information access history as their representational substrate.

3. A generate-and-filter inference architecture with bounded sophistication and explicit confidence accumulation.

4. The residue function as a mechanism for structurally informative mentalizing failure.

Critically, the Theory of Mind Utility not only generates predictions, it generates directional, falsifiable predictions about ToM failure modes, examples of which are in the discussion.

Together, these contributions offer the first formal specification of epistemic state inference as a distinct cognitive problem, one that existing accounts of mentalizing presuppose but do not model, and from which directional, falsifiable predictions about the conditions and character of ToM failure follow directly from the architecture rather than from auxiliary assumptions.

2 The Theory of Mind Utility↩︎

The Theory of Mind Utility, ToM-U, is a domain-agnostic mentalizing mechanism. It is engaged whenever a parent process invokes theory of mind. Its sole function is constructing candidate models, which I call local epistemic world models, in service of belief-like state inference. Other functions, such as action prediction, planning, and behavioral responses, are explicitly downstream consumers of its output, thus outside scope. The core functionality of the mechanism relies on generation of truncated models and assessing their viability as accounts of reality. Accordingly, ToM-U is specified as five definitions covering world model representation, agent node structure, proliferation mechanism, inference procedures, and residue function.

ToM-U is specified at Marr’s [12] computational level of analysis, meaning it defines the what and why of epistemic state inference without committing to how that computation is implemented (the algorithmic level) or where it is realized neurally (the physical level). ToM-U describes the function, its inputs and outputs, as well as the underpinning motivation. In other words, ToM-U provides the structure of the problem from which algorithmic and implementational accounts can be independently developed. More specifically, ToM-U is precise about the representational substrate, the inference architecture, and boundary conditions, but it relies on intentionally coarse specifications for the functional form of candidate generation, fit scoring, and residue decay. The internal mechanisms of the latter components are algorithmic-level questions, and specifying them prematurely would conflate levels of analysis. This deliberate methodological choice ensures the theory remains open to multiple algorithmic realizations.

Readers familiar with Bayesian cognitive modeling will recognize this as analogous to the relationship between a rational analysis and its process-level implementation; the computational specification defines the target the algorithm must hit, not the mechanism by which it hits it. The analogy is limited to the level-of-analysis relationship, however, and should not be read as softening the distinctions between ToM-U and Bayesian approaches drawn elsewhere in this paper.

2.1 Illustrative Example: The Popcorn Bag↩︎

I reference the following scenario, an adapted version of the unexpected contents task [2], [3], [16], throughout this document to illustrate ToM-U in operation. The scenario is used here for two reasons: it is a salient example in the recent literature on ToM-like reasoning in large language models [3], [16], and it is more transparent as an illustration than traditional false belief tasks, where the physical staging of the scenario can introduce confounds irrelevant to the epistemic inference problem ToM-U addresses. ToM-U’s primary contribution is to human ToM theory rather than to LLM evaluation, and this paper accordingly forgoes an extensive treatment of that literature.

A bag contains popcorn. The label on the bag reads “chocolate.” Sam has never seen the bag before and cannot see inside it. Her only available information is the label. A straightforward theory of mind question is: what does Sam believe is in the bag? The correct answer, grounded in Sam’s information access history, is chocolate: the label is her only information source and she has no reason to distrust it.

A slightly more demanding variant introduces a trusted friend who told Sam beforehand that the bag contains popcorn and that the label should be ignored. Sam believes her friend, but she reads the label anyway. The correct theory of mind answer is now popcorn; the trusted testimony overrides the label because it carries higher credibility. Systems that fail this variant, as large language models have been shown to do [3], fixate on the most recent or salient surface signal rather than maintaining a coherent belief state for Sam that properly tracks her full information access history. We argue that the ToM-U approach handles this naturally, as the worked example following the formal definitions demonstrates.

2.2 Motivation: Local Epistemic World Models as Uncertainty Resolution↩︎

Theory of Mind engagement is fundamentally a response to uncertainty about the mental states of others. When an agent cannot explain observed behavior through available situational or analytical information alone, they face an inference problem: what belief states, intentions, or attitudes must the other agent(s) hold for their behavior to be coherent? I refer to the agent whose mentalizing is being modeled as the focal agent and the subject(s) of the focal agent’s mentalizing as the target agent(s). A local epistemic world model (LEWM) is a representation (specifically, a directed typed graph) that is occasioned by a particular inferential need and anchored to the focal agent’s perspective. Its content is restricted to agents, the state nodes representing objects, situations, and conditions relevant to the occasion, and the epistemic relationships among them, including beliefs, informational channels, and credibility judgments. The LEWM is the representational structure through which the ToM inference problem is posed and resolved.

In the abstract, LEWMs might not appear to meaningfully differ from mental models in the tradition of Johnson-Laird [13]. However, LEWMs formalize much of what is only conceptual in the mental modeling account. That formalization facilitates hypothesis testing and, importantly, theory falsification. Adjacent to the mental modeling account is situation modeling [17], [18]. The situation model approach shares much of the conceptual architecture of ToM-U, (i.e., a local representation that is occasion-specific, constructed on the fly, evaluated for coherence, etc.) but, like mental modeling, lacks the formal apparatus introduced by the LEWM approach.

Given its main role in the ToM-U, the term local epistemic world model warrants a more thorough introduction. Working backwards through its parts, “world model” is used in a sense more similar to Johnson-Laird’s [13] mental models, a representation constructed on the fly to support a specific inferential task, than to the reinforcement learning sense of a learned, persistent model of environment dynamics [19]. A LEWM is a hypothesis about a particular epistemic situation, not a general-purpose predictive model of the environment. Use of the term world model rather than mental model serves to signal both the formal precision required by the inference machinery and the functional role the representation plays within the utility. In the ToM-U case, a world model serves as a structured object that is constructed, evaluated, and updated rather than as a theory of human reasoning per se. Mental models in Johnson-Laird’s sense provide the psychological grounding and motivation; the world model formalization provides the representational substrate the utility operates over.

The modifier “epistemic” restricts the representation to knowledge and belief relationships among agents, distinguishing it from world models that encode physical states or causal dynamics. The modifier “local” reflects three properties of the representation: it is occasioned by a specific inferential need rather than maintained persistently, anchored to the focal agent’s perspective, and bounded in content to agents and epistemic relationships relevant to that occasion.

A LEWM is not a description of a physical situation, nor a persistent generative model of the environment. It is a hypothesis about the state of the world, specifically the epistemic state, that is valuated for coherence against observed behavior and accepted or rejected as a viable account of reality. This distinguishes the LEWM approach from purely reactive signal processing: rather than responding to surface features of observed behavior, the focal agent constructs a candidate representation and asks whether that representation makes the behavior intelligible.

Central to the LEWM are the epistemic relationships encoded in its edges, the belief-like states (BLSs). The qualifier “like” signals agnosticism about whether the attributed states meet the full conditions for propositional belief in the philosophical sense, that is, attitudes such as credences, dispositions, and representational states that play a belief-like functional role without necessarily satisfying stronger conditions on intentionality or content. This is consistent with Dennett’s [20] intentional stance, in which belief attribution is a practical interpretive act rather than a metaphysical commitment.

Accepting a LEWM therefore implies accepting its BLSs as valid. They are not decorative annotations; rather, they are the load-bearing elements of the model. A LEWM in which Sam believes the bag contains chocolate is a fundamentally different hypothesis about the state of the world than one in which Sam believes it contains popcorn, even if all other elements are identical. The inference procedures in Definition 4 are therefore not merely estimating BLS values; they are evaluating competing LEWMs as candidate solutions to an uncertainty resolution problem, where the LEWM that best explains observed behavior given available information access histories and observability constraints is accepted as the working account of reality.

This framing has two important implications. First, it establishes why the generate-and-filter mechanism in Definition 4 is the appropriate inference architecture; the focal agent is not optimizing over a continuous distribution of BLS values but selecting among discrete candidate LEWMs. Second, it establishes why rejection of a LEWM is informationally meaningful rather than a null event; a rejected LEWM is a hypothesis about reality that failed, and that failure carries evidence about the reliability of the epistemic edges involved, formalized in Definition 5 as the residue function.

3 Formalization of the Theory of Mind Utility↩︎

The utility is specified as five definitions in the mathematical sense. Each introduces a named, formally bounded structure or procedure that subsequent definitions build on. It is useful to read each definition as a precise answer to the question: what is this component, and what are its parts?

Definition 1 specifies the LEWM, the directed typed graph that serves as the representational substrate for all subsequent operations. Definition 2 specifies the agent node structure, defining the internal properties of each agent represented in the LEWM, including information access history, sophistication, and temporal anchoring. Definition 3 specifies the proliferation mechanism by which the focal agent recursively instantiates projected agent nodes, constructing a bounded branching tree that represents nested mentalizing. Definition 4 specifies the three inference procedures (backward inference, self-projection, and mutual reconciliation) that operate over the proliferated LEWM to produce belief-like state estimates. Definition 5 specifies the residue function, which captures the informational trace left by rejected LEWMs and feeds into downstream reasoning processes. Together, these definitions constitute the theory of mind utility.

3.1 A Note on Notation↩︎

The formalization draws on notational conventions from: graph theory and theoretical computer science, set-theoretic mathematics, and cognitive science. This reflects the interdisciplinary position of the model, which is intended to be psychologically interpretable, formally precise, and computationally tractable.

Structures are defined as tuples using angle brackets \(\langle \cdot \rangle\), following standard practice in graph theory and formal language theory. Sets are denoted with uppercase Roman letters (\(A\), \(O\), \(E\)) and their elements with subscripted lowercase counterparts (\(a_i\), \(o_j\)). Functions are written in the form \(f: X \rightarrow Y\), denoting a mapping from domain \(X\) to codomain \(Y\). The calligraphic typeface \(\mathcal{T}\) denotes a vocabulary or type set. The natural numbers \(\mathbb{N}\) and reals \(\mathbb{R}\) follow standard mathematical convention.

Readers unfamiliar with tuple notation may find it useful to read \(W = \langle A, O, E, \text{BLS}, \text{obs}, \text{cred} \rangle\) as “a local epistemic world model \(W\) is fully described by six components: \(A\), \(O\), \(E\), BLS, obs, and cred.” Readers unfamiliar with the belief-like state framework may find it useful to read BLS-typed edges as typed, weighted directed edges in a knowledge graph, where the type label carries semantic content about the nature of the relationship.

3.2 Preliminary Definitions↩︎

The following quantities appear across multiple definitions and are collected here to avoid forward dependencies. Quantities that appear only within a single definition are introduced inline.

Observed behavior \(\beta_i\) denotes the observable actions, signals, or outputs of target agent \(a_i \in A \setminus \{a_1\}\) that serve as the primary input to inference. In the multi-agent case, each target agent has their own observed behavior \(\beta_i\), and inference procedures are indexed accordingly. In the single-agent case this reduces to a scalar \(\beta\) for the sole target agent. \(\beta_i\) is treated as given; the processes generating it are outside the scope of ToM-U.

Fit score \(f_j \in [0,1]\) is a scalar measure of how well the \(j\)-th candidate BLS configuration accounts for observed behavior \(\beta_i\) of target agent \(a_i\), given that agent’s information access history \(H_i\) and observability constraints on relevant edges. The candidate index \(j\) is distinct from the target agent index \(i\) used elsewhere. \(f_j = 1\) indicates a perfect account; \(f_j = 0\) indicates no explanatory purchase. Its functional form is deliberately underspecified at the computational level, consistent with the treatment of candidate generation.

Confidence \(C_n \in [0,1]\) is a cumulative weight reflecting how well the generate-and-filter procedure has accounted for observed behavior after \(n\) iterations:

\[C_n = 1 - \prod_{j=1}^{n}(1 - f_j)\]

\(C_n\) increases monotonically with each iteration as fit scores accumulate, approaching but never exceeding 1. A single high fit score produces rapid confidence accumulation; consistently low fit scores produce slow accumulation bounded by the sophistication ceiling. The scalar accumulator is consistent with empirical evidence that human judgment relies on sparse sampling rather than full distributional inference [21], [22], supporting its psychological plausibility alongside the bounded rationality framing [15].

Marginal coherence gain \(\Delta C(k)\) is the expected improvement in confidence from proliferating to depth \(k\) versus depth \(k-1\):

\[\Delta C(k) = \mathbb{E}[C \mid W^k] - \mathbb{E}[C \mid W^{k-1}]\]

\(\Delta C(k)\) captures the marginal value of an additional level of mentalizing. When this gain falls below a cost threshold, further proliferation is unproductive relative to its cognitive cost.

Threshold parameters. Two threshold parameters govern stopping conditions in the inference procedures:

• \(\tau_{\text{high}} \in (0,1)\): confidence threshold for early stopping. When \(C_n > \tau_{\text{high}}\), sufficient confidence has been achieved and the procedure halts.

• \(\tau_{\text{low}} \in (0, \tau_{\text{high}})\): implicit lower bound distinguishing accepted from low-confidence output at the sophistication ceiling. Output returned at \(n = S_1\) is treated as low-confidence when \(C_n < \tau_{\text{low}}\).

All threshold parameters are treated as fixed for initial analyses. Empirical calibration across contexts and cultures is a priority for future work. ToM-U assumes a parent process that determines whether and how the utility is invoked via a two-stage process. In the first stage, engagement is triggered when a weighted combination of factors, such as information asymmetry, social setting complexity, and sophistication miscalibration, exceeds a threshold [23]. For example, information asymmetry functions as an enabling cause: without it, ToM engagement has no useful causal work to do, and the utility is not invoked. In the second stage, conditional on engagement, the focal agent evaluates the ToM-U output and decides whether to accept or reject it based on signal strength, sophistication, and complexity. This produces a three-state variable: one indicating that engagement was not triggered; one indicating that the utility ran and its output was accepted as the operative LEWM; and one indicating that the utility ran but its output was rejected. The present specification takes this three-state variable as given and concerns itself entirely with what happens downstream of the triggering decision.

The trichotomy matters for two reasons. First, ran but rejected is the condition under which Definition 5’s residue function is triggered: rejection is not a null event but an informative one that leaves a structured trace shaping future invocations. Second, the accepted and rejected distinction supports recursive application: a LEWM accepted at one level of mentalizing may serve as input to a subsequent invocation at a deeper level, with the sophistication ceiling in Definition 3 providing the hard bound on recursion depth. Rejection is treated as binary in the current specification; partial trust in rejected output is identified as a boundary condition for future work at the algorithmic level.

3.3 Definition 1: Local Epistemic World Model↩︎

A LEWM \(W\) is a directed typed graph defined as a tuple:

\[W = \langle A,\, O,\, E,\, \text{BLS},\, \text{obs},\, \text{cred} \rangle\]

Where:

• \(A\) is a finite set of agent nodes \(\{a_1, a_2, \ldots, a_n\}\), disjoint from \(O\). The disjoint treatment of agent and state nodes is psychologically motivated: neuroimaging evidence suggests that reasoning about intentional agents and reasoning about objects of evaluation rely on dissociable neural systems, specifically the temporoparietal junction and medial prefrontal cortex respectively [24], [25], consistent with maintaining separate representational structures for each.

• \(O\) is a finite set of state nodes \(\{o_1, o_2, \ldots, o_m\}\) representing objects, situations, and conditions relevant to the LEWM, disjoint from \(A\).

• \(E \subseteq (A \cup O) \times (A \cup O)\) is a set of directed typed edges, where at most one edge of each BLS type exists between any ordered node pair.

• \(\text{BLS}: E \rightarrow \mathcal{T} \times [0,1]\) is a function mapping each edge to a belief-like state, where \(\mathcal{T}\) is the open vocabulary of attitude types \(\{\text{belief},\, \text{trust},\, \text{intention},\, \text{desire},\, \ldots\}\) and \([0,1]\) is the scalar value. Since edges are directed, the ordered pairs \((a_x, a_y)\) and \((a_y, a_x)\) are distinct and may independently carry edges of the same BLS type, representing the respective attitudes each agent holds toward the other.

• \(\text{obs}: E \rightarrow [0,1]\) is the observability function mapping each edge to a scalar where \(0\) indicates perfectly random and \(1\) indicates perfectly accurate access.

• \(\text{cred}: E \rightarrow [0,1]\) is the credibility function mapping each edge to a scalar reflecting its accumulated inferential reliability across invocations, updated by the residue function in Definition 5. \(\text{cred}\) is initialized at \(1\) for all edges in the absence of past invocation history, consistent with the parsimony principle of assigning full inferential weight to relationships before any disconfirming evidence has accumulated. \(\text{cred}\) is formally distinct from \(\text{obs}\); the distinction is elaborated in Definition 5.

3.4 Definition 2: Agent Node↩︎

Each agent node \(a_i \in A\) is a tuple:

\[a_i = \langle H_i,\, S_i,\, t_i \rangle\]

Where:

• \(H_i\) is the information access history of \(a_i\), that is, the ordered set of state nodes and epistemic edges \(a_i\) has been exposed to up to the current invocation. The epistemic status of \(H_i\) differs by agent role. For the focal agent \(a_1\), \(H_1\) is directly held. For target agents \(a_2, \ldots, a_n\), \(H_i\) is the focal agent’s reconstruction of their information access history, estimated via the BLS edge structure and observability constraints on relevant edges; it is not directly held but inferred. For projected nodes \(a_{ij}\), \(H\) is a filtered subset of the parent node’s \(H\), constrained by \(\text{obs}\) on the edges between parent and child, and is therefore progressively more impoverished at greater proliferation depths. In all cases, \(H_i\) is treated as given without veridicality constraints. The processes generating \(H_i\), including confabulation and reconstructive memory, are upstream and outside the scope of ToM-U.

• \(S_i \in \mathbb{N}\) is the discrete sophistication parameter of \(a_i\), representing stable reasoning capacity. For the focal agent, \(S_i\) is a directly held parameter. For other agent nodes, \(S_i\) is estimated via incoming BLS-typed edges from the focal agent, making sophistication both an intrinsic node property and a target of relational inference. \(S_i\) maps onto a continuous sophistication parameter via: \[S_{\text{continuous}} = \frac{S_i}{S_{\max}}\] where \(S_{\max}\) is the empirical ceiling, recommended at \(6\) following [26] but is a provisional parameter.

• \(t_i \in \mathbb{R}\) is the snapshot timestamp of \(a_i\), representing the time point at which this agent node’s state is modeled, consistent with the snapshot assumption. \(t_i\) advances with each iterative pass in mutual reconciliation.

Edges in \(H_i\) carry structural significance through their presence as well as their content. An agent who has never been exposed to a relevant state node has no epistemic edge connecting them to it in \(H_i\); this is a structural property of the graph, not a missing value. The ToM-U formalism distinguishes two cases that informal accounts of theory of mind tend to collapse. An absent belief obtains when no edge exists between the target agent and a relevant state node in \(H_i\), meaning the target has simply never been exposed to the relevant information. A false belief obtains when an edge exists but the BLS value it carries is inconsistent with the actual state of the world, consistent with the target’s exposure history [2]. The distinction is consequential for inference: a target agent for whom the relevant edge is absent cannot be attributed knowledge of the corresponding state node, constraining the space of viable candidate world models accordingly. Failures of source monitoring, in which a focal agent misrepresents whether or not a target has been exposed to relevant information, therefore produce systematically miscalibrated LEWMs rather than randomly wrong ones [27].

The dual role of \(S_i\) is resolved by agent position in the proliferated tree. For the focal agent at the root, \(S_i\) is directly held and operative, governing the sophistication ceiling on proliferation and inference. For target agents and projected nodes, including projected instances of the focal agent at depth > 1, \(S_i\) is an external estimate derived from incoming BLS edges. Because projected instances of the focal agent are treated as constructed others rather than as the focal agent with adjusted inputs, a property established formally in Definition 3, the directly held and estimated roles of \(S_i\) never apply to the same node simultaneously.

3.5 Definition 3: Agent Proliferation↩︎

Given a focal agent \(a_1 \in A\) with discrete sophistication parameter \(S_1 \in \mathbb{N}\), the proliferation mechanism instantiates a set of projected agent nodes representing recursive mentalizing as a bounded branching tree rather than explicit recursion. A theoretical property of this architecture worth making explicit is that the focal agent at each depth is treated as a distinct projected entity rather than as the focal agent with adjusted inputs. That is, \(a_{1(1)}\), the projection of \(a_1\) as modeled by \(a_2\), is not \(a_1\) running its own reasoning in a transformed mode; it is a constructed other that \(a_1\) reasons about from the outside, progressively impoverished by observability filtering at each depth. This prevents the infinite regress common in recursive mentalizing accounts, where reasoning about oneself requires a homunculus, and falls directly out of the proliferation architecture rather than requiring an additional theoretical commitment.

The proliferated LEWM \(W^k\) at depth \(k\) is defined as:

\[W^k = \langle A^k,\, O,\, E,\, \text{BLS},\, \text{obs},\, \text{cred} \rangle\]

Where \(A^k\) is organized as a labeled tree \(\mathcal{N} = \langle V, P \rangle\):

• \(V\) is the set of agent nodes with \(a_1\) as root.

• \(P: V \setminus \{a_1\} \rightarrow V\) is the parent function mapping each non-root node to its parent.

• Each node \(v \in V\) is identified by a path index \(v = (i_1, i_2, \ldots, i_k)\) where \(k\) is the depth of \(v\) and each \(i_j \in \{1, \ldots, b(k, S_1)\}\) identifies the branch at depth \(j\).

The agent nodes are constructed as follows:

• \(a_1\): the focal agent, with full \(H_1\), directly held \(S_1\), and snapshot at \(t\).

• \(a_2, \ldots, a_n\): the target agents, one for each non-focal agent in \(A\). Each has \(H_i\) reconstructed by the focal agent via the BLS edge structure and observability constraints, \(S_i\) estimated via incoming BLS edges from \(a_1\), and snapshot at last observed \(t\). At depth 1, the children of \(a_1\) in the proliferated tree are exactly \(\{a_2, \ldots, a_n\}\), determined by the composition of \(A\) rather than by the branching width function. The branching width function \(b(k, S_1)\) governs proliferation at depth \(k > 1\) only.

• \(a_{ij}\): a projected node at depth \(j > 1\) on branch \(i\), representing an agent as modeled by its parent node in the tree, containing only the subset of the parent’s \(H\) accessible given \(\text{obs}\) on relevant edges, and therefore progressively more impoverished at greater depths.

Branching width at depth \(k\) is governed by:

\[b(k, S_1) = \lfloor S_1 \cdot \rho^k \rfloor\]

where \(\rho \in (0,1)\) is a decay parameter governing how rapidly branching width decreases with depth, reflecting empirical findings on mentalizing accuracy degradation with recursion depth. \(\rho\) is treated as fixed for initial analyses. Consistent with the agent node construction above, \(b(k, S_1)\) governs proliferation at depth \(k > 1\) only; depth 1 is populated by all non-focal agents in \(A\).

Stopping Rule. Proliferation halts when either:

\[k = S_1 \quad \text{(sophistication ceiling reached)}\]

\[\text{or} \quad \Delta C(k) < \kappa\] where the marginal coherence gain falls below the proliferation cost threshold, and \(\kappa \in \mathbb{R}^+\) is a fixed cost threshold governing when further mentalizing is unproductive relative to its cognitive cost.

3.6 Definition 4: Inference Procedures↩︎

Tom-U implements a generate-and-filter architecture for inference that operates over discrete candidate LEWMs rather than a continuous distribution over belief states. This commitment is theoretically motivated: mentalizing produces a categorical outcome in which the focal agent either adopts a LEWM as their working account or does not, consistent with Dennett’s [20] intentional stance and with empirical evidence that human judgment relies on sparse sampling rather than full distributional inference [14], [21]. This computational-level commitment is agnostic about whether the underlying implementation is stochastic; its theoretical implications are taken up in 5.1.

Three domain-agnostic inference procedures operate over \(W^k\). Each is a mapping from the LEWM to a belief state estimate, differing in directionality and the agent nodes involved. These procedures carry no inherent assumption about the relationship between agents. Note that the generate-and-filter procedure is a computational level description of the inference problem structure and is therefore agnostic about whether the underlying process is serial, parallel, stochastic, or approximate.

3.6.1 Backward Inference↩︎

Given observed behavior \(\beta_i\) of target agent \(a_i \in A \setminus \{a_1\}\), backward inference recovers a BLS configuration making \(\beta_i\) coherent given \(a_i\)’s information access history and observability constraints, via a generate-and-filter procedure. Backward inference operates over a single target agent at a time; in the multi-agent case it is applied once per target agent in \(A \setminus \{a_1\}\), each with their own observed behavior \(\beta_i\) as input:

\[\text{BI}: \beta_i \times W^k \rightarrow \langle \text{BLS}(a_1, a_i),\, C \rangle\]

Where \(C \in [0,1]\) is the confidence weight on the output, defined as:

\[C_n = 1 - \prod_{j=1}^{n}(1 - f_j)\]

where \(f_j \in [0,1]\) is the fit score of the \(j\)-th candidate BLS configuration evaluated against \(\beta_i\), \(H_i\), and \(\text{obs}\) on relevant edges. The numbered steps describe the computational structure of backward inference rather than constituting an algorithmic specification.

The procedure runs as follows:

1. Generate a candidate BLS configuration consistent with \(H_i\) and \(\text{obs}\)

2. Evaluate fit score \(f_j\) against observed behavior \(\beta_i\)

3. Update confidence \(C_n\)

4. If \(C_n > \tau_{\text{high}}\): accept output, halt

5. If \(n = S_1\): halt, return current best candidate with confidence \(C_n\) as low-confidence output

6. Otherwise: increment \(n\), return to step 1

Two stopping conditions govern the procedure. Early stopping occurs when \(C_n > \tau_{\text{high}}\), i.e., sufficient confidence achieved. The hard bound occurs when \(n = S_1\), i.e., the sophistication ceiling is reached and output is returned regardless of confidence, adopted on bounded rationality grounds following [15].

The output is a tuple \(\langle \text{BLS}(a_1, a_i),\, C \rangle\) rather than a point estimate, preserving uncertainty for downstream consumption.

The functional form of candidate generation in step 1 is deliberately underspecified. Generation is constrained by \(H_i\) and \(\text{obs}\) and is assumed to have access to anything with objective or subjective relevancy to the world model. Its internal mechanism is outside the scope of ToM-U, consistent with specification at the computational level of analysis. Similarly, the functional form of \(f_j\) is underspecified. The theoretical claims encoded in \(C_n\) concern the structure of confidence accumulation (that it is cumulative, monotonically increasing, and bounded) rather than the magnitude of individual fit scores. These properties hold for any \(f_j\) satisfying the \([0,1]\) boundary condition, making the aggregation logic the theoretically meaningful commitment rather than the scoring mechanism itself.

3.6.2 Self-Projection↩︎

Self-projection is a forward inference procedure that recovers how \(a_1\)’s belief states appear from the perspective of a given target agent \(a_i \in A \setminus \{a_1\}\). It operates over the two-node degenerate case of the proliferated tree involving \(a_1\) and its projected node \(a_{1(i)}\), representing \(a_1\) as observable to \(a_i\). In the multi-agent case, self-projection is applied once per target agent, producing a distinct output for each:

\[\text{SP}: W^k \rightarrow \langle \text{BLS}(a_i, a_{1(i)}),\, C \rangle\]

Given \(a_1\)’s full \(H_1\) and the observability constraints on edges between \(a_1\) and \(a_i\), self-projection recovers how \(a_1\)’s belief states appear from \(a_i\)’s perspective. The same generate-and-filter procedure as backward inference applies, with anchoring on \(a_1\)’s own perspective as the starting estimate, consistent with the finding that perspective taking begins from an egocentric anchor and adjusts insufficiently [28]. The projected node \(a_{1(i)}\) is a distinct constructed entity representing \(a_1\) as seen from \(a_i\)’s observational position, not \(a_1\) with adjusted inputs, consistent with the nested-selves-as-others property established in Definition 3.

3.6.3 Mutual Reconciliation↩︎

Mutual reconciliation is a sequential iterative procedure combining backward inference and self-projection to identify a LEWM configuration that makes all agents’ behaviors jointly coherent. It is the primary inference procedure in contexts where information asymmetry and belief miscalibration are the principal drivers of ToM engagement. In the multi-agent case, mutual reconciliation operates over all target agents \(a_i \in A \setminus \{a_1\}\) simultaneously, with backward inference and self-projection passes running over each target agent at every iteration:

\[\text{MR}: \{\beta_i\}_{i=2}^{n} \times W^k \rightarrow \{\langle \text{BLS}(a_1, a_i),\, C_i \rangle,\, \langle \text{BLS}(a_i, a_{1(i)}),\, C_{1(i)} \rangle\}_{i=2}^{n}\]

where \(C_i\) is the confidence on the backward inference direction for target agent \(a_i\), and \(C_{1(i)}\) is the confidence on the self-projection direction as seen from \(a_i\)’s perspective. The numbered steps describe the computational structure of mutual reconciliation rather than constituting an algorithmic specification.

The procedure runs as follows:

1. Initialize: set \(n = 0\), anchor on \(a_1\)’s perspective per Epley et al. [28]

2. Backward inference passes: for each \(a_i \in A \setminus \{a_1\}\), run \(\text{BI}(\beta_i, W^n) \rightarrow \langle \text{BLS}(a_1, a_i),\, C_{in} \rangle\), update LEWM

3. Self-projection passes: for each \(a_i \in A \setminus \{a_1\}\), run \(\text{SP}(W^n) \rightarrow \langle \text{BLS}(a_i, a_{1(i)}),\, C_{1(i)n} \rangle\), update LEWM

4. Update \(W^{n+1}\), advance snapshot timestamps, rebuild or prune proliferated tree as needed

5. Evaluate joint confidence: \[C_{\text{joint}} = \min_{i \in \{2,\ldots,n\}} \min(C_{in},\, C_{1(i)n})\]

6. If \(C_{\text{joint}} > \tau_{\text{high}}\): accept output, halt

7. If \(n = S_1\): halt, return current best configuration with \(C_{\text{joint}}\) as low-confidence output

8. Otherwise: increment \(n\), return to step 2

Two properties of the procedure are worth noting. First, anchoring on \(a_1\)’s perspective in step 1 is consistent with the finding that perspective taking begins from an egocentric anchor and adjusts insufficiently toward the other’s viewpoint [28]. The iterative structure models this adjustment process explicitly, with the \(S_1\) hard bound capturing the bounded rationality constraint on full adjustment. Second, partial alignment is the expected outcome rather than full belief reconciliation, following [29] and [30]. The low- confidence output path captures residual belief discrepancy that persists at termination. In the multi-agent case, partial alignment may hold across some agent pairs but not others at termination, with per-agent confidence weights preserving that structure in the output.

The joint confidence formulation \(C_{\text{joint}} = \min_{i} \min(C_{in}, C_{1(i)n})\) reflects that joint coherence across all agents is bounded by the weakest inference direction among all agent pairs. This is conservative but defensible: a LEWM that fails to account for any single agent’s behavior or appearance is not jointly coherent. The min formulation reflects a specific theoretical commitment: joint coherence across agents is only as strong as its weakest component. That commitment means that partial success across some agent pairs cannot compensate for failure in others. An alternative weighted average formulation would embody a different commitment, that coherence is distributed across agent pairs and partial alignment is aggregable. These are empirically distinguishable positions that generate different predictions about mentalizing success in asymmetric multi-agent contexts; the theoretical and empirical consequences of this choice are taken up in 5.1.

3.7 Definition 5: Residue↩︎

When the invoking model returns a rejection, the inference procedures in Definition 4 have run but their output has not been accepted as the operative LEWM. The failure is not informationally inert. A rejected LEWM carries evidence about the epistemic relationships that were relevant to its failure, and that evidence should influence how future LEWMs are constructed. The residue function formalizes this: it is the mechanism by which failed mentalizing attempts leave a persistent trace that shapes subsequent invocations of the ToM-U.

The distinction between \(\text{obs}\) and \(\text{cred}\) introduced in Definition 1 is motivated here. Observability encodes the focal agent’s epistemic access to a relationship at a given moment: how clearly the relationship can be read at invocation time. Credibility encodes the accumulated inferential reliability of that relationship across invocations: how reliably it has supported coherent LEWMs in the past. These can come apart in both directions. A clearly observable relationship may carry low credibility if it has repeatedly failed to support coherent accounts of observed behavior. A partially observable relationship may carry high credibility if it has consistently done so when invoked. The residue function updates \(\text{cred}\) rather than \(\text{obs}\) because it is the track record of a relationship, not its current accessibility, that failed mentalizing attempts are informative about.

The residue updates the inferential weight carried by specific epistemic relationships in the LEWM, specifically those whose contribution was relevant to the failed account. Relationships that played no relevant role in the failure are unaffected. This targeting is logically motivated: a failed LEWM is informative specifically about the relationships that mattered to its construction, not about the epistemic structure as a whole.

Not all failures carry equal information. A LEWM that was strongly disconfirmed—one whose candidate accounts consistently failed to make observed behavior coherent—provides stronger evidence of unreliability in the relevant epistemic relationships than one that simply exhausted the sophistication ceiling before achieving sufficient confidence. The residue is therefore weighted by degree of disconfirmation, with capacity-exhausted failures producing a weaker trace than incoherence-driven failures.

The trace left by a failed LEWM persists across subsequent invocations and influences them in a specific way: epistemic relationships that have accumulated residue carry reduced inferential weight when future LEWMs are constructed, making candidate accounts that depend heavily on those relationships less likely to achieve sufficient confidence. This does not exclude those relationships from future models but it shapes which candidates are viable. The trace diminishes over time, never fully disappearing. New information bearing on a previously failed epistemic relationship interacts with accumulated residue, and a fresh strongly supported signal can partially rehabilitate inferential weight, but the history of failure persists at a reduced level.

The formal signature of the residue function is:

\[R: E \times \mathbb{R} \rightarrow [0,1]\]

where \(R(e, t)\) maps an edge \(e \in E\) and a time \(t \in \mathbb{R}\) to a scalar reflecting the accumulated residue on that edge up to time \(t\), which in turn determines the reduction in \(\text{cred}(e)\) for future invocations. The function \(R\) is required to satisfy the following boundary conditions:

• Targeting: \(R(e, t) > 0\) only for edges whose contribution was relevant to at least one rejected LEWM before time \(t\). Edges that played no relevant role in any past rejection carry zero residue.

• Monotonicity in disconfirmation: \(R(e, t)\) is non-decreasing in the degree of disconfirmation of LEWMs implicating edge \(e\). Strongly disconfirmed failures contribute more residue than capacity-exhausted failures.

• Monotonicity in time: \(R(e, t)\) is non-increasing in elapsed time since the most recent rejection event on edge \(e\). The trace diminishes over time.

• Asymptotic lower bound: The credibility \(\text{cred}(e)\) remains strictly greater than zero for all \(e\) and all finite sequences of rejection events. No finite failure history eliminates the inferential weight of an epistemic relationship entirely.

• Partial rehabilitability: New information supporting the inferential reliability of edge \(e\) can reduce \(R(e, t)\), partially counteracting accumulated residue. The specific form of this interaction is deferred to the algorithmic level.

The boundary conditions on \(R\) are consistent with independent lines of evidence. For example, decay-weighted updating of the kind specified by the monotonicity in time condition follows recency-weighted models in behavioral economics, where accumulated experience is discounted as a function of elapsed time [31]. Meanwhile, the broader notion that failed inference attempts leave a persistent trace that shapes subsequent judgment is consistent with the Appraisal Tendency Framework [32], which documents how residual appraisals from previous evaluative episodes carry over to influence unrelated subsequent judgments, though the residue function formalizes this carry-over at the level of epistemic relationships rather than affective states.

The residue function applies only to LEWMs that were fully evaluated and rejected by the invoking model. LEWMs that were partially constructed and abandoned before evaluation, for example because new external information arrived mid-construction, do not generate residue under the current specification. This is a deliberate boundary condition consistent with the computational level of analysis. The interaction between mid-construction abandonment and residue accumulation is identified as a natural extension. The specific functional form of \(R\), including the mathematical structure of disconfirmation weighting, time decay, and rehabilitation, is deferred to the algorithmic level of analysis.

4 Worked Example: The Popcorn Bag Scenario↩︎

I trace the full operation of the ToM utility through the popcorn bag scenario, following the definitions in order. The focal agent is an observer attempting to model Sam’s belief state. I assume the invoking model has called the process and that the output is accepted and that \(S_1 = 3\), placing the focal agent at a moderate sophistication level.

4.1 Local Epistemic World Model Construction (Definition 1)↩︎

The observer constructs an initial local epistemic world model \[W = \langle A,\, O,\, E,\, \text{BLS},\, \text{obs},\, \text{cred} \rangle\] as follows:

• The agent set is \(A = \{a_1, a_2\}\) where \(a_1\) is the focal agent (the observer) and \(a_2\) is Sam.

• The state node set is \(O = \{o_1, o_2, o_3, o_4\}\) where \(o_1\) is the bag, \(o_2\) is its contents (popcorn), \(o_3\) is the label (reads “chocolate”), and \(o_4\) is the trusted friend’s testimony (“the bag has popcorn, ignore the label”).

• The edge set \(E\) includes directed epistemic edges connecting agents to state nodes and to each other. Relevant edges include: \((a_2, o_3)\), Sam read the label; \((a_2, o_4)\), Sam received the friend’s testimony; \((a_1,a_2)\), the observer is modeling Sam.

• The BLS function assigns typed values to edges. The edge \((a_2, o_3)\) carries a belief-typed BLS reflecting Sam’s exposure to the label. The edge \((a_2, o_4)\) carries a trust-typed BLS reflecting Sam’s relationship to the friend’s testimony. The edge \((a_1, a_2)\) carries a belief-typed BLS representing the observer’s current estimate of Sam’s belief state; this is the target of inference.

• The observability function assigns scalars to edges. The observer has direct access to Sam’s observable behavior and the situational structure, so \(\text{obs}(a_1, a_2)\) is high. The trusted friend’s testimony is known to the observer, so \(\text{obs}(a_2, o_4)\) is also high. The label is directly observable, so \(\text{obs}(a_2, o_3) \approx 1\).

4.2 Agent Node Structure (Definition 2)↩︎

Sam’s agent node is \(a_2 = \langle H_2, S_2, t_2 \rangle\).

• \(H_2\) is the ordered information access history for Sam: \([o_4, o_3]\). Sam received the friend’s testimony first, followed by the label. This ordering is critical. A system that does not maintain ordered \(H_i\) will treat both signals as equivalent and default to the most salient, producing the observed failure mode of large language models [3].

• \(S_2\) is estimated via incoming BLS edges from the observer. For this example we treat Sam as a typical adult reasoner, so \(S_2 = 3\).

• \(t_2\) is the snapshot timestamp representing Sam’s current state the moment after she has read the label.

4.3 Proliferation (Definition 3)↩︎

The observer is running backward inference (modeling Sam’s belief state) so no deep proliferation is required. The tree \(\mathcal{N}\) has depth \(k = 1\): the observer \(a_1\) at the root, Sam \(a_2\) as the single child node. The width bound \(b(1, 3) = \lfloor 3 \cdot \rho^1 \rfloor\) yields a single branch for any \(\rho < 1\), consistent with this being a simple one-agent modeling task. Proliferation stops at \(k = 1\) because \(\Delta C(2) < \kappa\) meaning that adding a second level of mentalizing does not improve confidence sufficiently to justify the cost.

4.4 Inference (Definition 4)↩︎

The observer runs backward inference \(\text{BI}(\beta_2, W^1) \rightarrow \langle \text{BLS}(a_1, a_2), C \rangle\) where \(\beta\) is Sam’s observable behavior: she is treating the bag as though it contains popcorn, consistent with the friend’s testimony.

Before addressing the main scenario variants, it is also worth noting another important case, the naive variant. In this case, Sam has never encountered the bag, has received no testimony, and has no relevant edges in \(H_i\) connecting her to any state node about the bag’s contents. The observer’s reconstruction of Sam’s \(H_i\) contains no epistemic edge connecting her to \(o_2\) (popcorn) or \(o_3\) (label). Backward inference therefore generates no candidates that attribute knowledge of the contents to Sam. The correct ToM output is that Sam holds no belief about the bag’s contents; the belief is absent, not false.

In the standard variant (no trusted friend), \(H_2 = [o_3]\), Sam has only seen the label. The generate-and-filter procedure produces a single strong candidate: Sam believes the bag contains chocolate. The fit score \(f_1\) for the first candidate is high given that the label is Sam’s only information source and \(\text{obs}(a_2, o_3) \approx 1\). Confidence \(C_1\) clears \(\tau_{\text{high}}\) immediately. The procedure halts at iteration 1, returning \(\langle \text{BLS}(a_1, a_2) = \text{chocolate},\, C \rangle\) with high confidence.

In the trusted testimony variant, \(H_2 = [o_4, o_3]\), testimony first, then label. The generate-and-filter procedure evaluates two candidate LEWMs: one in which Sam believes chocolate (label dominates) and one in which Sam believes popcorn (testimony dominates). The fit score for the popcorn candidate is higher because it is consistent with both the ordering of \(H_2\) and the trust-typed BLS on \((a_2, o_4)\), which carries high credibility. The chocolate candidate receives a lower fit score because accepting it requires discounting the testimony, conflicting with the trust weighting. The popcorn candidate clears \(\tau_{\text{high}}\) first. The procedure halts and returns \(\langle \text{BLS}(a_1, a_2) = \text{popcorn},\, C \rangle\) with high confidence.

This is the result that systems lacking ordered \(H_i\) and BLS-typed trust edges cannot reliably produce.

4.5 Residue (Definition 5)↩︎

In this scenario the LEWM is accepted, so Definition 5 does not apply directly. However, we can briefly illustrate the residue mechanism by considering a motivated reasoning variant: suppose the observer has a strong belief that Sam believes the bag contains pickles, encoded in \(H_1\) via past interaction history.

The generate-and-filter procedure produces pickle candidates that consistently receive low fit scores against Sam’s observable behavior and \(H_2\). The LEWM built around the pickle hypothesis is strongly disconfirmed: candidate accounts depending on it fail to make Sam’s behavior coherent. By iteration \(n = S_1 = 3\) the sophistication ceiling is reached and the invoking model rejects the low-confidence output.

Per Definition 5, this rejection is not informationally inert. Residue accumulates on the epistemic relationships whose contribution was relevant to the failed pickle LEWM, specifically the edges most implicated in constructing and evaluating pickle candidates. The credibility of those relationships is reduced for subsequent invocations, making future candidate accounts that depend heavily on them less likely to achieve sufficient confidence. The observer returns a low-confidence output, most likely popcorn or chocolate, and the pickle hypothesis is weakened as an inferential resource going forward.

Because the failure here was driven by strong disconfirmation rather than mere capacity exhaustion, the residue is weighted accordingly: the credibility reduction on implicated edges is larger than it would be had the procedure simply run out of sophistication before converging. The motivated reasoning did not succeed in overriding the evidence, but left a structured trace that will shape how the observer approaches the same epistemic relationships in future invocations.

5 Discussion↩︎

Reasoning about and inferring others’ mental states is a fundamental human cognitive ability. The theory of mind utility formalizes it as an epistemic inference problem at the computational level. Five definitions constitute ToM-U, positioning it as a domain-agnostic mentalizing mechanism. The formal model and worked example provide grounding for a more thorough assessment of ToM-U’s theoretical positioning, facilitate the development of empirical predictions, and reveal opportunities to extend the theory.

5.1 Theoretical Positioning of ToM-U↩︎

The most consequential theoretical implication of ToM-U’s formalism is the derivation of belief like states via the LEWM from ordered information access history and source credibility. This feature contrasts with Bayesian approaches that rely on planning models as their representational objects and that presuppose BLSs [4]–[6]. This presupposition of BLSs extends to other inverse planning models, such as the naive utility calculus account of social evaluation [33]. Approaches to modeling knowledge and beliefs in neighboring disciplines rely on the same presupposition. In dynamic epistemic logic (DEL), which uses Kripke structures to model how agent knowledge updates in response to information events, epistemic states are primitive structural features of the model [34]. Additionally, DEL collapses knowledge and beliefs into formally similar modal operators, a conflation the typed BLS vocabulary in Definition 1 explicitly avoids. The Alchourrón Gärdenfors & Makinson (AGM) framework in formal epistemology also assumes rather than derives a belief set [35]. Because beliefs are given, the AGM framework lacks machinery in its postulates to handle failed belief derivation. Because beliefs are not given for ToM-U, it formalizes what happens when BLS derivation fails through a novel residue function. In a sense, ToM-U is situated upstream of BToM and related approaches, providing the epistemic state estimates that these accounts presuppose but do not derive, and, when needed, a means of handing failure.

ToM-U does share features with other formal cognitive theories. For example, the sophistication architecture of ToM-U and the handling of strategic uncertainty under cognitive hierarchy theory (CHT) both use discrete depth levels to modeling bounded mentalizing [36]. Whereas ToM-U is about epistemic state inference, CHT is about action choice under strategic uncertainty. Speculatively, cognitive hierarchy models may also benefit from ToM-U style epistemic state estimates as inputs, though the relationship between the two frameworks is an open theoretical question.

ToM-U also shares modeling intent with active inference accounts of social cognition (e.g., [37]). However, they operate at Marr’s algorithmic level, specifying a mechanistic process of variational inference under a generative model. ToM-U by contrast is specified at the computational level, defining what epistemic state inference computes and why without commitment to an implementing mechanism. The two accounts are therefore not in direct competition and may be complementary across levels of analysis.

Additionally, the generate-and-filter of Definition 4 shares structural similarity with abductive inference: generating candidate explanations and evaluating them for coherence with available evidence rather than computing likelihoods over a distribution [38], [39]. This positions fit scoring as coherence evaluation, consistent with the computational level framing of ToM-U and the scalar confidence accumulator.

ToM-U occupies an unusual position in the broader debate around how people mentalize, largely thanks to its formal epistemic state inference apparatus. Neither the simulation theory [8], [9] nor the theory-theory [10], [11] tradition is so formalized. Interestingly, the formalization of ToM-U resulted in it having processes that reflect both traditions. For example, the generative logic of ToM-U’s third-person hypothesis construction about another agent’s epistemic state, rather than self-as-model, nods to theory-theory. Meanwhile, its evaluate-against-behavior structure in which candidates are evaluated by how well they account for observations acknowledges simulationism. Importantly, ToM-U does not simply take the best from either tradition. It has sharp distinctions, such as positing nested-selves-as-others in situations demanding recursive reasoning of the I-know-that-you-know-that-I-know... type. The projected self at each proliferation depth of such reasoning is a constructed other reasoned about from the outside. The second “I” is not the focal agent running its own machinery in a transformed mode, a feature that falls out of the proliferation architecture rather than requiring an additional theoretical commitment. Another distinction is the LEWM itself, the representational object neither tradition formalized: a temporary, occasion-specific working model of another agent’s epistemic situation, constructed from information access history, evaluated for coherence with observed behavior, and adopted or discarded as the operative account of that situation. ToM-U posits that mentalizing is accomplished by generating and evaluating these working models, not through a process analogous to scientific hypothesis testing or by simulating another mind.

Another unique theoretical position of ToM-U is the implication of the discrete LEWM commitment: understanding a target’s BLS well enough to act naturally yields a categorical outcome. Although confidence is graded, the focal agent’s LEWM is a commitment rather than a distribution. This maps onto Dennett’s intentional stance [20]: attributing beliefs to an agent is inherently a categorical act, one either adopts the intentional stance or not, even if the underlying process supporting that attribution are graded or probabilistic [14].

The interaction between the termination structure of mutual reconciliation and the residue function illustrates the internal coherence of ToM-U as a formal system. The choice between the min and weighted average termination formulations is not merely a parameter decision, it has downstream consequences for residue accumulation dynamics. Under the min formulation, consistently weak inference directions dominate termination, implicating the same epistemic edges repeatedly and producing stronger residue accumulation over time. A weighted average formulation would distribute termination pressure across agent pairs, producing a qualitatively different residue profile. The two formulations are therefore not interchangeable at the level of system behavior, even if they might produce similar outputs in symmetric multi-agent contexts. This interdependence between formally distinct components (inference termination and mentalizing failure traces) is a property of the architecture rather than an auxiliary assumption, and reflects the kind of theoretical integration that formal specification makes visible.

5.2 Empirical Predictions of ToM-U↩︎

An agent’s information access history, \(H_i\), governs what candidate LEWMs are constructable. A focal agent that misrepresents whether or not a target has been exposed to relevant information will systematically produce miscalibrated LEWMs, not randomly wrong ones [27]. This corruption of the candidate space can lead to two different types or focal agent errors: under and overattribution, or (not) importing world knowledge the target could plausibly have accessed. These errors will produce directionally incorrect belief attributions. For example, if the observer did not know that the testimony of the popcorn bag’s contents came from a trusted source, then they would produce a different LEWM than an observer who knows Sam trusts the source. More broadly, the focal agent’s past relationship history with the target, including trust history and accumulated mentalizing residue, is encoded in the BLS edge structure and shapes candidate generation accordingly; an observer with no past relationship to Sam has a different and likely impoverished candidate space than one with an established trust history.

LEWM construction errors are distinct from observability-driven errors. The observability function, obs, dictates the clarity of the relationships in the LEWM at invocation. A focal agent may generate a correct LEWM, but poor observability on relevant edges could produce low confidence output or premature termination. Those failure modes are certainty rather than content errors.

BLS types are not free parameters in ToM-U. The type assigned to an edge is constrained by the graph structure, the observability function (obs), \(H_i\), and the agent’s traits. In other words, type attribution is a structured inference problem rather than an unconstrained labeling task. This means that a LEWM in which Sam wants the bag to contain popcorn is different from one in which she believes it does. There are at least three associated classes of failures yielded by this theoretical position. First, when evidence supporting the correct BLS type is weak, i.e., obs is low, an observer defaults to a contextually accessible BLS type rather than the most accurate one. Not knowing Sam wants popcorn, an observer would likely default to a belief labeled edge. Second, consistent with the curse of knowledge [40], an overconfident attribution in one BLS type may contaminate estimates in another. A focal agent who is confident about Sam’s belief state may overestimate their access to her desires. Third, low sophistication agents operating with impoverished proliferation trees are predicted to collapse BLS type distinctions more readily, attributing a single undifferentiated mental state where higher sophistication agents would maintain independent belief and desire edges. This predicts that ToM sophistication should correlate with the ability to dissociate belief and desire attribution, consistent with the developmental literature on mature ToM [41].

Another failure mode associated with sophistication occurs when a focal agent systematically under or overestimates a target’s \(S_i\) and the proliferation tree is built to the wrong depth. Such construction failures result in impoverished or unnecessarily complex LEWMs. A high (low) \(S_i\) estimate will produce a tree with more (less) recursive reasoning than the target actually does, which are effects inline with empirical work demonstrating that people systematically underestimate the depth of other’s strategic reasoning [36].

ToM-U’s formalization also surfaces distinct failure modes for absent and false beliefs. Informal accounts of theory of mind, including the canonical false belief task literature, tend to collapse this distinction. A child who fails the Wimmer and Perner task may have no representation of the target’s belief state at all, or may have a representation but attribute the wrong one [2], [42]–[44]. These are structurally different failures that the ToM-U formalism distinguishes as absent (no edge in \(H_i\)) and false (edge exists with wrong BLS value) belief respectively.

More broadly, as a formalized theory ToM-U is falsifiable via its core architectural commitments. Concretely:

• Evidence that belief attribution is insensitive to the ordering and provenance of information access would undermine the representational foundation of the LEWM.

• Evidence that mentalizing is best characterized as maintenance of continuous distributions over others’ mental states rather than commitment to discrete candidate world models would falsify the generate-and-filter architecture at the computational level.

• Evidence that failed mentalizing attempts leave no structured trace on subsequent inference about the same epistemic relationships would falsify the residue function.

These are architectural commitments, not just parameter adjustments, meaning each is tractable for empirical investigation.

5.3 Motivated Reasoning and ToM-U↩︎

ToM-U also makes three theoretical contributions related to motivated reasoning, each corresponding to a different formal component. Importantly, they are empirically dissociable via distinct predicted error patterns. First, motivated reasoning impacts what information we attend to, retain, and the merit we give it [45]. In these ways, it can shape a target’s \(H_i\) and the focal agent’s reconstruction will inherit any such memory distortions. Of course, the focal agent is also subject to motivated reasoning, like wishful thinking, that can distort what they observe, meaning \(O\) might carry biased affective or motivational information that corrupts backward inference [45]. More specifically, during self-projection, motivated reasoning can distort the focal agent’s estimates of how observable their own mental states are to the target, systematically biasing the obs scalars on relevant edges [46]. There is a good argument that such miscalibration of obs estimates can serve as a substrate for self-deception as seen in the illusion of transparency [46] or spotlight effect [47]. More broadly, systematic miscalibration of obs estimates may contribute to mentalization failures documented in clinical populations [48].

5.4 Domain Generality and Extensions↩︎

The formal machinery of ToM-U is consistent with theoretical work in communications. The generate-and-filter procedure aligns with relevance theoretic accounts of utterance comprehension, and ordered \(H_i\) functions analogously to the cognitive environment construct, o the information an agent has access to at a given moment [49]. At the level of joint understanding, mutual reconciliation in Definition 4 formalizes the grounding process described by Clark and Brennan [29], in which mutual understanding is achieved through iterated exchange rather than simultaneous joint inference.

The \(H_i\) framework is well positioned for modeling episodic memory of social interactions [17]. Tracking who said what to whom in what order is essentially \(H_i\) with temporal indexing. Similarly, snapshot timestamps and proliferation tree pruning connect to event segmentation theory in which event boundaries update situation models [50]. Future work on tree rebuilding and pruning between passes, which are not specified at the computational level, may find inspiration in this literature.

Perhaps most notably, ToM-U outputs epistemic state estimates that downstream processes can use. For example, a higher order process that invokes ToM-U can use its estimates to accomplish social outcomes such as coordination, cooperation, or avoiding conflict. The domain agnosticity of ToM-U means that any parent process can invoke it; for example, curiosity pointed at social observation might give rise to spontaneous ToM [51]. It is also worth noting speculatively that cognitive hierarchy models in game theory, which presuppose that agents reason about other agents’ strategic reasoning to some bounded depth, may benefit from ToM-U style epistemic state estimates as inputs, another opportunity for future theoretical work [36].

5.5 Limitations and Extensions↩︎

ToM-U’s parameters are presented as fixed for initial analyses; each warrants contextual and cultural calibration. Cross-cultural work documents variation in the timing of false belief task success across cultures [52] as well as differences in the sequence of ToM development [53], suggesting that parameters such as \(S_{\max}\) and \(\tau_{\text{high}}\) may require cultural adjustment. Likewise, the cognitive hierarchy literature documents substantial variation in sophistication levels, \(S_i\), across contexts and populations [36].

Relatedly, individual variation in mentalizing ability, including differences in sophistication calibration, mentalizing style, and sensitivity to residue accumulation, represents a natural source of parameter heterogeneity and opportunity for future empirical work, particularly in populations where mentalization difficulties are well documented [48]. More broadly, ToM-U was developed as a theory of adaptive mentalizing, Maladaptive cases giving rise to self-deception and related clinical phenomena are left for future work, with miscalibration of the observability scalar representing a natural formal substrate for modeling clinical mentalization failures.

ToM-U can be enriched in a number of ways. First, adding a facility for multiple typed edges per node pair with independent observability scalars would allow tracking the readability of each BLS for a given relationship, enabling more detailed inference about specific mental states. Second, giving \(H_i\) veridicality weights would enable modeling confabulation and reconstructive memory [54], [55]. Third, a means of abandoning a LEWM mid-construction while still accumulating residue would extend the model to capture a broader class of inference failures. Fourth, further developing the multi-agent architecture of mutual reconciliation is warranted, as it provides a natural formal substrate for modeling collective epistemic dynamics and distributed sensemaking. Lastly, the residue function’s accumulation dynamics may also provide a formal substrate for modeling hypervigilance in interpersonal contexts, with repeated rejection events on specific edge types producing heightened mentalizing sensitivity.

Beyond these extensions, ToM-U is specified at Marr’s computational level of analysis [12]. As such, it does not provide guidance for the functional forms of fit scoring, candidate generation, or residue decay. Extending the model to the algorithmic level with prospective functional forms for these is a natural next step.

6 Conclusion↩︎

The Theory of Mind Utility formalizes how a person infers epistemic states when reasoning about other minds. This is a distinct cognitive problem that existing theoretical accounts, such as theory-theory and simulation theory, as well as formal accounts, such as Bayesian ToM and the POMDP lineage from computer science, presuppose but generally do not model. The five definitions that constitute ToM-U ensure that the mechanism is domain-agnostic and that it provides directional predictions about failure. Understanding the full range of social cognitive phenomena, from false belief to motivated opacity to collective sensemaking, necessitates a formal account of how minds construct candidate models of other minds; the Theory of Mind Utility addresses this need.

References↩︎

---

1. Discrete candidate evaluation at the computational level is agnostic about whether the underlying implementation is stochastic; in practice, a Bayesian sampler model a la Sanborn and Chater [14] could be a viable implementation.↩︎