Autoregressive models; Hidden-state probing; Interpretability; Imperfect information; Opponent modeling

1 Introduction↩︎

Autoregressive models trained on imperfect-information histories must compress public observations into hidden states. Because public histories are statistically correlated with hidden private variables, probes trained on frozen hidden states can often recover latent labels such as an opponent range. That result is not self-interpreting. The same probe score may reflect a posterior-sensitive state or a useful public-history compression whose statistics correlate with the hidden target. Recoverability alone is therefore insufficient evidence for posterior-sensitive representations, since the same probe signal may arise from publicly observable betting patterns that happen to correlate with opponent range.

Therefore, the usual binary framing of either tracking or shortcutting is too coarse. A positive probe is often read as belief tracking, while failed robustness checks are then described as mere shortcut behavior. However, probe outcomes can also exist outside these extremes. A behavior-trained sequence model may learn a predictive public-state representation: it is useful for action or value prediction and carries latent-correlated information, but the latent-label information is bounded by visible public-history composition and does not preserve opponent-range distinctions under matched-composition tests. We refer to this pattern as composition-bounded predictive support: evidence that a representation is behaviorally useful and correlated with hidden variables, while remaining insufficient to support a stronger posterior-tracking interpretation.

We study this distinction as a case study in a no-range Limit Hold’em autoregressive (AR) model trained on behavior targets. On a locked test split, the behavior head outperforms public-history lookup baselines on held-out action prediction across all three seeds. A separate representation-to-behavior audit shows that frozen hidden states remain predictive of downstream targets, especially value, although much of this relationship is captured by input-only composition features. Frozen hidden-state probes also recover nontrivial opponent-range signal: after action/value controls, residual top-10 range signal remains positive in two of three seeds, while the third remains inconclusive. Taken together, these results reject the weakest interpretation that the hidden state contains no opponent-range information.

Stronger controls alter the interpretation. Full visible-history composition features explain more range signals than residual hidden states, and residual representations fail to outperform action/value+composition priors within matched-composition buckets across all seeds. A range label proxy gap analysis likewise finds no stable residual tracking once composition is controlled. Accordingly, this case study provides evidence about opponent-range representations rather than exact Bayesian posterior tracking. The best-supported interpretation is composition-bounded predictive support: the hidden state remains behaviorally useful and correlated with opponent range, but does not support a stronger posterior-tracking claim.

This refinement avoids two overstatements: the poker AR should not be described as having learned opponent-range belief, and the result should not be dismissed as a meaningless probe artifact. Known-control, posterior/oracle, and non-poker learned-positive controls show that exact posterior states pass, raw composition states collapse, and the audit has a positive path.

Therefore, we summarize this paper’s exclusive contributions as:

1. We introduce a claim-safe three-way interpretation for hidden-state belief probes: artifact, composition-bounded predictive support, and posterior-tracking support.

2. We show that a no-range Limit Hold’em AR model falls into the CBPS regime: its states are behavior-predictive and latent-correlated, but the latent signal is bounded by public-history composition.

3. We validate the distinction with known-control CBPS, posterior-state, oracle, non-poker latent-order rows, and a controlled behavior-trained positive benchmark.

4. We provide reproducible audit outputs that separate strict case verdicts, refined interpretations, oracle validation, and forbidden stronger claims.

Table 1 previews the evidence-to-claim mapping, and Figure 1 shows the claim-gated audit ladder. The central paper contract is that protocol-validation rows do not upgrade the audited poker instance label, and the CBPS interpretation remains below an ordered posterior-tracking claim.

2 Related Work↩︎

Probing and claim control. Linear probes are widely used to inspect intermediate representations [1]. Prior work shows that probe accuracy must be interpreted through controls rather than treated as direct evidence of a represented structure [2]–[4]. Our setting adds a domain-specific confound: in imperfect-information behavior histories, visible public composition can be highly informative about the hidden target. Thus the relevant question is not just whether the label is recoverable, but whether the recovery survives action/value, composition, and matched-composition alternatives.

Belief states and imperfect information. In partially observable decision processes, a belief state is a posterior over latent state given observed history [5]. Recurrent agents and world models use hidden state to integrate partial observations over time [6], [7], and sequential-decision interpretability work studies how policies can be summarized or explained [8], [9]. This paper does not claim to identify a belief-state planner. It asks when hidden activations justify the weaker or stronger representational claims that are often attached to belief probes.

Reliability audits for interpretation. Interpretation methods can create a false sense of explanation when familiar signals are not checked against targeted alternatives. Examples include saliency-map audits and robustness specifications [10], [11], attention interpretability analysis [12], uncertainty-aware global explanations [13], concept explanations for time series [14], and robust counterfactual explanations [15]. Our analogue is a claim-gated audit for hidden-state belief probes. The CBPS interpretation is useful only because the gate structure keeps the allowed claim below posterior tracking.

3 Interpretations and Audit Protocol↩︎

Let \(x_{1:t}\) be an observable history, \(z_t\) a hidden target such as an opponent private-state class, and \(h_t=f_\theta(x_{1:t})\) a frozen hidden activation. A probe \(q_\phi(z_t\mid h_t)\) tests recoverability. The interpretation question asks which alternative explanations have been ruled out.

Table 2 states the three interpretations used in this paper. The middle row is the new framing. It should not be confused with a mechanism claim: composition-bounded predictive support means that the evidence supports a behavior-predictive, latent-correlated, composition-bounded public-state pattern under the audit. It does not prove that a particular subspace causally mediates action selection.

Definition 1 (Composition-bounded predictive support). For a frozen behavioral sequence model and latent target \(Z\), composition-bounded predictive support is the diagnostic status in which: (i) held-out behavior prediction is positive beyond public-history lookup baselines; (ii) frozen hidden states recover latent-correlated signal after label, prior, history, and action/value controls; and (iii) full and matched public-history composition controls block an ordered posterior-tracking upgrade. Thus CBPS names a positive but bounded evidence package: the state is behavior-predictive and latent-correlated, while the strongest surviving explanation for the stronger belief claim remains public-history composition. CBPS is not assigned by the failure of posterior-tracking tests alone: it requires positive behavior-prediction evidence and positive latent-label recoverability before the composition block; otherwise the audit returns only an artifact, nuisance, shortcut, or inconclusive label. It is not a causal mediation or posterior-tracking claim.

The claim-gated audit maps evidence to admissible claims rather than presenting a dashboard of ablations. A claim gate maps an evidence object to pass, blocked, untested, or insufficient_support. The case gates evaluate probe recoverability, label controls, no-observation priors, history controls, action/value nuisance, full visible composition, and matched-composition support. The protocol returns a strict case label, an oracle/protocol status, and forbidden stronger claims. This structure prevents the refined CBPS interpretation from silently replacing the stricter verdict.

Table 3 states the gate semantics used throughout the paper. A gate is not an ablation result by itself; it is a precondition for a stronger interpretation. For G1–G4, the audit delta rule requires that the configured held-out delta is positive with the reported uncertainty source. For G5–G7, the claim-critical comparisons use paired or bucket bootstrap intervals. A blocked composition gate does not prove that the model lacks every form of belief-relevant information. It blocks the stronger ordered-belief claim because a named observable alternative remains sufficient under the audit.

Table 4 makes the evidence-to-claim ladder explicit. The CBPS interpretation is intentionally placed below posterior tracking: it requires positive predictive evidence and recoverable latent-correlated signal, while composition gates still block an ordered-belief upgrade. This is the main safeguard against post-hoc renaming. If the same evidence package lacks predictive support, the audit can still return a strict shortcut verdict, but it should not receive the CBPS interpretation.

The audit configuration is fixed before interpreting the final label. The main poker case uses locked layer encoder_layer_2, the predeclared full visible-history composition summary, 1,000 bootstrap resamples where applicable, and matched-composition support with reported buckets, pairs, and dropped fraction. Oracle controls are evaluated separately from the audited case. The supplementary early-gate manifest records the statistic, pass rule, uncertainty source, and artifact path for G1–G4. The positive path is defined before the poker downgrade: exact posterior-state, oracle posterior, and belief-auxiliary latent-order representations can pass matched-composition gates, while composition shortcut controls collapse. Those rows validate protocol behavior; they do not upgrade the audited poker AR.

4 Known Controls Separate CBPS from Belief↩︎

A CBPS label would be empty if the audit could only distinguish posterior states from raw composition states. We therefore use a known-control environment with three constructed states. The posterior state carries the exact posterior and should support posterior tracking. The composition state carries only visible composition and should collapse under matched composition. The CBPS state is predictive and latent-correlated but bounded: it carries a nontrivial representation difference under matched composition while its posterior-gap slope is not stably positive.

Table 5 shows that the three labels are separated. The CBPS row has a large behavior-prediction delta against \(C_{\min}\), a positive label-control delta, and nonzero matched representation JS, but the posterior-gap slope interval crosses zero. This validates CBPS as a diagnostic label. It does not by itself claim that the poker AR has CBPS; the poker row still needs behavior and boundedness evidence.

The action-delta column validates the predictive requirement for CBPS, whereas the posterior-state row is judged by posterior-sensitive representation alignment. Thus a posterior state can validate the posterior-tracking path without satisfying the CBPS-specific predictive criterion. The CBPS state is also not the composition state: its matched representation JS is nonzero while the composition state collapses.

5 Poker: Predictive and Range-Correlated↩︎

The audited poker model is a four-layer AR Transformer with hidden size 96 and four attention heads. It is trained on action and value behavior targets with policy loss weight 1.0, value loss weight 0.3, and opponent-hand loss weight 0.0. For each seed, 50,000 teacher-policy hands are split by hand id into deterministic 60/20/20 train, validation, and test partitions. Opponent-range labels are used only after training for probes. The main rows use locked layer encoder_layer_2; best-layer curves are descriptive only. The probe target has 169 opponent-range classes, so uniform top-10 chance is \(10/169=5.9\%\). The relevant baselines are stronger empirical nuisances: no-observation prior, action/value baseline, composition-only baseline, and action/value+composition baseline. The action/value baseline trains the same linear classifier on public action and value features. The visible composition summary contains length, capped length, action counts, action proportions, action-presence indicators, first-action and last-action indicators, and position summaries for bet, call, check, and fold. It is predeclared and excludes hidden cards, opponent-range labels, teacher logits, future actions, and future outcomes. The held-out teacher-policy action mix is stable across no-range seeds: bet about 0.26, call 0.11, check 0.48, fold 0.15, with about 2.55 prior actions per decision. This makes composition a natural nuisance for poker belief claims, not an arbitrary post-hoc feature.

Behavior prediction. The first requirement for CBPS is predictive support. The behavior head predicts held-out action targets above public-history lookup baselines across all three seeds. Against the full visible-composition lookup baseline, action-accuracy deltas are about five points with tight confidence intervals (Table 6). This is not causal mediation evidence, and it does not identify which hidden subspace drives behavior. It supports the narrower claim that the no-range AR row is behavior-predictive beyond public-history lookup baselines.

Representation-to-behavior link. The representation-to-behavior audit asks whether frozen hidden representations themselves predict behavior targets when read out by independent linear probes. Against input-only \(C_{\mathrm{full}}\) linear features, hidden probes have mean action-accuracy delta +0.006609 with seed interval [-0.000407, 0.010489], and value-MSE improvement +0.318198 with seed interval [0.225321, 0.395974]. After residualizing hidden features against the same \(C_{\mathrm{full}}\) inputs, action and value deltas are negative (-0.222154 and -0.484330). The resulting gate is composition_bounded_behavior_link: it strengthens the predictive part of CBPS, but remains correlational and composition-bounded rather than causal mediation evidence.

Range-correlated signal. The action/value baseline is around 13.4–13.6% top-10 accuracy. Residual hidden-state probes after action/value control reach 14.0–14.6%. Seed k and seed m have positive lower confidence bounds; seed l is inconclusive. Thus the hidden state is not empty with respect to opponent range, but this evidence is still below the standard needed for an ordered posterior-tracking claim.

6 The Poker Signal Is Composition-Bounded↩︎

The decisive question is whether the range-correlated signal exceeds visible public-history composition. It does not. Action/value+composition baselines reach 16.5–16.7% top-10 accuracy, while the composition-residual hidden probe falls to 11.4–12.2%. Figure 3 shows that composition-aware nuisance baselines dominate residual hidden-state probes across seeds.

Matched-composition evaluation rules out a weaker objection: perhaps the composition baseline is only globally stronger, while residual hidden states still carry order-sensitive opponent-range evidence inside comparable histories. Table 8 answers that objection. Within matched buckets, residual hidden states do not beat action/value+composition priors in any seed, with confidence intervals below zero and 146 matched buckets per seed. The full artifact records matched support details beyond the bucket count; the main text reports the claim-critical per-seed deltas. The strict gate verdict remains composition_shortcut_supported.

As a same-domain gate-acceptance check, the poker oracle range-representation addendum replaces the audited AR hidden state with an oracle range representation under the same matched-composition support. The gate then accepts ordered evidence in all three seeds, with minimum lower delta 0.833489. This validates that the poker matched-composition gate can accept range-representation evidence when the representation actually contains it under same-domain support. It does not upgrade the no-range AR hidden-state label.

The boundedness audit adds two safeguards. First, the composition ladder shows that weaker summaries \(C_{\min}\) and \(C_{\mathrm{mid}}\) are visible proxy confounds, while the full visible-history summary \(C_{\mathrm{full}}\) is the decisive residual-block gate. Second, a matched range-label proxy analysis finds no stable positive residual tracking: seed confidence intervals are [-0.017343, 0.009236], [-0.026485, 0.000776], and [-0.008733, 0.023498]. This proxy analysis is not exact poker posterior evidence. It is boundedness support, and the manuscript keeps the claim at cbps_supported rather than posterior tracking.

This sensitivity result is important for the CBPS interpretation. If a very weak composition summary were the only blocker, the downgrade could look like an artifact of an arbitrary hand-engineered control. Instead, weaker summaries show that public-history proxies are informative, and the full declared summary supplies the actual blocking evidence. The conclusion is therefore not that any composition feature explains the hidden state. It is that the audit’s visible public-history composition is a sufficient alternative explanation for the stronger posterior-tracking claim.

7 Claim-Boundary Ledger↩︎

We keep several stress rows as a claim-boundary ledger rather than as main positive evidence. Range-supervised poker, MLP probe capacity, strong-teacher students, and linear range-probe subspace ablation each test a natural upgrade path. None clears the strict composition-sensitive gates required for posterior tracking. Their role is therefore narrow: they prevent stronger mechanism or belief claims from being inferred from probe recovery, while the main CBPS claim rests on the positive predictive evidence, recoverable latent-correlated signal, and the decisive composition bound. The full per-seed rows are reported in the supplement.

8 Posterior Controls Show the Positive Path↩︎

A strict audit should not only reject. We validate the distinction in settings where the correct representational structure is known or controlled. For ordered controls, the representation is the posterior or is trained to carry the posterior. For shortcut controls, the representation is a function of visible composition. When histories match on visible composition but differ in posterior, ordered controls should preserve representation divergence while shortcut controls should collapse. In notation, an ordered control uses \(r_{\mathrm{order}}(x_{1:t})=p(z\mid x_{1:t})\), while a shortcut control uses \(r_{\mathrm{short}}(x_{1:t})=g(C(x_{1:t}))\). When \(C(x_i)=C(x_j)\) but \(p(z\mid x_i)\neq p(z\mid x_j)\), the ordered control should retain representation divergence and the shortcut control should not.

Table 10 summarizes the validation and applicability rows. The explicit posterior-state RNN has posterior JS 0.462898 and representation JS 0.462898, with lower confidence bound 0.446727, while the composition-state RNN collapses to 0.000000. The latent-order benchmark is non-poker: each report has 2,048 sequences, four hidden types, length five, 111 matched buckets, and 4,096 matched pairs. Across three reports, the aggregate has 333 matched buckets and 12,288 matched pairs. Oracle posterior and belief-auxiliary GRU controls pass 3/3 reports; composition shortcut controls fail 3/3. The behavior-only GRU has high representation divergence but remains inconclusive, showing why divergence alone is not enough for a belief claim. This boundary row is useful because it separates order-sensitive variation from claim-sufficient belief evidence. An order-rich exact-posterior benchmark supplies the stronger learned-positive check: a behavior-only GRU trained only on posterior-sensitive behavior targets passes matched-composition diagnostic gates, and projection/removal of its posterior-correlated subspace changes behavior outputs more than same-rank random subspaces across three seeds. These rows validate the audit’s positive path without upgrading the Limit Hold’em poker AR verdict.

Figure 5 is the methodological counterpart to the poker downgrade. The ordered controls preserve representation divergence when the posterior changes under same-composition counterfactuals. The shortcut controls keep nonzero posterior divergence but collapse representation divergence to zero. The audit can therefore accept ordered posterior representations and reject composition shortcuts when the correct answer is known. The poker result is not a consequence of a rejection-only protocol; the audited poker case simply does not satisfy the positive path. The controlled learned-positive benchmark strengthens the audit validation but also preserves the paper’s claim discipline: random-source patching is not specific, so we report robust projection-level behavior relevance rather than full patch-specific causal belief tracking.

9 Discussion↩︎

The main lesson is that positive belief probes recover public-history information whose interpretation must be diagnosed. For the no-range poker AR row, the hidden state is not empty: it supports behavior prediction and carries recoverable opponent-range signal under weaker gates. Yet the range signal is composition-bounded. Full public-history composition explains more than the residual hidden state, and matched-composition tests block the ordered posterior-tracking claim.

CBPS is useful because it names the supported middle interpretation without erasing the strict verdict. The strict gate label remains composition_shortcut_supported. The refined interpretation is cbps_supported only with the scope diagnostic_interpretation_not_ordered_belief. This means the model has a behavior-predictive public-state pattern whose latent correlation and behavior-link evidence are bounded by visible composition. An input-level order-intervention stress test further bounds the shortcut reading: when earlier betting actions are permuted while preserving counts and the last action, hidden states and outputs move across all three seeds. Because these synthetic perturbations are not guaranteed to be legal poker trajectories, this shows order sensitivity but does not upgrade the claim to legal posterior tracking. It does not mean that the model lacks all belief-like information, and it does not identify a causal mechanism.

The controlled positives are equally important. Posterior-state, oracle, latent-order, and controlled learned-positive rows show that the audit can accept posterior-sensitive representations when they are present. In the learned-positive benchmark, a behavior-only GRU passes matched-composition diagnostic gates and has robust projection-level behavior relevance across three seeds. The known-control CBPS row shows that the middle label is distinguishable from both raw composition and posterior tracking. Together, these controls make the poker result more informative than a negative probe audit.

The practical implication is that belief-probing papers should report not only whether a latent label is decodable, but also which public-history explanations remain sufficient. CBPS is useful because it preserves a positive statement about the learned state while preventing an unsupported posterior-tracking upgrade.

10 Reproducibility↩︎

The replication package reports both the strict verdict and the refined interpretation. The source manifest records:

This prevents a presentation error in which the safer CBPS language silently replaces the stricter gate outcome. The audit operationalizes the claim gates rather than presenting a post-hoc script bundle. Given an archive root, the regeneration path produces diagnostic reports, strict and refined claim labels, paper tables, figure data, source manifests, and SHA-256 checksums. The manifest links each reported row to its source artifact, including the synthetic CBPS control, poker predictive-link report, representation-to-behavior report, composition-boundedness report, poker oracle gate-acceptance report, controlled learned-positive benchmark, composition-sensitivity report, early-gate threshold manifest, and capacity or stress-test reports. The complete audit records validation status for posterior-state, synthetic, toy, learned-positive, range-supervised, capacity, and latent-order rows without using those rows to upgrade the audited poker case. The paper figures are generated from JSON-backed figure packs with source checksums. Dense claim matrices, frozen-LM textual stress rows, strong-teacher rows, full per-seed stress tables, teacher action-composition rows, and full early-gate manifests are supplementary material. This split is intentional: the main paper shows the evidence path and decisive gate outcomes, while the supplement documents the full audit ledger and auxiliary checks.

11 Limitations↩︎

The main behavioral CBPS row is one Limit Hold’em AR family with three seeds. Controlled posterior, known-CBPS, latent-order, learned-positive, and POMDP addenda improve interpretation, but they do not prove that all behavior-trained imperfect-information AR models learn CBPS. The order-rich learned-positive benchmark is not the Limit Hold’em case, and its intervention evidence is projection-level rather than full patch-specific causal belief tracking. The poker boundedness proxy uses range-label gaps rather than exact posterior JS. The predictive-link and representation-to-behavior results are correlational and do not establish causal mediation from a representation subspace to behavior. The range-probe subspace ablation stress test is negative under the tested linear row-space intervention, so it further cautions against reading probe recoverability as behavior mediation. The MLP row is inconclusive, not a proof that higher probe capacity cannot reveal ordered evidence. Mechanistic claims would require exact-posterior poker abstraction, activation patching, stronger targeted interventions, or other identification assumptions.

12 Conclusion↩︎

Positive hidden-state belief probes should be treated as the start of an interpretation problem. In the audited no-range Limit Hold’em AR row, behavior prediction is positive and opponent-range signal is recoverable, but the signal is bounded by visible public-history composition and does not support ordered posterior tracking. The best-supported interpretation is composition-bounded predictive support, not belief tracking and not an empty artifact. Belief probing should therefore move from asking only whether a latent label is recoverable to asking what kind of public-history state makes it recoverable.

References↩︎