What Am I Missing? Question-Answering as Hidden State Probing
May 29, 2026
Abstract
Test-time reasoning has become a significant field of study since the introduction of chain-of-thought reasoning in large language models (LLMs). However, the mechanisms of this reasoning process are still under-explored — from the same input prompt, and even the same partial solution, LLMs can produce varied answers if sampled multiple times. We propose to leverage question-asking as an inference-time intervention that articulates information about the model’s hidden state. To achieve that, we present a student-teacher setting where a student asks questions to a teacher. We train a probe on the student’s hidden state before and after asking a question and find it is predictive of the trajectory’s final correctness, even before generating the teacher’s answer. This suggests there is a meaningful signal from the self-diagnosis that occurs during question generation rather than information transfer from the teacher. We then frame question-asking as a sequential decision problem, using this probe as a quality score, and define a gating policy to ask questions that maximize likelihood of correctness. We find that the success of question-asking as an intervention is largely dependent on the model’s self-consistency. Our empirical results show a gap between detection and recovery; while our gating policy captures model correctness and uncertainty, interventions are equally likely to harm correct trajectories as they are to recover incorrect ones. This gap between diagnosis and correction has broader implications on language models’ capacity for self-refinement under uncertainty. 1
1 Introduction↩︎
Since the introduction of chain-of-thought reasoning, test-time technologies have become a significant field of study for large language models (LLMs). This has also led to the introduction of Large Reasoning Models (LRMs) that undergo post-training to encourage the model to produce intermediate tokens before formulating the final answer [1]. There have been various interventions to improve LLMs’ performance at inference time by manipulating intermediate reasoning tokens [2]–[4]. For example, [5] demonstrate that simply replacing the stop token with a continuation such as “wait,” often leads to significant improvements in accuracy. In addition, other works treat language model generation as an exploration in the generation space and employ branching to search for optimal reasoning trajectories [6]–[8]. The third class of intervention is through question-asking (QA) — questions are an effective mechanism for clarification [9], [10], gathering new information [11], or encouraging the decomposition of complex tasks [12]. These interventions aimed to compensate for the same fundamental issue: LLMs struggle to consistently self-correct reasoning without external feedback [13], [14].
Inspired by previous work [15], [16], we investigate whether question-asking can be used to guide and correct test-time reasoning. Question-asking is theorized to contain more semantic information, which explicitly encourages the language model to guide itself to the correct answer [17]. We hypothesize that the process of formulating a question can inherently change the model’s reasoning trajectory at a semantic level — introducing new perspectives or angles on a problem — and examine how this translates to self-recovery. We propose two specific hypotheses: First, if the reasoning tokens are meaningful, then introducing new semantic information would help steer the trajectory. Second, if a model is truly “proficient” at reasoning, it would be able to ask questions that can correct its own logic. The objective of this study is to leverage question-asking to predict the correctness of the reasoning trajectory, and to introduce interventions to question and recover from uncertain steps. We frame our study as a decision-making problem in a student-teacher setup, where a student is asking questions to a teacher and re-incorporating the teacher’s answer into its own reasoning trajectory. We introduce a gating policy that uses hidden-state probing to assess two axes of good questions: (i) which to ask: what student questions would lead to the most fruitful outcomes? and (ii) when to ask: when is an intervention likely beneficial to the student?
Our contributions are summarized as follows:
Our study shows that prompting a reasoning model to generate questions at test-time produces a measurable shift in the hidden state before incorporating the answer. We train a hidden state probe and empirically demonstrate the probe’s scoring is reasonably well-calibrated to the final correctness rate. Additionally, we observe the range of predicted correctness from a sampled set is well-correlated to correctness rate, more than per-set semantic diversity.
We frame question-asking as a sequential decision problem in a student-teacher setting and propose a pipeline with a gating mechanism based on our probe-estimated correctness, to intervene on a reasoning trajectory when it is likely to be wrong. This pipeline frames question-asking as trajectory recovery, where we need to shift incorrect answer distributions towards a desired direction.
Our research presents thorough ablations to analyze the mechanisms of question-asking and find, despite identifying incorrect trajectories, there exists a gap between detection and recovery. We show that the success of an intervention is strongly correlated to baseline self-consistency. Even though we can predict correctness from the hidden state, the student does not necessarily utilize this signal to assess its own trajectory.
2 Related work↩︎
2.0.0.1 Self-correction and self-refinement.
There is a large body of work that studies whether language models can improve their own reasoning through iterative refinement [18] or, more broadly, self-evolution [19]. [20] propose self-consistency, which samples multiple chains-of-thought (CoTs) and aggregates via majority vote. This provides a critical baseline for test-time compute: if repeated sampling achieves the same gains as interaction, external feedback may be unnecessary. [21] introduce STaR, where models iteratively refine reasoning using hints derived from correct answers. Similarly, [22] propose Reflexion, where models critique failed attempts and retry using feedback signals. These approaches demonstrate that retrying with additional information can improve performance. However, they rely on oracle signals (e.g., correctness labels or ground-truth answers) to guide refinement. [13] and [14] demonstrate that LLMs struggle with self-correction in the absence of external feedback. We investigate whether there are internal signals of correctness, and whether or not they translate into action.
2.0.0.2 Asking clarifying questions.
Several works study whether models can improve reasoning by asking follow-up questions. Self-Ask [12] prompts a model to decompose problems into sub-questions that it asks itself, or feeds into an external search engine. Prior work in dialogue and information retrieval [9], [10] studies when clarification questions are worth asking, framing the problem as a trade-off between cost and expected utility. Our work differs in two key ways. First, we explicitly separate the student (reasoner) and teacher (answering model), allowing us to analyze the effect of interaction. Second, we evaluate whether asking questions provides value beyond simply triggering regeneration.
2.0.0.3 Internal representations and probing.
There are many works that study what information is encoded in neural representations via probing classifiers [12], [16], [23]. [15] finds that it is possible to predict the final correctness from the input question with a linear probe, which they use to inform their budget for test-time scaling. In contrast, we use the hidden state shift over a question and provide detailed analysis of the actual utility of our probe at inference time.
3 Methodology↩︎
Figure 1: An illustration of our experimental setting. There is a student \(\theta\) that generates a partial solution \(\tau\) for a problem. The decisions of what to ask and when to ask are determined by a gating policy \(\pi\). First, it chooses a step in the reasoning trajectory to pause the generation. We sample \(n\) questions from the student and choose the best one by predicted correctness to be sent to the teacher \(T\). The student then incorporates the answer into a rewrite of the uncertain step..
3.1 Setup↩︎
Our setup is illustrated in 1. We frame our study as a sequential decision problem, where a student is gated by an external policy to ask questions to a teacher. To focus on the model’s capacity for self-correction, most of our study is self-play — i.e. the same model plays the role of both student and teacher with the same available information. Let \(x\) denote a problem instance and \(y \in \mathcal{Y}\) a discrete answer. A student model \(\theta\) induces a distribution \(p_\theta(y \mid x)\) over answers by sampling a full rollout. A question \(q\) posed to a teacher \(p_T\) produces a response \(a\), after which the student updates its answer distribution to \(p_\theta(y \mid x, q, a)\).
At a time step \(t\), we decide whether to trigger the intervention. Let \(\tau_t = (x, z_{1:t})\) denote the partial reasoning trajectory. We intervene on the partial reasoning trajectory with a gating policy \(\pi(\tau_t)\), formally defined as the following threshold decision at step \(t\):
\[\label{eq:gate} \pi(\tau_t) = \begin{cases} \pi(\cdot \mid \tau_t), & s_t \le \gamma \\ \varnothing, & \text{otherwise}. \end{cases}\tag{1}\]
The intuitive idea is simple; we do not need to intervene on problems the student already answers well on its own. We wish to find a score \(s_t\) that measures the expected quality of the current trajectory.
3.2 Measuring Intervention Quality↩︎
We study the effect of test-time interaction through the change in the student’s answer distribution. This can be conceptualized as a Jensen-Shannon divergence between the student’s distribution in the answer space pre- and post- question, i.e. \(\Delta_q(x) \;=\; D_{\mathrm{JS}}\big(p_\theta(y \mid x) \,\|\, p_\theta(y \mid x, q, a)\big)\). We refer to \(\Delta_q(x)\) as the distributional shift induced by question \(q\).
The quantity \(\Delta_q(x)\) captures the magnitude of change, but not whether the change improves correctness. One primary goal of this study is to assess the direction of the change in trajectory induced by a question \(q\), such that we can distinguish and characterize helpful from harmful questions. Let \(y^\star\) denote the correct answer.
We define the expected accuracy change induced by a question and a sampled answer as: \[\Delta_{\mathrm{acc}}(q;x) = \mathbb{P}_{y \sim p_\theta(\cdot \mid x,q,a)}(y = y^\star) - \mathbb{P}_{y \sim p_\theta(\cdot \mid x)}(y = y^\star)\]
In practice, we estimate this quantity via discrete rollouts sampled over the reasoning trajectory with and without the question: \[\hat{\Delta}_{\mathrm{acc}}(q;x) = \frac{1}{K} \sum_{i=1}^{K} \mathbf{1}[y_{i,q} = y^\star] - \frac{1}{K} \sum_{i=1}^{K} \mathbf{1}[y_i = y^\star] \label{eq:acc}\tag{2}\] where \(y_{i,q} \sim p_\theta(\cdot \mid x,q,a)\) and \(y_i \sim p_\theta(\cdot \mid x)\).
A question is beneficial if \(\Delta_{\mathrm{acc}}(q;x) > 0\) and harmful if \(\Delta_{\mathrm{acc}} < 0\). The expected accuracy gain \(\Delta_{\mathrm{acc}}(q;x)\) involves two sources of stochasticity: (i) the teacher response \(a \sim p_T(\cdot \mid x,q)\) and (ii) the student’s decoding \(y \sim p_\theta(\cdot \mid x,q,a)\). This shows that \(\Delta_{\mathrm{acc}}\) is estimating the Value of Information induced by the act of asking a question. See Appendix 7 for more details.
This \(\Delta_{acc}\) formulation is meant to capture the benefit of question-asking compared to the counterfactual (no intervention), but it is measured post-hoc. We do not assume access to the baseline answer at inference time. In other words, an optimal intervention is one that maximizes this shift in the answer distribution, not necessarily one where the right answer is reachable but rarely sampled. We measure this by reporting majority vote accuracy.
3.3 Probing Test-time Question-Asking↩︎
With these definitions of question quality as an effect on the final answer distribution, we seek to predict the effectiveness of question-asking as a test-time intervention strategy. We study the value of the question itself, and also the timing of the question — that is, does there exist a critical point where a trajectory can be rescued and guided towards correctness. Empirically, we find that we can predict \(p_\theta(y \mid x, q, a)\) using a probe trained on the hidden state shift induced by the question \(q\). Using this probe, we can approximate the correctness of the current reasoning trajectory. We utilize this correctness score to decide whether or not to ask a question, formulated as a sequential decision problem. See Appendix 7 for full details.
3.3.0.1 Belief State Probe.
We measure the belief state as the answer distribution obtained by sampling full rollouts from a reasoning trajectory. For a partial reasoning trajectory \(\tau_t\), the student maintains a distribution \(Y_t\) over the answer space obtained by sampling continuations, i.e. \(Y_t := p_\theta(y \mid \tau_t).\) At time \(t\), the student samples a question \(q_t\) from a feasible set \(\mathcal{Q}_\theta(\tau_t)\), and updates the context to the triplet \((\tau_t, q_t, a_t)\). The answer distribution after a QA intervention \(Y_q\) is defined as: \[Y_q := p_\theta(y \mid \tau_t, q_t, a_t).\]
We train a hidden state probe to predict the correctness rate at time step \(t\) before generating the teacher answer. That is, for a hidden state \(h_\theta\), we train a probe to predict the correctness rate \(P(y_q = y\star)\) where \(y_q \sim Y_q\). We take the hidden state as the embedding of the last token at the final layer of the model, i.e. the same embedding used for token prediction. We concatenate the hidden states \(h_\theta(\tau_t)\) and \(h_\theta(\tau_t,q_t)\), before and after asking the question.
3.3.0.2 Question-Asking as a Test-time Intervention
We formulate what to ask as the question that maximizes the expected intervention outcome, or maximally increases \(\Delta_{acc}\). Next, we discuss the timing — when to ask — as a decision made over the belief state \(\tau\). Unlike 2 , we assume no access to the baseline answer distribution \(\mathbb{P}_{y \sim p_\theta(\cdot \mid x)}(y = y^\star).\) We sample \(N\) discrete questions \(Q(\tau_t)\), and take the score that triggers the gating policy \(\pi(\tau_t)\) as the following: \[s_t = \min_{q \in Q(\tau_t)} \mathbb{E}{y \sim p_\theta(\cdot \mid \tau_t, q)}[\mathbf{1}[y = y^\star]]\]
Note the minimum term — this again derives from the idea of treating a hidden state as a belief state, and asking questions is an articulation of possible trajectories that could occur from a specific reasoning trajectory. If there is at least one question sample that is predicted to lead to a weak belief state, then the gate triggers. This is intuitively equating model uncertainty to trajectory variance; that is, if a bad final state is reachable from the current reasoning step, then that can be treated as information on the trajectory as a whole. We then choose the maximum-scoring question from the sampled set to ask the teacher, the teacher answers, and we rewrite the step triggered with the teacher’s answer in context.
3.3.0.3 Gate Scheduling
Another benefit of studying math problems is that there are many well-trained process reward models (PRMs) that provide an external source of correctness. We test several timing strategies to decide when to check the gating policy:
Fixed interval: Always trigger the gate check every \(n\) reasoning steps. Additionally, we run \(m\) iterations of the QA loop and keep the final answer at iteration \(m\).
Adaptive (PRM-weak): Only trigger the gating decision on steps where the PRM score falls below a threshold \(\alpha\). We additionally continue to employ the idea of answer mass as an early stopping criteria — that is, if all sampled trajectories exceed the threshold and there is low variance between the samples, we assume sufficiency and stop.
4 Experimental Settings↩︎
4.0.0.1 Models
We use Qwen3 [24] at 1.7B, 4B, and 14B parameters. For the PRM, we use Qwen2.5-Math-PRM-7B [25]. We theorize Qwen3 models are well-calibrated to math correctness, and have a stronger signal to be surfaced from their hidden state. To analyze the transferability to another model family, we also include results on Olmo-3-7B-Think [26].
We mostly test math reasoning to study the correlation between proficiency and recovery — Qwen3 models already achieve strong baseline performance on these datasets, which allows us to test self-correction on known capability. Also, process reward models are more calibrated on verifiable reasoning, so they are an effective external signal.
4.0.0.2 Datasets
We evaluate on three math datasets:
GSM8k[27] — 1,319 samples of grade school level word problems in math. We take 100 samples of the train split for training our hidden state probe. oh
Math500[28] — 500 samples of competition-level math. We take the first 15 samples as a training set for the hidden state probe, and report results on the 485 remaining samples.
AIME24[29] — A held-out set of 30 Math Olympiad questions from
4.0.0.3 Hyperparameters
For our gating threshold, we choose \(\gamma\) to be 0.6 after a hyperparameter sweep. We set \(n\) to be 5 reasoning steps (split by newlines, not individual sentences), and \(m\) to be 5, as we find that there are minimal gains beyond 5 interventions. Unless otherwise specified, the default metric is Majority Vote accuracy over five rollouts. Since we report majority vote, we use bootstrapping to approximate confidence intervals in our results. Please see Appendix 8 for more details.
5 Results↩︎
3pt
| Accuracy, mv (%) | Mean Questions | Mean Tokens | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 4-6 (lr)7-9 (lr)10-12 Model | Setting | Mode | GSM | MATH | AIME | GSM | MATH | AIME | GSM | MATH | AIME |
| Qwen3-1.7B | Baseline | (nt) | 84.9\(_{\scriptstyle \text{[82.9--86.9]}}\) | 76.5\(_{\scriptstyle \text{[72.6--80.4]}}\) | 16.7\(_{\scriptstyle\phantom{0}[\text{3.3--30.0}]}\) | 265 | 871 | 2,933 | |||
| Baseline | (t) | 91.4\(_{\scriptstyle \text{[90.0--92.9]}}\) | 92.8\(_{\scriptstyle \text{[90.3--95.1]}}\) | 48.3\(^\star_{\scriptstyle\phantom{00000000}}\) | 1,380 | 5,074 | 13,570 | ||||
| Adaptive | (nt) | 86.1\(_{\scriptstyle \text{[84.2--88.0]}}\) | 76.9\(_{\scriptstyle \text{[73.2--80.8]}}\) | 13.3\(_{\scriptstyle \phantom{0}\text{[3.3--26.7]}}\) | 0.1 | 0.5 | 1.2 | 318 | 1,260 | 4,584 | |
| Fixed | (nt) | 84.3\(_{\scriptstyle \text{[82.4--86.3]}}\) | 75.9\(_{\scriptstyle \text{[72.2--79.6]}}\) | 13.3\(_{\scriptstyle \phantom{0} \text{[3.3--26.7]}}\) | 0.3 | 2.1 | 3.8 | 1,356 | 5,391 | 16,772 | |
| Adaptive | (t) | 91.6\(_{\scriptstyle[\text{88.5--94.4}]}\) | 92.8\(_{\scriptstyle \text{[90.2--94.8]}}\) | 53.3\(_{\scriptstyle \text{[36.6--70.0]}}\) | 0.1 | 0.3 | 0.8 | 840 | 2,939 | 13,744 | |
| Qwen3-4B | Baseline | (nt) | 90.6\(_{\scriptstyle[\text{88.5--92.5}]}\) | 86.6\(_{\scriptstyle[ \text{83.5--89.5}]}\) | 36.7\(_{\scriptstyle[\text{20.0--53.3}]}\) | 270 | 698 | 1,271 | |||
| Baseline | (t) | 95.2\(_{\scriptstyle \text{[94.0--96.4]}}\) | 96.7\(_{\scriptstyle[\text{94.9--98.5}]}\) | 76.7\(_{\scriptstyle \text{[60.0--90.0]}}\) | 2,327 | 6,897 | 12,259 | ||||
| Adaptive | (nt) | 93.1\(_{\scriptstyle \text{[91.7--94.5]}}\) | 85.6\(_{\scriptstyle \text{[83.0--88.8]}}\) | 30.0\(_{\scriptstyle[\text{13.3--46.7}]}\) | 0.0 | 0.2 | 1.0 | 296 | 1,104 | 5,317 | |
| Fixed | (nt) | 92.6\(_{\scriptstyle[\text{91.1--93.9}]}\) | 86.0\(_{\scriptstyle[\text{83.7--89.7}]}\) | 28.6\(_{\scriptstyle[\text{14.3--46.4}]}\) | 0.0 | 1.0 | 2.3 | 1,374 | 4,885 | 20,523 | |
| Adaptive | (t) | 96.1\(_{\scriptstyle \text{[94.0--96.4]}}\) | 96.7\(_{\scriptstyle \text{[94.8--98.6]}}\) | 80.0\(_{\scriptstyle[\text{66.7--93.3}]}\) | 0.0 | 0.0 | 0.0 | 568 | 942 | 1,300 | |
| Qwen3-14B | Baseline | (nt) | 96.0\(_{\scriptstyle[\text{93.8--98.0}]}\) | 88.5\(_{\scriptstyle[\text{85.4--91.6}]}\) | 43.3\(_{\scriptstyle[\text{26.7--60.0}]}\) | 192 | 840 | 4,324 | |||
| Baseline | (t) | 96.6\(_{\scriptstyle[\text{95.5--97.6}]}\) | 97.1\(_{\scriptstyle[\text{94.2--99.4}]}\) | 80.0\(_{\scriptstyle[\text{50.0--100.0}]}\) | 2,699 | 4,611 | 11,871 | ||||
| Adaptive | (nt) | 95.4\(_{\scriptstyle[\text{94.1--96.5}]}\) | 89.1\(_{\scriptstyle[\text{86.2--91.8}]}\) | 36.7\(_{\scriptstyle[\text{20.0--53.3}]}\) | 0.0 | 0.2 | 0.9 | 200 | 979 | 3,915 | |
| Fixed | (nt) | 95.2\(_{\scriptstyle[\text{93.6--96.5}]}\) | 87.8\(_{\scriptstyle[\text{84.7--90.5}]}\) | 40.0\(_{\scriptstyle[\text{23.3--60.0}]}\) | 0.1 | 3.9 | 3.1 | 1,115 | 4,058 | 17,730 | |
| Adaptive | (t) | 96.6\(_{\scriptstyle \text{[94.7--98.4]}}\) | 97.3\(_{\scriptstyle[\text{95.6--98.8}]}\) | 86.7\(_{\scriptstyle[\text{73.3--96.7}]}\) | 0.1 | 0.0 | 0.0 | 533 | 914 | 1,133 | |
| Olmo-3-7B-Think | Baseline | (t) | 94.9\(_{\scriptstyle \text{[92.4--97.2]}}\) | 91.2\(_{\scriptstyle \text{[87.1--94.8]}}\) | 76.7\(_{\scriptstyle \text{[60.0--90.0]}}\) | 2,066 | 5,026 | 15,429 | |||
| Adaptive | (t) | 95.5\(_{\scriptstyle[\text{93.2--97.5}]}\) | 97.0\(_{\scriptstyle[\text{95.4--98.4}]}\) | 76.7\(_{\scriptstyle[\text{60.0--90.0}]}\) | 0.0 | 0.3 | 1.0 | 2,087 | 7,668 | 33,243 | |
| Fixed | (t) | 95.7\(_{\scriptstyle[\text{93.5--97.8}]}\) | 97.3\(_{\scriptstyle[\text{95.4--99.0}]}\) | 73.3\(_{\scriptstyle[\text{56.7--86.7}]}\) | 0.3 | 1.0 | 2.0 | 10,103 | 31,212 | 93,340 | |
5.1 Hidden state probe↩︎
5.1.0.1 Correlation to actual correctness.
We take the predicted correctness rate from our probe in each model, and measure it against the actual correctness rate of the final rollouts. Our probe is reasonably calibrated to actual correctness; we achieve Brier scores of 0.157 for Qwen3-1.7B to 0.121 for Qwen3-14B without the teacher answer. The calibration improves linearly with model size; the full list of scores along with p-value significance can be found in Appendix 9. We tested various formulations — including just the pre-question embedding \(h_\theta(\tau_t)\) or just the post-question embedding \(h_\theta(\tau_t,q)\) — but the combination of pre+post was the most predictive of final correctness rate. This exemplifies the concept of hidden state shift; the question itself induces a change in the hidden state representation that is indicative of final performance.
5.1.0.2 Predictive power of probe score.
For our fixed setting, we track the range of predicted probe scores as the diff between max. and min. scored correctness. The results are available in 2. Binning the result into four quartiles, we observe that accuracy is negatively correlated with range; that is, the wider the range of sampled values, the lower the final model accuracy. This indicates that answer mass is detectable from the questions asked by the model, even before incorporating the teacher’s answer, and this is strongly correlated with the student’s final correctness. We also analyze the semantic diversity of the probe scores with a cheap TF-IDF metric, as shown in 3. There is some weak relationship between semantic diversity and probe spread in the Qwen3 models — we measured \(\rho\) scores between 0.258-0.619 — but the correlation is weaker in Olmo-3-7B-Think (\(\rho=-0.042\)). This indicates the hidden state information learned by the probe is not strongly related to the semantics of the question.
5.2 Effectiveness of Question-Asking as an intervention.↩︎
Next, we use this hidden state probe score as a threshold value for our gating policy \(\pi(\tau_t)\) in an LLM’s reasoning trajectory. We seek to understand if this signal can be translated into a test-time intervention that can consistently detect and correct weaker trajectories. Our main findings are summarized in 1. We compare our main findings on Qwen3 with thinking mode and non-thinking mode, as well as Olmo-3-7B-Think which always uses thinking mode. Please refer to 8 for Pass@5 results. We consistently outperform the non-thinking baseline in Majority Vote accuracy by 1-3 points on GSM8k. We hypothesize that GSM8k’s problem set is more suited to question-asking; there is, for example, a socratic subset where each solution can be framed as a series of questions. However, after accounting for confidence intervals, this gain is only significant on Qwen3-4B. Comparing like modes — baseline-thinking and adaptive-thinking — we make consistent gains on most models except for Qwen3-1.7B on hard math benchmarks, and Qwen3-14B on easy benchmarks. For Qwen3-1.7B we believe the model is too small to think of any effective question, whereas on Qwen3-14B, the model is already saturated on GSM8k and more likely to get confused on easier samples. To study \(\Delta_{acc}\), the rest of our ablations only use non-thinking mode. We conduct further analysis below.
5.2.0.1 Question-asking vs. increased thinking.
The main benefit of our intervention is token efficiency. Overthinking is well-studied in previous works [30], [31] — in thinking mode, Qwen3-14B uses 20x more tokens than non-thinking mode on GSM8k for only +0.6pp improvement. However, as shown in 1, our adaptive setting achieves greater token efficiency for the gained accuracy (+4.7 points accuracy compared to the non-thinking baseline for \(\approx\)2x the tokens in our adaptive setting on Qwen3-1.7B, compared to thinking which is +10.7 points accuracy for \(\approx\)10x the tokens). The gain is more apparent with the Adaptive (t) setting. Our QA pipeline allows the model to complete generation faster than zero-shot prompting (10x reduction in Qwen3-4B and Qwen3-14B). While there are no dramatic performance improvements (the Confidence Interval at 30 samples is \(\approx \pm\)15pp), our adaptive setting observes consistent improvements.
We also observe our gating policy is weakly aligned to difficulty and supports the concept of token budgeting by naturally gating more for uncertain or weak reasoning steps. In 1, the mean number of backtracks typically increases with dataset difficulty. AIME24, our most challenging dataset, has the most mean backtracks per sample. For Math500 in our adaptive setting, the number of iterations is weakly correlated to the level of difficulty (\(\rho\) between 0.143-0.179 per model), which indicates that the samples models struggle with are only weakly correlated to human-perceived difficulty of the questions. In 3 we also find the probe score \(s_t\) is even more strongly correlated to the difficulty level. This indicates that \(s_t\) tracks difficulty more closely than the probe score range, even though these measures are correlated.
4pt
| :========== 3-5 (lr)6-8 (lr)9-11 Direction | :======== Bin | Adaptive, Belief-gating ======:+======:+================: n | nf | \(\Delta\)acc | | | | | | | Fixed, Belief-gating ======:+======:+================: n | nf | \(\Delta\)acc | | | | | | | Fixed, No gating ======:+=====:+================: n | nf | \(\Delta\)acc | | | | | | | ==: | ==: | ==: | ==: | ==: | ||||||
| Correct | \(\le\) 2/5 | 30 | 14 | +4.67 | 60 | 29 | +1.33 | 48 | 23 | +0.00 | |||||
| 3/5 | 39 | 11 | +2.56 | 88 | 30 | +2.27 | 69 | 17 | +5.80 | ||||||
| 4/5 | 58 | 12 | -5.52 | 137 | 22 | -7.01 | 129 | 17 | -3.15 | ||||||
| 5/5 | 135 | 15 | -13.78 | 582 | 25 | -6.39 | 1,084 | 13 | -2.79 | ||||||
| Wrong | \(\le\) 2/5 | 168 | 136 | +5.60 | 204 | 170 | +6.37 | 189 | 157 | +6.88 | |||||
| 3/5 | 45 | 27 | +6.22 | 69 | 40 | +9.28 | 62 | 28 | +8.06 | ||||||
| 4/5 | 36 | 14 | +8.33 | 48 | 17 | +10.00 | 42 | 14 | +10.48 | ||||||
| 5/5 | 28 | 8 | +19.29 | 45 | 7 | +8.00 | 63 | 8 | +3.17 | ||||||
| Direction split (fired) | 262C / 277W (51.4% W) | 867C / 366W (29.7% W) | 1,330C / 356W (21.1% W) | ||||||||||||
| Pool n (fire rate) | 3,839 (14.0%) | 4,188 (29.4%) | 1,708 (98.7%) | ||||||||||||
| \(\Delta\)acc (fired) | \(+\)0.22% | \(-\)1.31% | \(-\)0.35% | ||||||||||||
| \(\Delta\)acc (pool) | \(+\)0.031% | \(-\)0.387% | \(-\)0.343% | ||||||||||||
5.2.0.2 Measuring \(\hat{\Delta}_{acc}\) and \(\Delta_{q}\) in Question-asking.
In 2, we isolate the benefit of our question-asking intervention from pure resampling by taking the subset of all samples and models where the gating policy triggered. We define a wrong sample as one where the baseline’s majority-vote accuracy was wrong, and divide the results by correct vs. incorrect at the zero-shot baseline. The No Gating setting is where we intervene on every sample for the full \(m\) iterations, where wrong samples only represent 23.3% of the samples; belief-state gating with fixed scheduling fires on wrong samples even less frequently at 17%. In our adaptive setting, the gating policy uses the internal hidden state probe and an external PRM; of the trajectories caught by this policy, 51.4% were wrong in their measured baseline. This demonstrates the policy itself is efficient at catching samples that are likely wrong.
However, the effectiveness of our intervention is double-edged. In confident baselines (5/5 correct or 5/5 incorrect), \(\delta_{acc}\) increases monotonically with selectivity (\(+\)3.2 \(\to\) \(+\)8.0 \(\to\) \(+\)19.29 pp for wrong baselines). This demonstrates the gating policy is effectively firing on model uncertainty, even when the final answer is 5/5 certain. We observe recovery of wrong samples, but our intervention can also harm confidently correct samples; for the fixed trigger scheduling, \(|\hat{\Delta}_{acc}|\) is greater in magnitude on confidently correct samples than on confidently incorrect ones. While there are many samples that flip predictions in less confident bins (e.g. 136/168 had some shift in the adaptive setting, \(\leq\)2/5 bin), the magnitude of \(\hat{\Delta}_{acc}\) is low because the trajectory does not shift towards the right answer. The marginal \(\Delta_{acc}\) is relatively even, within 1% pp of the baseline.
This asymmetry reflects the same mechanism identified by [32], where a model is more susceptible to changing its opinion when it is underconfident. Our question-asking intervention implicitly signals doubt in the current trajectory and the model treats this as necessary feedback, updating its distribution even when the original answer was correct. Additionally, the PRM threshold filter also catches samples where, even though the trajectory is confidently correct, might contain reasoning steps where the model is less confident.
5pt
| Adaptive | Forced | |||||||
|---|---|---|---|---|---|---|---|---|
| 2-5 (lr)6-9 | Mean of min | Trajectory min | Mean of min | Trajectory min | ||||
| 2-3 (lr)4-5 (lr)6-7 (lr)8-9 Model | \(\rho\) | p-val | \(\rho\) | p-val | \(\rho\) | p-val | \(\rho\) | p-val |
| Qwen3-1.7B | -0.256 | \(3.8\!\times\!10^{-6}\) | -0.252 | \(5.2\!\times\!10^{-6}\) | -0.430 | \(9.7\!\times\!10^{-17}\) | -0.420 | \(5.9\!\times\!10^{-16}\) |
| Qwen3-4B | -0.269 | \(2.7\!\times\!10^{-6}\) | -0.289 | \(4.2\!\times\!10^{-7}\) | -0.358 | \(2.9\!\times\!10^{-13}\) | -0.373 | \(2.4\!\times\!10^{-14}\) |
| Qwen3-14B | -0.252 | \(3.4\!\times\!10^{-5}\) | -0.251 | \(3.7\!\times\!10^{-5}\) | -0.485 | \(3.1\!\times\!10^{-7}\) | -0.496 | \(1.5\!\times\!10^{-7}\) |
| Olmo-3-7B-Think | -0.332 | \(9.3\!\times\!10^{-7}\) | -0.342 | \(4.4\!\times\!10^{-7}\) | -0.277 | \(3.8\!\times\!10^{-3}\) | -0.248 | \(1.0\!\times\!10^{-2}\) |
5.2.0.3 Effect of question context
We conduct an additional ablation on Math500 across Qwen3-1.7B and Qwen3-14B to validate our two decision axes: when to ask and which to ask. We compare the value of question-asking interleaved with reasoning, as compared to pre-solution, where all questions are asked before the student attempts to answer the original query. As shown in 4, the partial solution \(\tau_t\) consistently achieves stronger performance on plain logprob question selection, which we theorize is because the partial solution gives a semantic anchor to the model. Our probe-based reranking generally results in better outcomes compared to just selecting the most probable question, where the pre-solution timing becomes equal to — or better than — interleaved for Qwen3-1.7B. This demonstrates that our hidden probe captures features that are not necessarily captured by the semantic generation, and this scoring surfaced from the hidden state is essential to guide the intervention. Additionally, this exemplifies a separation between what information is available in the hidden state, and what is used for predicting tokens. It is possible to surface information about correctness but it is not naturally available to a model through token generation.
Figure 5: Effect of teacher model and multiple question-asking iterations on Qwen3-1.7B, measured as Majority vote accuracy on Math500 and GSM8k. Self-play: student and teacher are both Qwen3-1.7B. Strong teacher: teacher is Qwen3-14B.. a — Math500., b — GSM8k.
5.2.0.4 Effect of the teacher’s answer.
In 5, we additionally analyze the effect of the teacher answer on the performance of our weakest model, Qwen3-1.7B. We compare self-play to a stronger model, Qwen3-14B, on GSM8k and Math500, and show the effect of capping our intervention by early stopping at each iteration. The fixed setting actively removes all benefits of the strong teacher and results in worse performance than self-play. However, the adaptive setting slightly improves the best teacher performance. This validates our hypothesis of when to ask — there exists an ideal timing where an intervention helps turn an otherwise incorrect trajectory into a correct one. As discussed, the adaptive setting has slightly more positive shifts because it is selective and allows more targeted interventions. However, these are not significant considering the confidence intervals of these experiments. The performance improvements are still bounded by 1. the quality of the questions asked by the student, and 2. the distribution shift asymmetry observed in 2.
6 Conclusion↩︎
This work investigates the dynamics of question-asking as a test-time intervention. Our study shows that (1) models partially encode their own correctness in the hidden state shift induced by asking a question; (2) this signal is predictive of final correctness, independent of semantic diversity and weakly correlated to difficulty; and (3) even after gating, interventions are equally likely to harm correct trajectories as they are to recover incorrect ones. While new semantic information can change the reasoning trajectory, there is no consistent shift towards correctness. Instead, we discover that the rate of distribution shift is more correlated with the model’s confidence, where targeting uncertainty is also likely to cause the model to second-guess correct trajectories. We demonstrate that comparing logprob-based selection to our probe-based scoring, the best question is often not the most probable one. These results demonstrate a gap between hidden state detection and recovery in the reasoning trace. LLMs still struggle to identify flaws in their own logic, even though the signal exists in the hidden state. This gap has broader implications on self-refinement, especially when the model is uncertain, and we encourage further research to improve the model’s inherent ability to self-diagnose or understand its own actions.
6.1 Limitations↩︎
We would like to highlight several limitations of this study. Due to computational limitations, we only sample 5 trajectories as an estimate of the answer distribution. However, this is a coarse estimate of the theoretical JSD value; many works [33] sample up to 64 candidates for a more accurate estimation. While the principles discussed are applicable to non-verifiable domains, the contents of this study are intentionally limited to mathematical reasoning, and we leave exploration for future work.
7 Derivations↩︎
We conceptualize the QA test-time intervention as a sequential decision problem under partial observability, where the goal is to select questions that maximize expected improvement in downstream outcome. We estimate this expected improvement with a hidden state probe, but this section is
We define the utility of a question as the expected improvement in correctness: \[U(q_t; \tau_t) = \mathbb{E}_{a \sim p_T(\cdot \mid \tau_t, q_t)} \left[ \Delta_{\mathrm{acc}}(q_t; \tau_t) \right].\]
The optimal question is therefore the solution to a Value of Information (VoI) problem: \[q_t^\star = \arg\max_{q \in \mathcal{Q}(\tau_t)} \mathrm{VoI}(q; \tau_t),\] where \[\mathrm{VoI}(q; \tau_t) = \mathbb{E}_{a}\left[ \mathbb{E}_{y \sim p_\theta(\cdot \mid \tau_t, q, a)}[\mathbf{1}(y = y^\star)] \right] - \mathbb{E}_{y \sim p_\theta(\cdot \mid \tau_t)}[\mathbf{1}(y = y^\star)].\]
7.0.0.1 Frontloaded vs. Partial-Solution Intervention.
Different intervention regimes correspond to different information sets:
Frontloaded intervention: \[\mathcal{I}_0 = x, \quad q_0^\star = \arg\max_{q \in \mathcal{Q}_\theta(x)} \mathrm{VoI}(q; x).\]
Partial-solution intervention: \[\mathcal{I}_t = \tau_t, \quad q_t^\star = \arg\max_{q \in \mathcal{Q}_\theta(\tau_t)} \mathrm{VoI}(q; \tau_t).\]
While the set of all admissible questions, \(\mathcal{Q}_\theta\), is mostly bound by the student model, the value of a question depends on the underlying belief state. Let \[V(\tau) := \max_{q \in \mathcal{Q}_\theta} \mathrm{VoI}(q; \tau).\] As the trajectory \(\tau_t\) refines the model’s belief, it enables more targeted interventions. We posit the information available in \(\tau_t\) is strictly greater than \(x\) because \(\tau_t\) is a superset of \(x\) (a reminder that \(\tau_t = (x, z_{1:t})\)) and empirically, we observe this leads to greater information gain in 4.
7.0.0.2 Connection to Test-Time Intervention Objective.
Our empirical metric \(\Delta_{\mathrm{acc}}(q; \tau_t)\) is a Monte Carlo estimator of \(\mathrm{VoI}(q; \tau_t)\) under a discrete correctness variable. Thus, maximizing accuracy improvement is approximately maximizing expected value of information under the student’s belief model.
7.0.0.3 Gating as a Feasibility Test for Value of Information.
In practice, computing \(q_t^\star\) is intractable. Instead, we approximate whether any intervention is useful: \[\max_{q \in \mathcal{Q}(\tau_t)} \mathrm{VoI}(q; \tau_t) > 0.\]
We introduce a state signal \(s_t \approx \mathbb{P}(y = y^\star \mid \tau_t)\) and define a gating policy:
\[\pi(\tau_t) = \begin{cases} \pi(\cdot \mid \tau_t), & s_t \le \gamma \\ \varnothing, & \text{otherwise}. \end{cases}\]
Thus, the gating function is not selecting questions directly, but approximating whether the current belief state admits a positive expected value of information.
7.0.0.4 Asymmetric Value of Information.
A key empirical observation is that VoI is highly asymmetric. Let: \[\alpha = \mathbb{P}(s_t \ge \gamma \mid y = y^\star), \quad \beta = \mathbb{P}(s_t \ge \gamma \mid y \neq y^\star).\]
Then we posit intervention is optimally beneficial when the model is more likely to be incorrect: \[\mathbb{P}(y \neq y^\star) \cdot \beta \cdot \mathrm{VoI}^{-} > \mathbb{P}(y = y^\star) \cdot \alpha \cdot |\mathrm{VoI}^{+}|,\] where \(\mathrm{VoI}^{-}\) denotes expected gain when incorrect and \(\mathrm{VoI}^{+}\) denotes expected loss when already correct. The intuitive idea is simple; we do not need to intervene on questions the student already answers well on its own.
7.0.0.5 Interpretation.
From this perspective, our experimental setting can be understood as an instance of Bayesian experimental design where questions act as actions that maximize expected reduction in uncertainty over correctness. The observed superiority of partial-solution intervention follows directly from its access to a richer belief state \(\tau_t\), which enlarges the feasible set of informative experiments. Additionally, there is variance in the stochastic nature of Bayesian experiments. To maximize observed performance gains, we wish to only intervene when the model is weak.
8 Extended Experimental Details↩︎
For the hidden-state probe, we use a 64-dim MLP architecture, trained with a regression objective over self-consistency for the belief state prediction. We trained each probe for 10 epochs and a learning rate of 1e-3, over offline trajectories sampled from a mix of 100 samples of GSM8k, 15 from Math500, and 15 from Big-Bench Extra Hard [34]. We used a special data collection setting where, on the fixed intervention schedule, we sample self-play teacher answers, and five unique rollouts per question. See 6 for the validation accuracy results.
For our gating threshold, we choose \(\gamma\) to be 0.6 after an initial hyperparameter sweep on the training data at the intervals [0.1, 0.3, 0.5, 0.7, 0.9]. We set \(n\) to be 5 reasoning steps (split by newlines, not individual sentences), and \(m\) to be 5. We experimented with \(m = 15\), but find that there are minimal gains beyond 5 interventions in one reasoning trajectory. We measure the efficiency of the “natural” generation for a model — in other words, we allow the model to generate until the EOS token.
All experiments are performed on a cluster of Nvidia H100 80GB GPUs. Experimental time depends on the model and experimental setting, \(\approx\)6-20 GPU hours per experiment run. All experiments were run on vLLM with fp16 precision, including the hidden state probe - we modified vLLM to host the hidden state and feed a dual-head classifier and inference endpoint, so the probe and the token generation shared the same weights and KV cache. For the sake of efficiency in the KV cache we reserved one GPU per model, so one experiment uses 1-3 GPUs depending on the student-teacher model configuration and the presence of a PRM.
For Qwen3, except for the thinking baseline, we use non-thinking mode and the recommended decoding settings: temperature at 0.7, top-p at 0.9, and top-k at 20. Olmo-3-7B-Think has thinking mode on by default, and we also use the recommended sampling settings (temperature=0.6, top-p=0.95). We also include the full system prompts in 4 and user message templates in 5. For evaluation, we used standard math evaluation libraries available through lm-evaluation-harness2. To estimate confidence intervals on our main results we used bootstrapping — we resample with replacement 1000 times, recalculate the majority-vote accuracy, and report the 2.5th and 97.5th percentile as the general confidence interval.
All artifacts used in this work are released in the public domain. Qwen3, Olmo-3, and AIME243 all use Apache 2.0 licenses, while Math5004 and GSM8k5 were released under MIT licenses. Coding agents were used to write code and monitor and re-submit experiments, and a human manually inspected and hand-corrected the generated code. They were not used to formulate methodology or make scientific decisions.
| Prompt Name | Content |
|---|---|
| Default System Prompt (Qwen3 recommended setting) | You are solving a problem step by step. Show your reasoning clearly. Separate each reasoning step with a blank line. Put your final answer within \boxed{}. |
| Continuation | You are continuing your reasoning after receiving feedback on a previous attempt. Incorporate the feedback and continue from where you left off. Separate each reasoning step with a blank line. Put your final answer within
\boxed{}. |
| Student Question | You are working through a logical reasoning problem step by step. You have access to a knowledgeable teacher (oracle) who can answer questions about the problem’s facts and rules. You are reviewing a specific step in your reasoning — ask the teacher a question about that step that will help you answer the overall problem. DO NOT answer the question yourself. Do not ask the teacher for the final answer or whether an answer is correct — the teacher will not respond to such questions. Ask about specific facts, rules, or intermediate steps instead. |
| Teacher | You are a knowledgeable oracle for a logical reasoning problem. Answer the student’s question using ONLY the facts and rules provided. Do not speculate or add information not present in the problem. Your task is to answer any questions the student may have, but DO NOT PROVIDE THE CORRECT ANSWER DIRECTLY. If the student asks you whether an answer is correct, do not respond. Keep your responses short and concise — address only the specific fact, rule, or intermediate step the student raised. |
| Full Answer | You have been exploring a problem step by step and received feedback from a teacher. Based on your exploration and the feedback, produce a complete solution. Show your reasoning clearly and put your final answer within
\boxed{}. |
| Rewrite Step | You are solving a problem step by step. Show your reasoning clearly. Separate each reasoning step with a blank line. Put your final answer within \boxed{}.Important: {teacher_response} |
Dynamic prompt: {teacher_response}
is replaced with the teacher’s answer, stated as a bare fact so the model naturally incorporates it.
| System Prompt | User Message Content |
|---|---|
| Baseline | {task_question} |
| Continuation | Problem: {task_question} Feedback from previous attempt: {prior_feedback} Continue solving the problem, incorporating the feedback above. |
| Student Question | Problem: {task_question} Your reasoning so far: {partial_solution} The step you are reviewing: > {uncertain_step} Ask a probing question that stress-tests this step. |
| Teacher | Full problem: {problem_text} Student’s work so far: {partial_solution} Student’s question: {question} Answer using only the facts and rules above. |
| Full Answer | Problem: {task_question} Your exploration so far: {exploration} Teacher feedback: Q1: {q} / A1: {a} … Based on the exploration and feedback above, determine the answer. Show your complete reasoning and put your final answer within \boxed{}. |
| Rewrite Step | {problem_text} (Prior clean steps appended as generation prefix) |
9 Additional Experimental Results↩︎
9.0.0.1 Intervention strategies
We compare two intervention policy choices for \(\pi(\cdot \mid \tau_t)\) to analyze the effect of available information on the final success:
Backtrack: Framing the intervening step as a “bad” step, we prompt the model to re-generate the step based on the teacher’s feedback but without seeing the original question.
Rewrite: Our full QA intervention. The student has access to their original question, the teacher’s answer, and they re-generate the step with full information.
These results are available in 7.
| Model | Pairs | Frac.wrong | Val MSE | Target var | Var.explained | |
|---|---|---|---|---|---|---|
| Qwen3-1.7B | 511 | 0.881 | 0.049 | 0.122 | 60% | |
| Qwen3-4B | 926 | 0.773 | 0.073 | 0.175 | 58% | |
| Qwen3-14B | 39 | 0.949 | 0.272 | 0.049 | 0% | |
| Qwen3-1.7B | 988 | 0.598 | 0.048 | 0.243 | 80% | |
| Qwen3-4B | 1533 | 0.639 | 0.037 | 0.184 | 80% | |
| Qwen3-14B | 556 | 0.536 | 0.113 | 0.249 | 55% |
5pt
| Backtrack | Rewrite | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| 3-6(lr)7-10 Model | Dataset | Acc | MV | Bt | Tok | Acc | MV | Bt | Tok |
| Qwen3-1.7B | GSM8K | 93.2 | 85.6 | 0.0 | 359 | 91.0 | 86.1 | 0.1 | 318 |
| MATH500 | 80.0 | 73.2 | 0.2 | 1201 | 77.5 | 72.4 | 0.5 | 1260 | |
| AIME24 | 23.3 | 20.0 | 0.8 | 5180 | 23.3 | 13.3 | 1.2 | 4584 | |
| Qwen3-4B | GSM8K | 95.2 | 92.5 | 0.0 | 286 | 95.4 | 92.9 | 0.0 | 296 |
| MATH500 | 86.0 | 80.2 | 0.1 | 1152 | 90.8 | 85.6 | 0.2 | 1104 | |
| AIME24 | 36.7 | 23.3 | 0.0 | 5439 | 33.3 | 30.0 | 1.0 | 5317 | |
| Qwen3-14B | GSM8K | 97.7 | 96.6 | 0.0 | 207 | 96.6 | 95.2 | 0.0 | 200 |
| MATH500 | 85.6 | 84.7 | 0.1 | 1338 | 86.0 | 83.7 | 0.2 | 979 | |
| AIME24 | 50.0 | 30.0 | 0.0 | 4600 | 40.0 | 33.3 | 0.9 | 3915 | |
| Baseline | Front-loaded | Interleaved | ||||||||
| non-thinking | thinking | logprob | probed | random | Fixed | Adaptive | Adaptive (t) | |||
| GSM8K | P@5 | 92.6 | 95.2 | 25.2 | 88.0 | 78.8 | 87.9 | 91.0 | 94.2 | |
| Maj | 84.8 | 91.5 | 24.6 | 85.5 | 76.3 | 84.2 | 86.1 | 92.2 | ||
| MATH500 | P@5 | 81.6 | 95.2 | 67.6 | 77.9 | 70.9 | 76.5 | 75.9 | 96.0 | |
| Maj | 74.8 | 92.8 | 59.6 | 76.3 | 61.9 | 70.1 | 70.9 | 92.6 | ||
| AIME24 | P@5 | 30.0 | 48.3\(^\star\) | 10.0 | – | 0.0 | 20.0 | 23.3 | 66.7 | |
| Maj | 20.0 | 48.3\(^\star\) | 10.0 | – | 0.0 | 13.3 | 13.3 | 53.3 | ||
| GSM8K | P@5 | 96.7 | 96.7 | 69.9 | 93.7 | 46.1 | 97.4 | 95.4 | 96.9 | |
| Maj | 92.7 | 95.2 | 61.3 | 90.3 | 42.2 | 95.3 | 93.0 | 96.1 | ||
| MATH500 | P@5 | 87.6 | 98.0 | 43.1 | 80.8 | 12.6 | 83.1 | 82.6 | 98.5 | |
| Maj | 82.1 | 96.7 | 34.6 | 73.6 | 10.9 | 77.8 | 77.6 | 98.5 | ||
| AIME24 | P@5 | 33.3 | 80.0 | 66.7 | – | 64.3 | 39.3 | 33.3 | 80.0 | |
| Maj | 33.3 | 76.7 | 60.0 | – | 57.1 | 28.6 | 30.0 | 80.0 | ||
| GSM8K | P@5 | 95.0 | 97.6 | 51.0 | 95.0 | – | 95.8 | 96.6 | 97.9 | |
| Maj | 91.6 | 96.6 | 47.2 | 91.6 | – | 95.4 | 95.2 | 96.6 | ||
| MATH500 | P@5 | 88.8 | 98.8 | 59.9 | 84.3 | 12.8 | 86.9 | 83.9 | 98.5 | |
| Maj | 87.0 | 97.1 | 50.6 | 79.8 | 12.0 | 81.9 | 81.4 | 97.3 | ||
| AIME24 | P@5 | 46.4 | 80.0 | 66.7 | – | – | 50.0 | 40.0 | 90.0 | |
| Maj | 32.1 | 80.0 | 61.9 | – | – | 40.0 | 33.3 | 86.7 | ||
| GSM8K | P@5 | – | 98.0 | – | – | – | 96.9 | – | 97.7 | |
| Maj | – | 94.9 | – | – | – | 95.7 | – | 95.5 | ||
| MATH500 | P@5 | – | 94.3 | – | – | – | 98.3 | – | 98.0 | |
| Maj | – | 91.2 | – | – | – | 97.2 | – | 97.6 | ||
| AIME24 | P@5 | – | 76.7 | – | – | – | 86.7 | – | 83.3 | |
| Maj | – | 76.7 | – | – | – | 73.3 | – | 76.7 | ||
| Model | Spearman \(\rho\) | \(p\) | Pearson \(r\) | \(p\) | Mean iter. |
|---|---|---|---|---|---|
| Qwen3-1.7B | 0.143 | \(1.3\!\times\!10^{-3}\) | 0.132 | \(3.1\!\times\!10^{-3}\) | 1.23 |
| Qwen3-4B | 0.165 | \(2.2\!\times\!10^{-4}\) | 0.135 | \(2.5\!\times\!10^{-3}\) | 1.10 |
| Qwen3-14B | 0.137 | \(2.2\!\times\!10^{-3}\) | 0.084 | \(6.0\!\times\!10^{-2}\) | 1.15 |
| Olmo-3-7B-Think | 0.179 | \(5.8\!\times\!10^{-3}\) | 0.149 | \(2.2\!\times\!10^{-2}\) | 1.07 |
5pt
| Model | \(n\) | \(\rho\) (cons.) | \(\rho\) (correct) | Brier | ECE |
|---|---|---|---|---|---|
| Olmo-3-7B-Think | 532 | +0.203\(^{***}\) | +0.205\(^{***}\) | 0.177 | 0.109 |
| Qwen3-1.7B | 1562 | +0.213\(^{***}\) | +0.247\(^{***}\) | 0.197 | 0.103 |
| Qwen3-4B | 1767 | +0.269\(^{***}\) | +0.237\(^{***}\) | 0.209 | 0.157 |
| Qwen3-14B | 857 | +0.362\(^{***}\) | +0.331\(^{***}\) | 0.195 | 0.154 |
| Adaptive | Forced | |||||||
|---|---|---|---|---|---|---|---|---|
| 2-5 (lr)6-9 | Mean step | Trajectory | Mean step | Trajectory | ||||
| 2-3 (lr)4-5 (lr)6-7 (lr)8-9 Model | \(\rho\) | \(p\) | \(\rho\) | \(p\) | \(\rho\) | \(p\) | \(\rho\) | \(p\) |
| Qwen3-1.7B | 0.178 | \(1.4\!\times\!10^{-3}\) | 0.171 | \(2.2\!\times\!10^{-3}\) | 0.359 | \(8.9\!\times\!10^{-12}\) | 0.377 | \(6.7\!\times\!10^{-13}\) |
| Qwen3-4B | 0.182 | \(1.6\!\times\!10^{-3}\) | 0.232 | \(5.5\!\times\!10^{-5}\) | 0.347 | \(1.6\!\times\!10^{-12}\) | 0.366 | \(8.0\!\times\!10^{-14}\) |
| Qwen3-14B | 0.209 | \(6.2\!\times\!10^{-4}\) | 0.201 | \(1.0\!\times\!10^{-3}\) | 0.463 | \(1.2\!\times\!10^{-6}\) | 0.477 | \(5.1\!\times\!10^{-7}\) |
| Olmo-3-7B-Think | 0.193 | \(5.3\!\times\!10^{-3}\) | 0.225 | \(1.1\!\times\!10^{-3}\) | 0.232 | \(1.6\!\times\!10^{-2}\) | 0.221 | \(2.2\!\times\!10^{-2}\) |