1 Introduction↩︎

Large language models (LLMs) have demonstrated remarkable reasoning capabilities through chain-of-thought (CoT) prompting, where models generate intermediate reasoning steps before producing final answers [1]. Recent systems such as OpenAI’s o1 and DeepSeek R1 have shown that scaling test-time compute, allowing models to “think longer”, can dramatically improve performance on complex reasoning tasks [2], [3]. At the same time, it has been observed that simply increasing test-time computation can harm performance, a phenomenon known as overthinking: reasoning length does not directly convert to correct answers [4]. In general, one expects a model to engage in more deliberate reasoning for harder tasks and less for easier ones. Recent theoretical work has further shown that, for transformers trained on an in-context weight prediction task for linear regression, increasing test-time compute can harm performance when the skills required to solve the downstream task are insufficiently represented in the training data [5].

Despite these theoretical and empirical advances, fundamental questions remain: What makes a task hard for a general LLM? How does task difficulty affect the model’s internal representations, and can we predict it from those representations alone? Can we identify promising reasoning paths early, before generation is complete? These questions have profound practical implications: when generating \(n\) candidate solutions to a problem (best-of-\(n\) sampling), can we prioritize which paths to pursue based on the geometry of their early trajectories?

In this paper, we address these questions by studying the geometry of chain-of-thought trajectories in the hidden state space of transformer models. Our key insight is that as an LLM generates tokens during reasoning, the sequence of hidden states traces a discrete curve in \(\mathbb{R}^d\), and the geometric properties of this curve encode information about both task difficulty and solution quality.

Our contributions are as follows:

• Formal Framework for CoT Geometry (Section 3): We formalize CoT reasoning as a discrete curve in \(\mathbb{R}^d\) and introduce geometric functionals that extract spectral, positional, and kinematic properties of reasoning trajectories.

• Effective Dimension as Task Complexity (Section 4): We introduce a geometrical function capturing hardness of the task. More precisely, the effective dimension \(d_\rho\) of reasoning curves as a principled measure of task hardness. We further, characterize which curves attain the highest effective dimension, establishing them as geometric representatives of the hardest tasks.

• Hardness Prediction (Section 5.3): Using only effective dimension features, we achieve AUC \(> 0.93\) in predicting whether a mathematical problem is easy or hard.

• Correctness Prediction (Section 5.2): Seven kinematic and positional features of the trajectory predict solution correctness with AUC \(= 0.806\) from only the first \(20\%\) of generated tokens, with promising implications for early-exit strategies and best-of-\(n\) ranking.

2 Related Work↩︎

Chain-of-thought prompting [1], [6] has emerged as a powerful technique for eliciting multi-step reasoning in LLMs. Recent work has explored scaling test-time compute [7]–[9], with systems like OpenAI o1 [2] and DeepSeek R1 [3] demonstrating strong performance through extended reasoning chains. A complementary line of work has observed that more reasoning is not always better: overthinking can degrade performance when the skills required for a task are underrepresented in training [4]. Our work studies these phenomena from a geometric angle, asking not how long a chain is but what shape it traces in hidden state space.

[5] provide a theoretical analysis of test-time scaling for transformers trained on in-context weight prediction for linear regression. They characterize task hardness via the ratio of the trace with the minimum eigenvalue of the feature covariance matrix, showing that harder tasks require longer chains-of-thought to reach a given error level, and that insufficient task coverage in training can cause additional reasoning steps to hurt performance. Our work is complementary but distinct in two ways. First, we study task hardness empirically in a general LLM rather than deriving it from a tractable linear model. Second, and more fundamentally, we shift the unit of analysis from the output chain to the internal hidden state trajectory: we show that task hardness leaves a geometric signature in the model’s representation space, captured by the effective dimension \(d_\rho\) of the trajectory covariance, and that this quantity alone is highly predictive of problem difficulty

[10] argue that chain-of-thought reasoning offers a unique safety opportunity because, for sufficiently hard tasks, transformers must externalize reasoning through the CoT in order to complete it, making that reasoning in principle observable. They focus on the content of the generated text as the monitoring signal and discuss conditions under which this signal may degrade. Our work operates at a different level: rather than reading the textual content of the chain, we read the geometry of the hidden states that produce it. The two perspectives are complementary, CoT text monitoring and hidden-state trajectory analysis can in principle be combined, but our approach is model-internal and does not rely on the model producing legible natural language reasoning.

[11] study LLM reasoning as a structured trajectory in representation space, extracting hidden states at explicit step boundaries (“Step 1:”, “Step 2:”, …) and showing that these activations form linearly separable, step-specific subspaces that become more pronounced with layer depth. For correctness prediction, they achieve high AUC using late-step trajectory features, and explore inference-time interventions such as activation steering [12] to correct deviating trajectories. Our work shares the trajectory perspective but pursues different goals. Rather than analyzing step-boundary activations, we treat the full token-level hidden state sequence as a continuous curve and characterize it through spectral and kinematic geometric functionals. This allows us to ask whether trajectory geometry encodes task difficulty. We show that the effective dimension \(d_\rho\) of the trajectory covariance, a spectral property of the curve as a whole, predicts whether a problem is easy or hard with high AUC, and we provide a theoretical account of why harder tasks necessarily induce higher-dimensional trajectories. We further show that kinematic features of the trajectory carry an early correctness signal that is detectable from only the first \(20\) percent of generated tokens, opening a practical route to early stopping and best-of-\(n\) ranking without waiting for generation to complete.

Recent work also has proposed geometric frameworks for understanding how LLMs reason. [13] model reasoning as smooth flows in representation space, using the velocity and Menger curvature of the trajectory to show that logical structure, rather than surface semantics, governs the direction and magnitude of these flows. Their focus is on interpretability. Our work takes a complementary direction: we use geometric functionals of the hidden-state trajectory, specifically the spectral effective dimension and kinematic summaries, to predict task difficulty and solution correctness, connecting trajectory geometry directly to downstream performance.

Lastly, [14] show that effective reasoning strategies reduce the intrinsic dimensionality of the learning objective, measured as the minimum number of LoRA parameters needed to fine-tune a model to a given accuracy threshold on GSM8K. They fix the model and vary the reasoning strategy, finding that lower intrinsic dimensionality correlates strongly with better generalization. While both their work and ours use notions of dimensionality to characterize reasoning, the two measures are conceptually distinct. Their intrinsic dimension is a property of the learning problem induced by a reasoning strategy, it requires fine-tuning experiments and measures how compressible a dataset of reasoning chains is. Our effective dimension \(d_\rho\) is a property of a single inference trajectory, it is computed from the covariance of hidden states produced during one forward pass and requires no training. This makes our measure applicable at inference time and enables per-instance predictions of task difficulty and solution correctness.

3 Problem Formulation↩︎

In this section we formalize Chain-of-Thought reasoning and develop a mathematical framework for characterizing its dynamics via Geometrical Functionals. Consider a transformer language model with \(L\) layers and hidden dimension \(d\). Let \(\mathcal{V}\) denote the finite vocabulary, and let \(\mathcal{V}^* := \bigcup_{n=0}^{\infty} \mathcal{V}^n\) denote the set of all finite sequences over \(\mathcal{V}\) (the Kleene star of \(\mathcal{V}\)). Let \(\Delta(\mathcal{V})\) denote the simplex of probability measures over \(\mathcal{V}\). The model defines a map from finite token sequences to probability measures over the next token: \[\mu: \mathcal{V}^* \to \Delta(\mathcal{V}), \quad (x_1, \ldots, x_t) \mapsto \mu(\cdot \mid x_1, \ldots, x_t).\]

As such for each layer \(\ell \in \{1, \ldots, L\}\), the model also produces a hidden state representation in \(\mathbb{R}^{d}\) where \(d\) is the dimension of the latent representation, i.e.,: \[f^{(\ell)}: \mathcal{V}^* \to \mathbb{R}^d, \quad (x_1, \ldots, x_t) \mapsto H_t^{(\ell)}\in \mathbb{R}^{d}.\]

When the layer \(\ell\) is fixed or clear from context, we write \(H_t \equiv H_t^{(\ell)}\). Given the distribution \(\mu(\cdot \mid x_1, \ldots, x_t) \in \Delta(\mathcal{V})\), the next token is selected according to a temperature parameter \(T \geq 0\). In particular, At temperature \(T > 0\), we sample from a tempered distribution: \[X_{t+1} \sim \mu_T(\cdot \mid x_1, \ldots, x_t), \quad \text{where } \mu_T(x) \propto \mu(x)^{1/T}.\]

At temperature \(T = 0\), the distribution concentrates on the mode: \[X_{t+1} = \arg\max_{x \in \mathcal{V}} \mu(x \mid x_1, \ldots, x_t).\] This distinction is fundamental: at \(T = 0\), given a prompt, the generated sequence is unique; at \(T > 0\), the same prompt yields a distribution over sequences.

3.1 The Space of CoT Curves↩︎

At \(T = 0\), token selection is deterministic. Given a prompt, there is exactly one generated sequence of tokens, this motivates the following definition of the space of discrete curves,

We have a natural embedding of \(\mathcal{C}_{m} \subseteq \mathcal{C}_n\) for \(m \leq n\) by repeating the last element \(n-m\) times (in practice, we do not apply this padding but instead work directly with variable-length trajectories). Let \(\mathcal{P} \subset \mathcal{V}^*\) denote the space of input prompts. At \(T = 0\), the model defines a deterministic map from the set of prompts to the space of curves. In particular, we have: \[\mathcal{H}: \mathcal{P} \to \mathcal{C}_n, \quad P \mapsto (h_1, \ldots, h_m, h_m, \ldots, h_m),\] where \(m\) is the generation length and the final state \(H_m\) is repeated to fill length \(n\) (stationary extension). As such we introduce the length \(n\) chain of thought as an element in the space of discrete curves of length \(n\). In other words, given a \(\omega \in \mathcal{V}^*\) the length-\(n\) Chain-of-Thought curve is the element \(\mathcal{H}(\omega) = (h_1(\omega), \ldots, h_n(\omega)) \in \mathcal{C}_n\) produced by generation from \(\omega\), with stationary extension if necessary. We characterize CoT curves through real-or vector valued functionals, in particular:

As an example, for a curve \(\gamma = (h_1, \ldots, h_n) \in \mathcal{C}_n\), define the centered curve with elements \(\bar{h}_t = h_t - \frac{1}{n}\sum_{i=1}^n h_i\) and its trajectory covariance matrix, defined as \(C(\gamma) := \frac{1}{n} \sum_{t=1}^{n} \bar{h}_t \bar{h}_t^\top = \frac{1}{n} \bar{H}^\top \bar{H} \in \mathbb{R}^{d \times d},\) where \(\bar{H} = [\bar{h}_1, \ldots, \bar{h}_n]^\top \in \mathbb{R}^{n \times d}\). As such \(C\) is a geometrical functional that takes a curve and produces an element in \(\mathbb{R}^{d^2}\).

Let \(\nu_1 \geq \nu_2 \geq \cdots \geq \nu_d \geq 0\) be the eigenvalues of \(C(\gamma)\) ordered in a non-increasing order, then we introduce another such important geometrical functional as follows:

The effective dimension measures the intrinsic dimensionality of the reasoning trajectory. A low \(d_\rho\) indicates the trajectory lies near a low-dimensional subspace (simple, structured reasoning), while high \(d_\rho\) indicates the trajectory explores many directions (complex, multi-faceted reasoning).

Lastly, we introduce seven additional geometric functionals, which we use in Section 5.2 for correctness prediction. Let \(\gamma = (h_1, \ldots, h_n) \in \mathcal{C}_n\) be a trajectory, and let \(P \in \mathbb{R}^{d \times k}\) denote the top-\(k\) PCA basis fitted on the training set. Define the projected trajectory with elements \(\tilde{h}_t = P^\top h_t \in \mathbb{R}^k\) for \(t = 1, \ldots, n\), and let \(\tilde{\gamma} = (\tilde{h}_1, \ldots, \tilde{h}_n)\) be the projected curve. Note that to prevent data leakage, when a fraction \(\alpha \in (0,1]\) of the trajectory is observed, we restrict to the window \(\tilde{\gamma}^{(\alpha)} = (\tilde{h}_1, \ldots, \tilde{h}_{\lfloor \alpha n \rfloor})\). Noting that our framework is more general and one can extract many other meaningful functional as needed, we define the following functionals \(\varphi: \mathcal{C}_n \to \mathbb{R}^{5k+2}\) below.

Examples (Kinematic and Positional Geometric Functionals). Given a projected trajectory \(\tilde{\gamma} = (\tilde{h}_1, \ldots, \tilde{h}_m)\) for some \(m \leq n\), define the velocity increments \(\Delta_t := \tilde{h}_{t+1} - \tilde{h}_t \in \mathbb{R}^k\) for \(1 \leq \forall t \leq m-1\). The seven geometric functionals are:

1. Mean position, defined as \(\mu(\tilde{\gamma}) \;=\; \frac{1}{m}\sum_{t=1}^{m} \tilde{h}_t \;\in\; \mathbb{R}^k.\)

2. Positional dispersion, the coordinate-wise standard deviation: \[\sigma(\tilde{\gamma}) \;=\; \left(\frac{1}{m}\sum_{t=1}^{m} \bigl(\tilde{h}_t - \mu(\tilde{\gamma})\bigr)^{\odot 2} \right)^{\odot 1/2} \;\in\; \mathbb{R}^k,\] where \(\odot\) denotes elementwise operations.

3. Initial hidden state, the first token representation of the projected trajectory, i.e., \(\tilde{h}_1 \;\in\; \mathbb{R}^k.\)

4. Final hidden state, the last token representation of the projected trajectory, i.e., \(\tilde{h}_m \;\in\; \mathbb{R}^k.\)

5. Mean velocity, the average of successive differences: \[\bar{v}(\tilde{\gamma}) \;=\; \frac{1}{m-1}\sum_{t=1}^{m-1} \Delta_t \;\in\; \mathbb{R}^k.\]

6. Mean speed, the average step-wise Euclidean norm: \[\bar{s}(\tilde{\gamma}) \;=\; \frac{1}{m-1}\sum_{t=1}^{m-1} \|\Delta_t\|_2 \;\in\; \mathbb{R}.\]

7. Speed dispersion, the standard deviation of step-wise speeds: \[\sigma_s(\tilde{\gamma}) \;=\; \left(\frac{1}{m-1}\sum_{t=1}^{m-1} \bigl(\|\Delta_t\|_2 - \bar{s}(\tilde{\gamma})\bigr)^2 \right)^{1/2} \;\in\; \mathbb{R}.\]

The full feature vector is the concatenation \[\varphi(\tilde{\gamma}) \;=\; \Bigl(\mu,\;\sigma,\;\tilde{h}_1,\;\tilde{h}_m,\; \bar{v},\;\bar{s},\;\sigma_s\Bigr) \;\in\; \mathbb{R}^{5k+2}.\]

Note that mean velocity telescopes to \(\frac{1}{m-1}(\tilde{h}_m - \tilde{h}_1)\), making it a linear function of the already-included initial and final states. We retain it for completeness and its natural connection to mean speed and speed dispersion.

At temperature \(T > 0\), the same prompt could yields different curves at each generation time. Our formulation can extend to this framework: \[\mathcal{H}: \mathcal{V}^* \to \Delta(\mathcal{C}_n),\] where \(\Delta(\mathcal{C}_n)\) denotes distributions over curves. The induced distribution arises from the autoregressive measure: \[\mathbb{P}_{\mu_T}(v_1, \ldots, v_n) = \prod_{t=1}^{n} \mu_T(v_t \mid x_1, \ldots, x_k, v_1, \ldots, v_{t-1}).\]

4 Effective Dimension as Task Complexity↩︎

This section develops the theoretical core of the paper. We prove general spectral bounds for any covariance matrix, showing via a majorization argument that flat spectra maximize effective dimension (§4.1) and provide finite-sample stability. All proofs are in Appendix 7.

4.1 Spectral Bounds and the Role of Flatness↩︎

The effective dimension of any PSD matrix is controlled by its eigenvalue spread. These are purely linear-algebraic facts, independent of any dynamical model.

The ratio \(T/\nu_d\) is similar (they used covariance of the data rather than the dynamic) to the hardness measure of [5]; the upper bound shows it controls \(d_\rho\), but loosely. When the spectrum is flat, the bounds coincide at \(\lceil \rho d\rceil\). Flatness is extremal in a stronger sense, captured by majorization.

Majorization gives us a precise way to compare how “peaked” two spectra are. We use it to show that the flat spectrum \(\nu^\star\) is the most spread-out among all spectra with the same trace, and achieves the highest effective dimension. More formally we have:

4.2 Stability under small Perturbation↩︎

The following framework-independent and purely linear-algebraic result shows that if two covariance matrices are \(\epsilon\)-close in operator norm(which is \(\epsilon\)-close in operator norm (\(\|\widehat{C} - C\|_{\mathrm{op}} := \max_{\|v\|=1}\|(\widehat{C}-C)v\|\)), their effective dimensions are also close.

5 Experiments↩︎

5.1 Experimental Setup↩︎

5.1.0.1 Model.

We use Qwen2.5-0.5B-Instruct [16], a decoder-only transformer with 24 layers and hidden dimension 896. Despite its small size, it produces well-structured reasoning chains and allows us to extract hidden states at every layer and token position without prohibitive memory cost.

5.1.0.2 Dataset.

We focus on three categories from the MATH500 dataset [17]: Algebra and Counting & Probability and Precalculus. Problems in the MATH500 dataset are labeled with difficulty annotations ranging from \(1\)-\(5\). We consider the problems with annotations of \(1\), \(3\) and \(5\), which we respectively label as easy, medium and hard.

We use a fixed set of \(9\) (probability had \(2\) easy question rather than \(3\)) questions per category, drawn to ensure a balanced difficulty split. In the case of comparing effective dimension for task difficulty, we only use questions labeled as easy or hard (i.e. annotated as \(1\) or \(5\)).

5.1.0.3 Trajectory Collection.

For each problem, we generate 10 reasoning trajectories at temperature \(T = 0.7\) using two chain-of-thought prompting styles (medium and long which, which are provided in the appendix), pooled for analysis. Each trajectory is generated autoregressively with a maximum of 800 tokens. We extract the hidden state \(h_t^{(\ell)} \in \mathbb{R}^{896}\) at every generated token position \(t\) for all layers \(\ell \in \{0, \ldots, 23\}\), yielding a trajectory matrix \(H^{(\ell)}\) per layer per run. Correctness is determined by a symbolic answer checker combining SymPy expression matching and string normalization against the ground-truth boxed answer.

5.1.0.4 Features for Correctness Prediction.

For each trajectory, we project \(H^{(\ell)}\) onto the top 15 PCA components fitted on the training set, then extract seven kinematic and positional features from the windowed portion of the projected trajectory (which are defined in section 3.1): mean position, positional dispersion (standard deviation), initial hidden state, final hidden state of the truncated trajectories, mean velocity (mean of successive differences), mean speed (mean of step-wise norms), and speed dispersion. Features are standardized before classification.

5.1.0.5 Evaluation Protocol.

For correctness prediction, we use a stratified question-level 80/20 train/test split, repeated over 5 random seeds, ensuring that all trajectories from a given question appear entirely in train or entirely in test. We report AUC-ROC (Area Under the Receiver Operating Characteristic curve) and AUPRC (Area Under the Precision–Recall Curve) with \(\pm\) std across splits. For classifiers we use logistic regression (LR; \(\ell_2\) regularization \(C = 0.1\)), a two-layer MLP (hidden sizes 64 and 32, early stopping, \(\ell_2\) regularization \(\alpha = 0.01\)), and a two-layer GRU (hidden dim 64). We highlight AUPRC because our central question is whether trajectory geometry can rank correct solutions above incorrect ones. AUPRC summarizes this ranking through precision (how many of the trajectories flagged as correct actually are) and recall (how many of the truly correct trajectories are recovered) across all thresholds. This directly reflects our intended use, deciding which of the many candidate trajectories for a given question to continue, since a high-precision, high-recall score lets us concentrate on promising trajectories and potentially reach a correct solution faster.

5.2 Experiment 1: Correctness Prediction from Trajectory Geometry↩︎

We ask whether the geometry of a reasoning trajectory predicts whether it will reach a correct answer, and how early in generation this signal emerges.

5.2.0.1 Cross-question generalization.

Figure 2 shows AUPRC and AUC-ROC results for correctness prediction as a function of the fraction of the trajectory observed, for the LR, MLP, and GRU classifiers, under a stratified question-level 80/20 split averaged over 5 seeds. LR achieves AUC \(= 0.806 \pm 0.132\) at \(20\%\) observation and \(0.839 \pm 0.111\) at \(100\%\), with AUPRC remaining stable around \(0.868\) throughout. MLP reaches AUC \(= 0.723 \pm 0.154\) at \(20\%\), dips at \(30\%\), then recovers to \(0.828 \pm 0.093\) at \(100\%\). The GRU behaves qualitatively differently: it achieves AUC \(= 0.641 \pm 0.215\) at \(20\%\), peaks at \(0.754 \pm 0.078\) at \(70\%\), and then drops to \(0.577 \pm 0.144\) at \(100\%\), barely above chance.

5.2.0.2 Within-question prediction.

To understand the ceiling of the correctness signal, we also evaluate a setting where trajectories from the same question can appear in both train and test, while ensuring no trajectory is included in both (i.e. there is no data leakage, but rather leakage at the question level) with 5-fold cross-validation over the set of all pooled trajectories. Here LR achieves AUC \(\approx 0.90\) and MLP achieves AUC \(\approx 0.91\) using only \(20\%\) of the trajectory, with negligible improvement as more tokens are observed (Table 1). This confirms that the geometric signal for correctness is saturated very early, and that the gap between within-question (\(0.90\)) and cross-question (\(0.81\)) performance represents the portion of the signal that is question-specific rather than universally transferable, or that our sample size was not high enough to generalize to such an extent.

5.3 Experiment 2: Difficulty Prediction via Effective Dimension↩︎

We ask whether the effective dimension \(d_\rho\) of the hidden-state trajectory can predict whether a problem is easy or hard (MATH500 difficulty of \(1\) vs. \(5\)), without any access to the answer or the model’s output. For each problem, at layer \(\ell = 12\), we compute \(d_\rho(H^{(\ell)}, \rho)\) at \(\rho \in \{0.90, 0.95, 0.99\}\), yielding a three-dimensional feature vector per trajectory. We then run leave-one-question-out cross-validation: for each test question, we train an LR or MLP classifier on the effective dimension features from all other questions, and predict whether the test question is easy or hard.

5.3.0.1 Results.

Figure 3 (left) shows AUC-ROC as a function of layer for both classifiers. Effective dimension is predictive at every layer, with AUC rising from \(0.81\) at layer 0 to \(0.93\) at layer 21. Prediction is consistent and robust: even the earliest layers achieve AUC well above chance, and the signal strengthens through the network. The best layer is layer 21 (MLP AUC \(= 0.93\), accuracy \(= 86.8\%\)). Figure 3 (right) shows the distribution of \(d_\rho\) at layer 21 (\(\rho = 0.95\)) for easy and hard problems. The separation is apparent: hard problems have mean effective dimension \(\mu \approx 170\) versus \(\mu = 121.6\) for easy problems, a \(40\%\) gap, with a t-test \(p\)-value of \(5.79 \times 10^{-166}\). Hard trajectories explore a substantially higher-dimensional subspace of the model’s representation space, providing strong evidence for the relationship between effective dimension and task hardness.

6 Conclusion↩︎

We study the geometry of chain-of-thought reasoning trajectories in transformer hidden state space. Formalizing each reasoning chain as a discrete curve in \(\mathbb{R}^d\), we introduced the effective dimension \(d_\rho\) as a spectral measure of trajectory complexity and show theoretically that harder tasks necessarily induce higher-dimensional trajectories. Empirically, \(d_\rho\) predicts problem difficulty with AUC \(> 0.93\) via leave-one-question-out cross-validation, with hard problems exhibiting \(40\%\) higher effective dimension on average than easy ones. For correctness prediction, seven kinematic features of the trajectory achieve AUC \(= 0.806\) from only the first \(20\%\) of generated tokens under a question-level split, with a simple logistic regression outperforming a GRU on the full sequence, suggesting there are interesting signal lies in coarse geometric structure. Together, these results establish trajectory geometry as a practical window into both task hardness and solution quality.

Limitations. Our experiments use a single small model and three MATH500 categories, so generalization to larger models and other domains remains open. Our theoretical analysis connects spectral flatness to effective dimension but does not fully explain how task difficulty induces this flatness in a trained transformer. Finally, our correctness prediction is evaluated in terms of AUC; translating this into concrete early-stopping or best-of-\(n\) gains is left for future work.

Acknowledgments↩︎

AJ was supported in part by the NSF Award DMS-2311024, an Amazon Faculty Research Award, an Adobe Faculty Research Award, and an iORB grant form USC Marshall School of Business.

7 Proofs↩︎

In the following proofs, for simplicity of the notations we used \(T := \mathrm{tr}(C)\).

7.1 Proof of Proposition 1↩︎

7.2 Proof of Proposition 2↩︎

7.3 Proof of Theorem 3↩︎

8 Experimental Details↩︎

8.0.0.1 Hardware.

Trajectory collection and analysis were run on NVIDIA A100 GPUs (40 GB HBM2) via a SLURM cluster. Collection jobs used 16 GB of CPU RAM per job; effective dimension analysis used 24 GB; correctness prediction ran on CPU-only nodes with 32 GB of RAM.

8.0.0.2 Hyperparameters.

• Temperature: \(T = 0.7\)

• Maximum tokens: 800

• Runs per question: 10–15

• Minimum tokens for valid trajectory: 30

• \(\rho\) values: \(\{0.90, 0.95, 0.99\}\)

8.0.0.3 Difficulty Prediction Classifier Details.

• Logistic Regression: \(C = 1.0\), max iterations = 1000

• MLP: Hidden layers \((32, 16)\), early stopping with 15% validation, \(\ell_2\) regularization \(\alpha = 0.01\)

8.0.0.4 Correctness Prediction Classifier Details.

• Logistic Regression: \(C = 0.1\), max iterations = 1000, balanced class weights

• MLP: Hidden layers \((64, 32)\), early stopping with 15% validation, \(\ell_2\) regularization \(\alpha = 0.01\), balanced sample weights

• GRU: 2-layer, hidden dim 64, dropout 0.3, Adam optimizer (lr=\(1e-3\), weight decay=\(1e-4\)), 30 epochs, batch size 32, BCE loss with positive class weighting, gradient clipping norm \(1.0\)

8.0.0.5 Prompts.

We use three chain-of-thought prompting styles. The medium and long styles are pooled for all experiments reported in the main paper. The short style is included for completeness.

References↩︎