1 Introduction↩︎

1.1 Autoregressive language models do not distinguish computation from memory↩︎

Autoregressive language models do not distinguish tokens used for computation from tokens used for memory: once a reasoning token is generated, it is appended to the context and becomes input to future predictions. This default makes long reasoning traces both a source of computational burden and a source of spurious dependence. Downstream predictions may depend on failed attempts, dead ends, or private scratch work that should not enter the model’s persistent state.

Recent evidence suggests that longer chains of thought are not always beneficial: some reasoning tokens provide no positive contribution to final or downstream correctness and may even interfere with future predictions (see §2.1). This suggests that retaining every generated reasoning token as context is not a reasonable memory mechanism.

1.2 The state commitment problem and the internalization gap↩︎

We recast the above phenomenon as a training-objective problem that has been less explicitly modeled. Reasoning models lack a state commitment mechanism: after using hidden computation, a model should explicitly commit the information needed for future reasoning into a persistent answer state while discarding the remaining temporary computation.

Existing mitigations mostly operate at inference time on already generated thought traces, such as eviction, pruning, compression, or selecting shorter traces. They do not directly train the model itself to decide what is persistent and what is temporary. Their common blind spot is that they never optimize the property that “answer \(A\) remains a usable state interface after hidden thought \(H\) is erased.” We call this gap the internalization gap.

1.3 Method overview↩︎

The core distinction from prior work can be summarized as follows: compression asks how to preserve the past trace; state commitment asks what the future needs. Compression treats the reasoning trace as a past object to be compactly preserved. State commitment treats the visible answer as a future-facing state interface that must be explicitly committed.

We instantiate this idea with Counterfactual Erasure RL (CERL). Under the same problem prefix, CERL evaluates matched continuations in parallel: (i) a full-thought path that keeps hidden thoughts, (ii) an erasure path that progressively deletes generated hidden thoughts, and (iii) a skip baseline used to audit recomputation. The main reward is tied to erasure-path correctness, with auxiliary length and anti-postponement controls. This signal directly captures whether answer \(A\) can support downstream reasoning when hidden thought \(H\) is no longer visible.

Because prior work has not made this property a training objective, naively dropping thoughts at inference time often collapses accuracy. Training under counterfactual erasure instead lets the answer state itself serve as the persistent interface while preserving accuracy.

1.4 Contributions↩︎

Our contributions follow five levels: a new problem, a new objective, a new method, a new protocol, and new empirical findings.

(1) New problem definition. We propose State Commitment Learning, which captures the missing training objective that reasoning models should distinguish temporary computation from persistent state that can be relied on in the future. We give a concrete setting, Post-Answer Hidden-Thought Erasure: the model alternates between hidden thoughts and visible answer states; each thought is progressively erased when the next thought begins; the final context retains only visible answers. We make no a priori assumption about the content relation between \(A\) and \(H\); it emerges from the training objective.

(2) New counterfactual objective: persistent-state sufficiency. We define the counterfactual criterion of persistent-state sufficiency: under matched full-thought / erasure paths, erasing hidden thought should not reduce downstream correctness. This turns whether an answer has been committed into a future-reliable state into an optimizable and estimable condition; the paired counterfactual paths are formalized in Definition 1.

(3) New training algorithm: CERL+HSCO. We propose Counterfactual Erasure RL as an optimization algorithm for persistent-state sufficiency, implemented as Hierarchical State-Commitment Optimization (HSCO): two GRPO layers train the hidden-thought policy \(H_1\) and the answer state \(A_1\) over non-overlapping token ranges.

(4) New evaluation protocol: erasure dependence. We propose the Erasure Dependence Protocol. It reports four sufficiency metrics after erasure: Answer Sufficiency Gap (ASG, the accuracy gap between full and erasure paths, lower is better), Hidden Thought Dependency Rate (HTDR, the fraction of examples where the full path is correct but the erasure path is wrong, lower is better), Erasure Success Rate (ESR, conditional correctness after erasure given full-path correctness, higher is better), and Answer Interface Sufficiency (AIS, the erasure-path accuracy divided by full-path accuracy, higher is better). We also report Marginal State Gain (MSG), \[\mathrm{MSG} \;=\; \mathrm{Acc}(x + A_1) \;-\; \mathrm{Acc}(x + \texttt{empty}) \;=\; \mathrm{Acc}_{\mathrm{pe}} - \mathrm{Acc}_{\mathrm{skip}} .\] MSG rules out the narrow explanation that the erasure path succeeds by ignoring \(A_1\) and recomputing from \(x\); it should still be interpreted together with downstream length control and visible-CoT leakage audits.

(5) Empirical findings. Our experiments are designed to show that correctness-only reinforcement learning is insufficient for learning persistent-state sufficiency, while counterfactual erasure training substantially closes ASG without sacrificing accuracy and improves state management in BFCL-v3 multi-turn tool use. Long-answer SFT is used as a diagnostic control for the weaker alternative explanation that supervising longer visible answers alone naturally yields the same erasure-dependence signature; it is not used as a standalone causal attribution.

2 Related work↩︎

2.1 Long-CoT degradation as motivation↩︎

A growing body of work reports that longer chains of thought can be worse [1]–[4]. These observations motivate our central question: if many reasoning tokens do not help final or downstream reasoning, should every generated thought token be retained as future context?

2.2 Inference-time pruning and compression do not internalize the boundary↩︎

Existing methods mostly manage already generated traces through pruning, eviction, compression, selection, or external memory. These approaches ask how to control or preserve past reasoning traces. Our question is different: whether the model can be trained so that the visible answer state \(A\) remains a sufficient downstream interface after hidden thought \(H\) is erased. Detailed positioning is in Appendix 19.

2.3 Long-answer supervision is not state commitment↩︎

Long-answer SFT is a useful diagnostic control, but it supervises answer form rather than counterfactual sufficiency. It tests whether longer visible answer states alone can reproduce the erasure-dependence signature. In contrast, CERL optimizes whether \(A\) remains reliable after \(H\) is erased; \(A\) may be short or long, and form is not the objective.

3 Method↩︎

3.1 Problem definition: post-answer hidden-thought erasure↩︎

Given a problem \(x\), the model may generate hidden thought \(H\) to help produce answer \(A\). Before the next hidden-thought segment begins, the previous \(H\) is erased, and only the committed \(x+A\) remains for downstream context. The central question is whether the model can learn, during training, to make \(A\) correct, sufficient, and future-reliable after \(H\) is erased; equivalently, whether it can explicitly distinguish the boundary between computation and state.

3.1.0.1 Formal definition of progressive erasure.

For t = 1..n:
    H_t = Think(x, A_{1:t-1})
    A_t = Output_State(x, A_{1:t-1}, H_t)
    Before H_{t+1} starts generating: Erase(H_t)

The final context is \(x + A_1 + \cdots + A_n = x + A_{1:n}\), and all think segments have disappeared.

\(A_1,\ldots,A_n\) are all permanently retained answer tokens, and each \(A_i\) is produced with the help of its corresponding hidden thought \(H_i\). We do not model whether to enter thought as a separate learning target: if an intermediate step requires no hidden computation, it does not form a separate erasure segment and is treated as part of a neighboring answer state. Form-level differences are not prescribed; they are determined by the objective.

3.1.0.2 Sufficiency target.

We instantiate persistent-state sufficiency with two requirements:

• In single-turn settings, \(x + A_{1:n}\) should be sufficient to infer the final correct answer without \(H\).

• In multi-turn settings, the historical visible answers \(A_{1:n}\) should support follow-up queries after all \(H\) has been erased.

3.2 Tag semantics↩︎

Visible text is the default output channel. Only <think>...</think> is special and marks temporary hidden-thought spans. The generation alternates between hidden-thought spans and answer states; a full example appears in Appendix 11.

3.3 Training objective: persistent-state sufficiency↩︎

We directly define a future-facing objective in task-projected terms. Given the same generated tuple \((x,H,A)\), if \(A\) has already served as the persistent-state interface, then erasing \(H\) should not reduce downstream correctness: \[\Pr_\pi[C_{\mathrm{pe}}=1 \mid x,H,A] \;\ge\; \Pr_\pi[C_{\mathrm{full}}=1 \mid x,H,A].\] Here \(C_{\mathrm{full}}\) and \(C_{\mathrm{pe}}\) denote downstream correctness events under the full-thought and erasure paths, respectively. The meaning is that \(A\), as committed persistent state, should serve as the interface supporting downstream reasoning after hidden thought \(H\) is erased. In this sense, \(A\) crosses the computation–state boundary and becomes an object that can be relied on in the future. CERL’s binary correctness reward is a finite-sample surrogate for this objective.

We make no content-level assumption about \(A\) and \(H\): \(A\) may inherit, rewrite, extend, or be independent of \(H\). Its final form emerges from the training objective rather than from a form constraint. The commitment is an objective-level property, persistent-state sufficiency, not a form-level constraint.

3.4 Theoretical framework↩︎

This section defines the estimable objects used for training and evaluation. We do not define PSS as equality of full output distributions; instead, we define its task-projected version over evaluator-induced binary correctness events. To avoid notation overload, \(P_\pi(y\mid\cdot)\) denotes only the output distribution of policy \(\pi\); \(\Pr_\pi[\cdot]\) denotes event probabilities induced jointly by the policy, data distribution, decoder randomness, and evaluator; and \(\mathrm{Acc}\) abbreviates the probability of a correctness event. The Acc values in experimental tables are finite-sample estimates. Specifically, \[\mathrm{Acc}_{\text{full}}(\pi) = \Pr_\pi[C_{\text{full}}=1], \qquad \mathrm{Acc}_{\text{pe}}(\pi) = \Pr_\pi[C_{\text{pe}}=1],\] where \(C_{\text{full}}\) and \(C_{\text{pe}}\) denote final-answer correctness under the full-thought and progressive-erasure paths. The full / erasure comparison is paired counterfactual evaluation: we first fix the same generated tuple \((x,H,A)\) and then construct two evaluation contexts \[g_{\mathrm{full}}(x,H,A)=(x,H,A),\qquad g_{\mathrm{pe}}(x,H,A)=(x,A).\] Thus \(A\) is identical in the two paths, and the only intervention is whether \(H\) remains in the future context. We compare the effect of this intervention on correctness events; we do not require the two paths to generate identical downstream text or induce identical output distributions.

The proof is in Appendix 9. This decomposition shows that ASG is not a standard accuracy metric; it is the difference between hidden-thought dependence \(D_H\) and erasure purification \(D_E\). Reporting full-path accuracy alone cannot determine whether the answer state is sufficient after erasure. We therefore explicitly include \(\mathrm{Acc}_{\text{pe}}\), HTDR, ESR, and MSG in training and evaluation. In implementation, \(c_{\text{full}}\) and \(c_{\text{pe}}\) are Monte Carlo estimates from repeated rollouts under the two paired contexts above.

3.5 Training signal: CERL and HSCO↩︎

We implement erasure-dependence training at two complementary levels. CERL optimizes a finite-sample surrogate of persistent-state sufficiency through paired full-thought / erasure dual-context rollouts. HSCO uses two GRPO layers to update hidden thought \(H_1\) and answer state \(A_1\). Concretely, \(c_{\text{pe}}\) is a Monte Carlo estimate of \(\Pr_\pi[C_{\mathrm{pe}}=1\mid x,H,A]\), so the binary correctness reward directly corresponds to the task-projected PSS objective above, rather than to full output-distribution matching. For each \(x\), we sample 4 candidates for \(H_1\); for each \(H_1\), we sample 8 candidates for \(A_1\). Each \(A_1\) is evaluated under both a full-thought path that retains \(H_1\) and an erasure path that deletes \(H_1\), plus an additional skip baseline. Full sampling pseudocode and statistics are in Appendix 10.

For each \(A_1\), let \(c_{\text{full}}\) and \(c_{\text{pe}}\) denote correctness rates under the full-thought and erasure paths, and let \(L_E\) be the average downstream length after \(A_1\) under the erasure path. Let \(L_{F,\text{correct}}\), \(L_{E,\text{correct}}\), and \(L_{S,\text{correct}}\) denote the downstream lengths of correct trajectories under the full-thought, erasure, and skip paths, respectively; if a path has no correct trajectory, its length is set to \(+\infty\). Let \(\overline{L}_{E,\text{correct}}^{(k)}\) denote the mean \(L_{E,\text{correct}}\) over multiple \(A_1^{(k,i)}\) candidates under the same \(H_1^{(k)}\). We then define the downstream length baseline: \[\label{eq:lmin} L_{\min}^{(k,i)} = \min\bigl( L_{F,\text{correct}},\; \overline{L}_{E,\text{correct}}^{(k)},\; L_{S,\text{correct}} \bigr) .\tag{1}\] \(L_{\min}\) is the successful downstream-length baseline for the current sample and is used to constrain \(L_{E,\text{correct}}\); paths with no correct trajectory take length \(+\infty\).

3.5.1 \(A_1\)-level signal↩︎

The \(A_1\)-level signal uses erasure-path correctness \(c_{\text{pe}}\) as the main reward and adds two form-control penalties. For each \(A_1^{(k,i)}\) (superscripts omitted below), let \(m=\mathrm{mean}_i\,\mathrm{len}(A_1^{(k,i)})\) be the average length of the 8 \(A_1\) candidates under the same \(H_1\). The reward is \[\label{eq:a1-reward} R \;=\; \begin{cases} c_{\text{pe}} \;-\; \alpha \cdot \dfrac{\mathrm{len}(A_1)}{m} \;-\; \beta \cdot \max\!\Bigl(0,\, \dfrac{L_{E,\text{correct}}}{L_{\min}} - 1\Bigr), & \text{if } c_{\text{pe}} > 0,\\[2mm] -\psi, & \text{otherwise}. \end{cases}\tag{2}\] This reward learns the state-commitment boundary through erasure-after-correctness gating, rather than through a simple linear tradeoff between correctness and length. First, the main reward comes from erasure-path correctness: after \(H\) is deleted, downstream reasoning can rely only on \(x+A\), so if \(A\) has not committed information needed later, \(c_{\text{pe}}\) drops. This signal pushes future-relevant information into the visible answer state. Second, the reward is gated: when the erasure path fails, the sample receives \(-\psi\), so a short but insufficient \(A_1\) cannot benefit. Only when \(c_{\text{pe}}>0\) do the \(A_1\) length pressure and downstream anti-postpone pressure act among candidates that are already correct after erasure. The \(\alpha\) term provides peer-relative length pressure within the same \(H_1\), while the \(\beta\) term discourages postponing critical computation until after erasure or making \(A_1\) worse than the skip baseline. In other words, CERL first requires \(A_1\) to be sufficient for correct post-erasure reasoning, and only then optimizes the economy of the persistent state: information that remains needed downstream is encouraged to be committed to \(A_1\), while trial-and-error, exploration, and redundant computation are encouraged to remain in temporary <think> spans that can be erased. The hard penalty \(-\psi\) also closes a reward-hacking channel in which the model could make the full-thought path fail as well to avoid penalty. Design motivation, default hyperparameters, and GRPO implementation details are in Appendix 12.

3.5.2 \(H_1\)-level signal↩︎

The 4 sampled \(H_1\) candidates under the same \(x\) form the \(H_1\)-level GRPO group, with reward \[R_{H_1}^{(k)} \;=\; \mathrm{avg}_{i=1..8}\, c_{\text{pe}}^{(k,i)} .\] Thus, under the same problem, the preferred \(H_1\) is the one whose 8 \(A_1\) descendants are more often correct after erasure. This signal updates hidden-thought policies that better support commit-ready answer states; gradients are applied only to \(H_1\) tokens.

3.5.3 Coordination and curriculum activation↩︎

The two rewards backpropagate through non-overlapping token ranges to avoid interference. Training first activates only the \(A_1\)-level signal to stabilize the answer-state interface. Later end-to-end curriculum stages activate the \(H_1\)-level signal so that hidden-thought policy and answer state are optimized jointly. Layer structure and curriculum details are in Appendix 12.

3.6 Training pipeline↩︎

3.6.0.1 Stage 1: Skill data production.

We first construct successful trajectories with a progressive-erasure timeline: each substep generates a temporary <think>\(H_i\)</think> and then commits a visible state \(A_i\); before the next think segment begins, the previous \(H_i\) is deleted. Each example stores both the full version and the erased version.Further details are in Appendix 13.

3.6.0.2 Stage 2: Answer-state commitment SFT.

Each trajectory is split into multiple training examples according to the erasure timeline, so that each training context contains at most the current think segment and all previous think segments have already been erased. SFT therefore teaches the alternating format, answer-state commitment under hidden-thought assistance, and continued reasoning when earlier thoughts are not visible.

3.6.0.3 Stage 3: CERL.

Starting from the SFT model, we sample full-thought and progressive-erasure paths in parallel and optimize erasure-path correctness with Eq. 2 . The curriculum gradually expands the erasure range and gradient range: it first stabilizes local \(A_1\)-level PSS and then enables full-chain erasure with both \(A_1\) and \(H_1\) signals.

4 Experiments↩︎

4.1 Erasure dependence protocol↩︎

The Erasure Dependence Protocol reports both task accuracy and post-erasure sufficiency. The basic accuracy metric is Acc, reported in the tables as Avg@8. The core metrics are \[\mathrm{ASG}=\mathrm{Acc}_{\text{full}}-\mathrm{Acc}_{\text{pe}},\quad \mathrm{HTDR}=\Pr_\pi[C_{\text{full}}=1 \wedge C_{\text{pe}}=0],\] \[\mathrm{ESR}=\Pr_\pi[C_{\text{pe}}=1 \mid C_{\text{full}}=1],\quad \mathrm{AIS}=\mathrm{Acc}_{\text{pe}}/\mathrm{Acc}_{\text{full}} .\] Lower ASG is better, higher ESR/AIS is better, and HTDR is the empirical estimate of Definition 2. To rule out the alternative explanation of recomputation after erasure, we also report marginal state gain: \[\boldsymbol{MSG} \text{ (Marginal State Gain)} \;=\; \mathrm{Acc}(x + A_1) \,-\, \mathrm{Acc}(x + \texttt{empty}) .\] \(\mathrm{MSG}>0\) indicates that \(A_1\) contributes nontrivial marginal state information beyond the skip-context recomputation baseline. Algebraic relations among metrics and form reports (HTT/RHTT, \(\overline{\mathrm{len}}(A)\), TTF) are in Appendix 14.

4.2 Experimental setup↩︎

The goal is not to compare who compresses reasoning traces more aggressively, but to test the core claim: hidden thought can serve as temporary computation, and once its conclusion is committed to visible answer state, the original thought trace can be erased without harming downstream solving. Experiments therefore ask whether (i) the progressive-erasure path maintains high correctness without retaining hidden thought in the final context, and (ii) the gain comes from answer-state commitment rather than ordinary trace compression, long-answer supervision, test-time refresh, or correctness-only RL. Beyond mathematical, logical, and scientific QA tasks, we include BFCL-v3 to test whether the state interface is effective in multi-turn tool use.

4.2.0.1 Base model and training data.

The main table uses Qwen3-8B [5] as the base model; Appendix 18 reports same-protocol Qwen3-14B/32B results. Training data uses strictly decontaminated DeepMath-103K [6], converted in Stage 1 into answer-state commitment SFT data with a progressive-erasure timeline, followed by CERL training using the objective in §3.5. All experiments use a \(C=32\)k context window.

4.2.0.2 Evaluation sets.

We evaluate on AIME 2024, AIME 2025, ZebraLogic [7], AutoLogi [8], BFCL-v3 [9], and GPQA-Diamond [10]. AIME and GPQA-Diamond test difficult mathematical and scientific reasoning, ZebraLogic and AutoLogi test long-chain logic, constraint preservation, and multi-stage state updates, and BFCL-v3 (Berkeley Function-Calling Leaderboard v3) tests tool selection, parameter maintenance, and cross-turn state management in multi-turn function calling.

4.2.0.3 Decoding and statistics.

All methods use Temperature=0.6, TopP=0.95, TopK=20, and MinP=0. Each problem is sampled independently 8 times and reported as Avg@8. Task accuracies are reported over 5 random seeds as mean \(\pm\) 95% bootstrap CI, while erasure-dependence metrics are aggregate point estimates over the same evaluation set. Differences from baselines use paired bootstrap tests. The main tables report task correctness and core erasure-dependence metrics; HTDR, AIS, retained hidden-thought tokens, \(\overline{\mathrm{len}}(A)\), and TTF are protocol extensions defined in Appendix 14.

4.2.0.4 Baselines.

We group comparisons into four categories. The first contains base paths without state commitment: Base Qwen3 and Fixed-ratio erasure + free answer, which share the erasure mechanism but have no CERL training. The second contains alternative training objectives: Long-answer SFT, Correctness-only RL, and Length-penalty RL. Long-answer SFT supervises the erased version of the same successful trajectories, i.e., input \(x\) and target \(A_{1:n}\) only, with no \(H\); it is a diagnostic control for whether supervising long visible answer states naturally yields similar behavior, not a single-variable causal comparison. The third contains trajectory-retention and test-time management baselines, including TokenSkip [11], trajectory compression baselines, and Halo [12]. The fourth is CERL full. ASG/ESR/MSG are defined only for methods with a homologous multi-segment \(H_i/A_i\) interface that can run both full-thought and progressive-erasure paths; ordinary baselines without this interface use “-” in these columns.

4.3 Experiment 1: main results↩︎

The main experiment runs, for each problem, both the full-thought path and the progressive-erasure path, compares final-answer correctness, and reports ASG, ESR, and MSG. The protocol requires a locatable multi-segment \(H_i/A_i\) structure. Methods that produce ordinary answers, single-segment CoT, compressed traces, or external state management have no erasure points or \(A_1\) skip control aligned with CERL, so these metrics are not computed. \(\mathrm{Acc}_{\text{skip}}\) and MSG are jointly reported to rule out a narrow recomputation explanation: the erasure path does not succeed by completely ignoring \(A_1\) and recomputing from \(x\). We define \(\mathrm{MSG}=\mathrm{Acc}_{\text{pe}}-\mathrm{Acc}_{\text{skip}}\); positive MSG indicates that \(A_1\) has nontrivial marginal contribution to downstream reasoning, but does not by itself prove that \(A_1\) is an abstract state. We therefore interpret it together with visible-CoT leakage audits, downstream length control, and form statistics.

The table addresses four alternative explanations. If CERL were merely ordinary CoT compression or test-time trace management, TokenSkip / compression baselines and Halo should obtain similar gains. If correctness optimization were sufficient, Correctness-only RL should approach CERL full. Long-answer SFT improves some task accuracies, but it mainly tests the weaker alternative explanation of long visible-answer supervision; because it lacks a homologous erasure interface, it does not support ASG/MSG causal attribution. CERL’s unique signature is zero retained hidden-thought tokens at final evaluation, low ASG, high ESR, and high MSG, while \(\overline{\mathrm{len}}(A)\) and TTF are only form reports.

4.4 Experiment 2: training-objective ablation↩︎

This experiment provides a protocol-compatible comparison of erasure dependence: all variants that report ASG/MSG share the homologous multi-segment \(H_i/A_i\) interface, so differences can be more directly attributed to the training objective and curriculum. We remove RL, the progressive-erasure curriculum, and the anti-postpone penalty; these component removals retain the homologous multi-segment erasure interface and therefore report ASG/MSG.

Ablations show that removing CERL RL within the same training family substantially degrades ASG and MSG, indicating that SFT initialization alone does not learn answer-state commitment under erasure. Removing the progressive-erasure curriculum preserves an erasure objective but makes credit assignment less stable. Removing the anti-postpone penalty increases ASG, suggesting that the model more easily postpones critical computation until after erasure.

4.5 Experiment 3: failure-mode audit↩︎

We audit the main failure modes with two controls; the full rubric and length-control details are in Appendix 15. The skip audit compares \(\mathrm{Acc}(x+A_1)\) with \(\mathrm{Acc}(x+\texttt{empty})\); \(\mathrm{MSG}>0\) indicates that the erasure path does not succeed by completely ignoring \(A_1\) and recomputing from \(x\). The visible-CoT leakage audit checks whether \(A\) contains step-by-step trial and error, self-correction, or scratch-like derivations. Downstream length control in the CERL reward further discourages compensating for uncommitted state by extending downstream reasoning after erasure.

5 Empirical findings↩︎

The experiments support three conclusions. First, hidden thought can serve as temporary computation: CERL full maintains high Acc while final retained hidden-thought tokens are zero and ASG is low, showing that once conclusions are committed, the original thought trace can be erased. Second, correctness training is insufficient for this property: within the same CERL family, removing RL or counterfactual erasure components increases ASG and decreases MSG. Third, this ability appears not only in mathematical and long-chain logical tasks, but also in multi-turn tool use; this suggests that state commitment learning is learning a more reliable downstream state interface, not merely a longer answer form. Long-answer SFT only tests the weaker explanation of “longer visible answers,” whereas CERL’s signature is simultaneously high Acc, low ASG, and high MSG.

6 Discussion↩︎

6.0.0.1 Must long chains of thought be retained?

The current reasoning paradigm assumes that thought traces must remain permanently in context, which conflates computation with memory. We do not question whether a model should think deeply; we question whether thought conclusions must be explicitly committed to persistent state and whether the original trace must remain available. The model can still think extensively, but thought is temporary computation. Once its conclusion is committed into the answer state, the temporary thought is no longer the object that downstream reasoning must retain.

6.0.0.2 Connection to agent settings.

Long-horizon agents already require some form of summarization because context windows cannot hold full trajectories. Our method internalizes this capability into the model: rather than relying on an external summarizer, the model performs the loop “computation \(\to\) state commitment \(\to\) erasure” during generation. BFCL-v3 further indicates that this mechanism can transfer to state maintenance in multi-turn tool use, rather than being only a formatting trick for single-turn reasoning problems. This provides a training-time approach to context management for agents, rather than an inference-time post-processing step.

7 Broader impact↩︎

If state commitment learning scales, it may shift large-model reasoning from retaining full chains of thought to using temporary computation plus committed state. This could reduce long-context cost, improve agent robustness, reduce unnecessary retention of private intermediate reasoning, and make committed state easier to audit, offering substantial economic value in the current AI coding boom.

8 Limitations↩︎

CERL learns the allocation boundary between temporary computation and visible state under an explicit <think> / answer interface; it is a restricted instance of state-management training, not a solution to general memory writing or arbitrary state management.

9 Supplementary theory proof↩︎

10 CERL sampling and statistics↩︎

For curriculum step one, where only \(H_1\) is erased and tokens after \(H_2\) are neither erased nor updated, sampling proceeds as follows:

Input: x

Sample 4 H_1:
    H_1^{(1)}, H_1^{(2)}, H_1^{(3)}, H_1^{(4)}

For each H_1^{(k)}, sample 8 A_1:
    A_1^{(k,1)}, ..., A_1^{(k,8)}

For each A_1^{(k,i)}, under two conditions:
    Full-thought (keep H_1^{(k)}):
        ctx = x + H_1^{(k)} + A_1^{(k,i)}
        Generate 4 trajectories: F^{(k,i,1..4)}
    Erasure (delete H_1^{(k)}):
        ctx = x + A_1^{(k,i)}
        Generate 4 trajectories: E^{(k,i,1..4)}

Additionally sample Skip (per x):
    ctx = x + <think></think>
    Generate 4 trajectories: S^{(1..4)}

Total: 4 * 8 * (4+4) + 4 = 260 trajectories.

For each \(A_1^{(k,i)}\), we record \(c_{\text{full}}\), \(c_{\text{pe}}\), \(L_E\), \(L_{F,\text{correct}}\), and \(L_{E,\text{correct}}\). For each \(x\), we record shared skip statistics \(c_{\text{skip}}\) and \(L_{S,\text{correct}}\). A path with no correct trajectory takes length \(+\infty\) and automatically drops out of the \(\min\) in Eq. 1 .

11 Progressive erasure example↩︎

Visible text is the default output channel. Only <think>...</think> marks hidden-thought spans. The model alternates between hidden-thought spans and answer states \(A_i\); each \(H\) is erased before the next \(H\) begins, so the context contains at most one think span at any time.

11.0.0.1 Example.

<think>H_1: Analyze the variable relation...</think>   # temporary span
A_1: Let the ball price be x
<think>H_2: Set up the equation...</think>             # H_1 is erased
A_2: x+(x+1)=1.10, so x=0.05
A_3 (final): The ball costs 5 cents.

The process has three key properties: (i) the model alternates between temporary hidden thought and visible answer state; (ii) the context retains only the current think span, and previous thoughts are erased when a new span starts; and (iii) if a local step needs no hidden computation, it does not form a separate erasure segment and is merged into a neighboring answer state.

12 CERL reward-design details↩︎

12.0.0.1 Why CERL does not directly optimize the signed PSS gap.

Definition 1 gives a counterfactual sufficiency criterion, not a scalar reward that should be optimized literally. Directly optimizing the signed gap \(c_{\text{pe}}-c_{\text{full}}\) would create an unsafe incentive: the policy could increase this objective either by improving post-erasure correctness, which is desired, or by reducing full-thought correctness, which is a degenerate solution. CERL therefore uses a guarded one-sided surrogate. The main positive signal is \(c_{\text{pe}}\), because correctness after erasure is the intended direction of improvement; the full-thought path is retained for paired counterfactual evaluation, online monitoring, and successful-length baselines. At the same time, the hard failure penalty, skip baseline, and downstream-length penalty jointly discourage three reward-hacking modes: degrading the full-thought path, ignoring \(A_1\) and recomputing from \(x\), and postponing critical computation until after erasure. Thus the gap between Definition 1 and Eq. 2 is intentional: PSS is the population-level criterion, while CERL is a practical finite-sample surrogate designed to encourage PSS without rewarding reductions in full-thought competence.

12.0.0.2 Why \(c_{\text{pe}}=0\) always receives \(-\psi\).

A natural alternative is to skip an example when \(c_{\text{full}}=0\), setting \(R=0\), to avoid penalizing \(A_1\) on hard tasks. But this introduces a reward-hacking channel: when \(c_{\text{pe}}=0\), the model can make \(A_1\) mislead downstream reasoning so that \(c_{\text{full}}\) also fails, raising reward from \(-\psi\) to \(0\). Eq. 2 closes this channel: \(c_{\text{pe}}=0\) always receives \(-\psi\), independent of \(c_{\text{full}}\). When an entire hard group receives the same \(-\psi\), group variance is zero and GRPO advantage is zero, functionally equivalent to skipping but without the marginal benefit of hacking.

12.0.0.3 Motivation for the three penalties.

The \(\alpha\) term uses the mean length of the 8 \(A_1\) candidates under the same \((x,H_1)\) as a peer-relative efficiency baseline. The \(\beta\) term constrains two failure modes: (i) \(A_1\) postpones state that should have been committed until downstream post-erasure thought, causing \(L_{E,\text{correct}}>L_{\min}\); and (ii) \(A_1\) misleads downstream reasoning and becomes worse than the no-\(A_1\) skip baseline. The asymmetric \(\max(\cdot,0)\) imposes no penalty when \(L_{E,\text{correct}}<L_{\min}\), which may indicate noise purification or improved efficiency. The \(\psi\) term treats complete erasure-path failure as PSS failure.

12.0.0.4 Why \(L_{\min}\) comes from three paths.

\(L_{\min}\) is a compact known-success length baseline: \(L_{F,\text{correct}}\) gives the natural length when \(H\) is present; \(\overline{L}_{E,\text{correct}}^{(k)}\) gives the within-group typical length when the answer interface succeeds under the same \(H_1^{(k)}\); and \(L_{S,\text{correct}}\) gives the lower bound without \(A_1\). Taking the minimum imposes the strictest pressure. Paths with no correct trajectory take \(+\infty\) and need no extra indicator. The full-thought path cannot be replaced by within-group statistics, because the natural length when \(H\) is present is not recoverable from the 8 \(A_1\) candidates alone. The skip path is independent of \(H_1\) and \(A_1\), can be shared per \(x\), and costs only 4 additional trajectories.

12.0.0.5 Why the denominator uses \(\mathrm{mean}(A_1)\) rather than \(\mathrm{len}(H_1)\).

There are three reasons to choose the mean length of \(A_1\) under the same \(H_1\) rather than \(\mathrm{len}(H_1)\):

1. No compression assumption. The ratio \(\mathrm{len}(A_1)/\mathrm{len}(H_1)<1\) implies that \(A_1\) should be a compressed version of \(H_1\), i.e., \(A\approx\mathrm{Compress}(H)\). §3.3 explicitly rejects this view: the objective is PSS, not reconstructing \(H\). Using \(\mathrm{mean}(A_1)\) removes \(H_1\) from the denominator and keeps the reward aligned with the objective.

2. Task difficulty adapts without routing through \(H_1\). Easy tasks naturally produce shorter \(A_1\) and a smaller \(m\), creating stronger pressure for short states; hard tasks produce larger \(m\) and weaker length pressure. The policy’s own answer-length distribution is a direct implicit baseline for task difficulty.

3. Form emerges from the objective. Under the \(\mathrm{mean}(A_1)\) baseline, \(A_1\) can be shorter or longer than \(H_1\), depending on the task and answer strategy. \(A_1<H_1\), \(A_1\approx H_1\), and \(A_1>H_1\) are all valid if PSS holds.

12.0.0.6 Two roles of the full-thought path.

The full-thought path provides \(L_{F,\text{correct}}\) and also monitors training dynamics online.

Role 1: vertical scale for reward. Different problems have different natural downstream lengths; penalizing absolute \(L_{E,\text{correct}}\) would over-penalize hard problems and under-penalize easy ones. \(L_{F,\text{correct}}\) provides a per-instance scale for how long downstream solving naturally is when \(H\) is available. The \(\alpha\) term gives a horizontal scale across the 8 \(A_1\) candidates under the same \(H_1\), while the \(\beta\) term compares the current erasure-success length \(L_{E,\text{correct}}\) against the achievable baseline \(L_{\min}\); these two scales are orthogonal and not interchangeable. \(L_{\min}\) further tightens the baseline to a compact known-success length and provides stricter training pressure.

Role 2: online monitoring. The full-thought path produces \(c_{\text{full}}\), enabling core metrics to be computed within each training batch: \[\begin{align} \mathrm{ASG} &= c_{\text{full}} - c_{\text{pe}}, & \text{(sufficiency gap; lower is better)}\\ \mathrm{AIS} &= c_{\text{pe}} / c_{\text{full}}, & \text{(interface-sufficiency ratio)}\\ \mathrm{ESR} &= \Pr_\pi\bigl[C_{\text{pe}}=1 \mid C_{\text{full}}=1\bigr]. & \text{(erasure success rate)} \end{align}\] Without full-thought paths, these quantities would require a separate offline evaluation pass and would lose fine-grained training dynamics.

12.0.0.7 No length penalty on \(H_1\).

\(H_1\) is transient: it is erased before the next thought segment and does not enter permanent context. Its length only incurs one-time generation cost and is not part of the paper’s efficiency claim. If a longer \(H_1\) does not improve \(c_{\text{pe}}\), policy gradients will stop extending it; an explicit length penalty could harm thought quality.

12.0.0.8 Curriculum activation details.

• Early curriculum: activate only the \(A_1\)-level signal; gradients on \(H_1\) and tokens from \(H_2\) onward are masked. Because \(A_1\) directly corresponds to the PSS signal, it should stabilize before upstream \(H_1\) changes.

• End-to-end curriculum: activate both \(H_1\)-level and \(A_1\)-level signals; both \(H_1\) and \(A_1\) tokens receive gradients, jointly optimizing hidden-computation quality and state-commitment quality.

12.0.0.9 Default hyperparameters.

We use \(\alpha\approx0.3\) for peer-relative \(A_1\) length pressure, \(\beta\approx0.5\) for the \(L_{E,\text{correct}}/L_{\min}\) penalty, and \(\psi\approx0.5\) for hard answer-interface failure. In practice, we first tune \(\alpha\) to place the \(A_1\) length distribution in a reasonable range, then tune \(\beta\) to prevent postponement, and finally tune \(\psi\) to balance exploration and stability.

13 Training data and curriculum details↩︎

13.0.0.1 Skill construction and data production.

We first train a skill that answers problems and keep trajectories whose final answers are correct. The skill plans the problem into multiple steps. For each step, a subagent thinks and then answers; its thought and answer are concatenated back into the main-agent prefix, and the previous thought is deleted before dispatching the next subagent to think. This ensures that data generation itself performs progressive erasure, follows the format <think>\(H_1\)</think> \(\to A_1 \to\) erase \(H_1\) \(\to\) <think>\(H_2\)</think> \(\to A_2 \to \cdots\), and stores both full and erased versions.

13.0.0.2 SFT sample splitting.

For a trajectory with three think segments:

Each training example contains at most the current think segment in context, with all earlier thoughts erased. This teaches the alternating format, answer-state commitment under hidden-thought assistance, and continued reasoning when previous thoughts are invisible.

13.0.0.3 Curriculum learning.

We progressively expand the erasure and gradient ranges. Step one erases only \(H_1\) and masks tokens from \(H_2\) onward. Step two erases \(H_1,H_2\) and masks tokens from \(H_3\) onward. Later steps expand erasure points and gradient ranges. The end-to-end step enables full-chain erasure and both \(A_1,H_1\) signals. This makes early credit assignment controlled and later extends training to the full policy.

14 Evaluation metric details↩︎

The four erasure metrics are not fully independent. AIS and ASG are algebraic transformations of the two-dimensional state \((c_{\text{full}},c_{\text{pe}})\), with \(\mathrm{ASG}=c_{\text{full}}(1-\mathrm{AIS})\). HTDR and ESR imply one another through \(\mathrm{HTDR}=c_{\text{full}}(1-\mathrm{ESR})\). The two independent axes are ASG (net effect, including negative values) and ESR (conditional retention). We still report all four to support comparison under different narratives.

Form reports include \(\overline{\mathrm{len}}(A)\) and TTF (Transient Token Fraction), defined as \(\mathrm{HTT}/(\mathrm{HTT}+\overline{\mathrm{len}}(A))\). We avoid subtraction forms such as \((H-A)/H\) because they imply that \(A\) is a compression of \(H\), contrary to the training objective. \(\overline{\mathrm{len}}(A)\) and TTF are reported only as emergent properties, not as training signals or win conditions.

15 Failure-mode audit details↩︎

15.0.0.1 Skip audit.

The skip audit targets one narrow recomputation explanation: whether the erasure path completely ignores \(A_1\) and recomputes from \(x\). We compare \(\mathrm{Acc}(x+A_1)\) with \(\mathrm{Acc}(x+\texttt{empty})\) under the same downstream decoding budget and report \(\mathrm{MSG}=\mathrm{Acc}_{\text{pe}}-\mathrm{Acc}_{\text{skip}}\). Positive MSG indicates that \(A_1\) has marginal contribution to downstream reasoning, but does not by itself prove that \(A_1\) is an abstract state.

15.0.0.2 Visible-CoT leakage audit.

We inspect sampled answer states \(A\) using a fixed rubric: if \(A\) contains step-by-step trial and error, self-correction, explicit search branches, scratch-like derivations, or other process text resembling CoT, it is flagged as visible-CoT leakage. This audit is not a training objective or main-table win condition; it checks whether long-answer supervision degenerates into copying hidden thought into visible state.

15.0.0.3 Downstream length control.

The length term in the CERL reward constrains downstream length on the erasure path, discouraging the model from compensating for uncommitted state by extending downstream reasoning after erasure. Length control alone cannot rule out visible-CoT leakage; we therefore interpret MSG, visible-CoT leakage audits, downstream length control, and form statistics jointly.

16 Experimental scope and extrapolation boundary↩︎

Current experiments cover single-turn benchmarks, offline counterfactual erasure audits, and multi-turn tool use; AI coding tasks, open-ended tool environments, and multi-document reasoning require further validation.

17 BFCL-v3 public reference baselines↩︎

BFCL-v3 (Berkeley Function-Calling Leaderboard v3) evaluates function calling and agent-style tool use; its official leaderboard and code/data page is https://gorilla.cs.berkeley.edu/leaderboard. The Qwen3 technical report evaluates BFCL-v3 in FC format and uses a 64k context length for multi-turn evaluation [5]. Table 4 reports the public Qwen3 reference scores most relevant to our model scales; these are copied from the Qwen3 report and are not CERL erasure-dependence measurements.

18 Larger-model supplementary results↩︎

Tables 5 and 6 report supplementary results for Qwen3-14B and Qwen3-32B under the same evaluation protocol. Acc is Avg@8; task-accuracy columns report the mean and 95% bootstrap confidence interval over 5 random seeds. ASG, ESR, and MSG are aggregated under the Erasure Dependence Protocol to measure post-erasure state sufficiency and marginal state gain.

18.0.0.1 Interpreting negative ASG.

By Lemma 1, \(\mathrm{ASG}<0\) means that the erasure-purification term \(D_E\) exceeds the hidden-thought dependence term \(D_H\). We do not interpret negative ASG as evidence that erasure intrinsically improves reasoning. Instead, we interpret it as indicating that in some examples, retained hidden thoughts may introduce noise, erroneous branches, or redundant context, so the model can reason more reliably from the committed answer state after erasure. This is consistent with recent findings on long-CoT degradation and overthinking: longer or retained chains of thought do not always yield positive gains and may become a source of interference [1]–[4]. We therefore interpret low ASG as evidence for downstream-reliable answer state only when it co-occurs with high ESR and positive MSG.

19 Supplementary positioning of related directions↩︎

Long-CoT degradation provides the empirical motivation for state commitment learning. Don’t Overthink It observes that short chains can outperform long chains by up to 34.5% across reasoning tasks [1]. When More is Less characterizes a non-monotonic relationship between accuracy and chain length and estimates scaling laws for an optimal length [2]. Through the Valley names this phenomenon Long CoT Degradation [3]. The Curse of CoT reports settings where adding chains of thought harms performance [4]. Together, these results suggest that some reasoning tokens can become distractors rather than useful persistent state.

Inference-time methods mainly manage traces after they have already been generated. KV-cache eviction methods such as H2O, SnapKV, and LazyEviction remove tokens from attention caches [13]–[15]; token-pruning methods such as Think Clearly delete tokens by importance [16]; thought-aware compression methods such as ThinKV, KaVa, and LightThinker compress thought representations [17]–[19]; short-chain selection chooses shorter solutions from multiple samples [1]; gist tokens [20] and memory-augmented architectures such as NTM and DNC store or read information in more compact forms [21], [22]. Process-level supervision, step verification, and trace editing/intervention also study the controllability of intermediate reasoning. The distinction is that prior work mainly controls intermediate reasoning traces, whereas this paper trains whether the visible answer state \(A\) can serve as the downstream reasoning interface after hidden thought \(H\) is erased.

A superficially similar alternative is Long-answer SFT: supervise only the erased visible answer sequence \(A_{1:n}\), without hidden thoughts \(H\) and without full / erasure / skip counterfactual comparisons. This baseline is a diagnostic control rather than a strict causal ablation, because it changes both the supervision target and the optimization procedure. Its role is to test a weaker alternative explanation: whether supervising longer visible answer states alone naturally yields the same erasure-dependence signature. The key distinction is supervising answer form vs. training counterfactual sufficiency, or what is written now vs. what can still be relied on later.

4pt

References↩︎