1 Introduction↩︎

Reinforcement learning with verifiable rewards has emerged as an effective post-training paradigm for large language models (LLMs) [1], [2]. However, practical systems often require continual post-training across domains and skills [3], [4], raising a significant challenge: models must acquire new capabilities without forgetting prior ones. Existing work has suggested an optimistic picture for continual post-training: on-policy reinforcement fine-tuning (RFT) forgets less than supervised fine-tuning (SFT) [5]–[8], and this benefit has often been attributed to the use of on-policy data [6], [7], [9].

Recently, on-policy distillation has emerged as an attractive alternative to sparse reward-based reinforcement learning. By converting model-generated rollouts into dense token-level supervision, these methods promise better credit assignment, higher sample efficiency, and reduced dependence on external teachers or explicit reward models [10]–[12]. A particularly appealing variant is on-policy self-distillation, where the model itself acts as a teacher by conditioning on demonstrations, feedback, solutions, or other auxiliary context, and the resulting behavior is distilled back into the same model. For example, self-distillation fine-tuning (SDFT) uses a demonstration-conditioned model as its own teacher and demonstrates that such on-policy distillation can acquire new skills while preserving prior capabilities [13]. Self-distillation policy optimization (SDPO) similarly uses a more-informed, feedback-conditioned teacher to provide dense token-level supervision to the student policy [14]. Together, these works suggest that on-policy self-distillation may be a practical recipe for continual learning, combining the on-policy nature of RFT that mitigates forgetting with the efficiency of dense supervised updates for learning new knowledge.

However, this view conflates two distinct ingredients: where the data come from and which objective is used to update the model. On-policy sampling determines the histories on which the model is trained, but the training objective determines which continuation distribution is reinforced at each history. This distinction is especially important for dense self-distillation. When every token becomes a training target, teacher drift, noisy rationales, formatting conventions, and domain-specific artifacts can be reinforced as repeatedly as useful behavior [15], [16]. Thus, on-policy self-distillation may avoid the off-policy mismatch of SFT, but it does not automatically inherit the conservative update bias of sequence-level on-policy RFT.

This paper studies whether on-policy self-distillation enables reliable continual post-training. We focus on self-distillation policy optimization (SDPO, [14]) and compare it with sequence-level reward optimization (GRPO, [17]), from two perspectives:

To answer these questions, we conduct post-training experiments in both single-domain and multi-domain continual settings. We vary the supervision density by comparing GRPO, standard SDPO, and SDPO variants with additional teacher strategies (3.1) or supervision tweaks (3.2), and evaluate on in-distribution and general benchmarks to assess specialization, retention, and transfer ([sec:sdpo_vs_grpo:single,sec:sdpo_vs_grpo:cl]). Finally, we connect behavior to diagnostics—parameter and response drift (5.1), collapse modes (5.2), and theoretical analysis (5.3) to interpret our observations.

Our results show that on-policy self-distillation offers a powerful yet fragile training signal (1). SDPO can substantially improve performance on the current training domain, but it also increases the risk of drift, interference, and even collapse. Our observations suggest that supervision density introduces a trade-off: it strengthens the local learning signal, but also increases sensitivity, domain mismatch, and accumulated artifacts. Our main contributions are:

• We separate two factors often conflated in continual post-training: on-policy data and the objective used to turn on-policy rollouts into updates. This lets us ask whether on-policy self-distillation inherits the retention benefits attributed to on-policy RL.

• We show that SDPO is a strong specialization method but a fragile continual-learning signal. Its benefits depend on teacher stability and token reliability; CoT and fast teacher updates can amplify noise and artifacts.

• Across single-domain and staged continual post-training, SDPO exhibits weaker retention than GRPO, with forgetting concentrated on neighboring-yet-misaligned tasks and accumulating across domains.

• Parameter drift, response drift, collapse analysis, and an excess-KL view diagnose how and why SDPO yields such forgetting behavior.

2 Preliminaries↩︎

2.1 Post-Training of Large Language Models↩︎

2.1.0.1 Reinforcement Learning for Post-Training.

Reinforcement learning is widely used to post-train large language models with preference signals or reward functions derived from human feedback, automated evaluators, or verifiable correctness criteria [18]–[21]. Recent methods, such as direct preference optimization [22], recast preference optimization as a supervised objective, while on-policy algorithms like group relative policy optimization [17] optimize relative advantages over sampled response groups, often improving stability and scalability in large-scale settings.

2.1.0.2 On-Policy Distillation.

On-policy distillation (OPD) methods construct training targets from samples generated by the current policy or its improved variants, thereby converting reinforcement signals into supervised learning updates [10], [11]. Recent self-distillation methods remove the need for a separate teacher by running the same model under different information conditions: the student observes only the original prompt, while the teacher is augmented with privileged context \(c\). The privileged signal can come from demonstrations or examples [13], [23], correct solutions or textual feedback [14], or revision contexts, hints, and task-level feedback transformations [24], [25].

2.1.0.3 Chain-of-Thought Reasoning.

Chain-of-thought (CoT) prompting and training encourage LLMs to generate intermediate reasoning before final answers, often improving performance on arithmetic, symbolic, and commonsense tasks [26], [27]. Under final-answer supervision, this lengthens the credit-assignment horizon; with step-level or process rewards, it offers denser feedback and improves interpretability [28], [29]. CoT also has drawbacks: it increases inference cost, may propagate errors across steps, and the generated rationales need not faithfully reflect the model’s actual decision process [30].

2.1.0.4 Continual Post-Training.

Continual learning [31] studies how models incorporate new knowledge without catastrophic forgetting of acquired ones. Common approaches use regularization, replay, or reference-model constraints to mitigate catastrophic forgetting across training stages [32]–[34]. A growing line of work shows that RFT can substantially mitigate forgetting in continual post-training [5]–[7]. For example, [5] find that this benefit is not mainly due to explicit KL penalties or chain-of-thought prompting, but rather to the implicit regularization induced by reward-guided policy updates. Complementarily, [7] argue that online RL is biased toward KL-minimal solutions among the policies that solve a new task, thereby reducing the distributional shift from the base model that is predictive of forgetting. Similarly, [6] identify the use of on-policy data as a key factor behind RL’s robustness.

2.2 Group Relative Policy Optimization↩︎

Group relative policy optimization (GRPO, [17]) assigns rewards at the sequence level. Given a prompt \(x \sim \mathcal{D}\), GRPO samples a group of \(G\) responses \(\{y_i\}_{i=1}^G\) from the current policy, i.e., \(y_i \sim \pi_{\theta_{\mathrm{old}}}(\cdot \mid x)\), evaluates each response with a reward model to obtain rewards \(\{r_i\}_{i=1}^G\), and computes group-relative advantages by normalizing rewards within the group, e.g., \(A_i = (r_i-\mathrm{mean}(\{r_j\}_{j=1}^G))/(\mathrm{std}(\{r_j\}_{j=1}^G)+\epsilon)\). Training optimizes a PPO-style clipped objective: \[\mathcal{J}_{\mathrm{GRPO}}(\theta) = \frac{1}{G}\sum_{i=1}^G \frac{1}{|y_i|}\sum_{t=1}^{|y_i|} \min\bigl( \rho_{i,t} A_i, \mathrm{clip}(\rho_{i,t},1-\varepsilon,1+\varepsilon)A_i \bigr) - \lambda D_{\mathrm{KL}}\bigl(\pi_\theta(\cdot\mid x)\parallel\pi_{\mathrm{ref}}(\cdot\mid x)\bigr), \label{eq:grpo}\tag{1}\] where \(\rho_{i,t}=\frac{\pi_\theta(y_{i,t}\mid x,y_{i,<t})}{\pi_{\theta_{\mathrm{old}}}(y_{i,t}\mid x,y_{i,<t})}\) and \(\pi_{\mathrm{ref}}\) is a reference policy used for KL regularization. In essence, GRPO is a critic-free policy optimization method that stabilizes reinforcement learning by using relative rewards among multiple sampled responses to the same prompt.

2.3 On-Policy Self-Distillation↩︎

Self-distillation policy optimization (SDPO, [14]) instead provides dense token-level rewards using the same model as both teacher and student. For each student-generated response \(y\sim\pi_{\theta}(\cdot\mid x)\), the student is trained to align with the more-informed teacher \(\pi_{\theta}(\cdot\mid x,c)\) having richer context \(c\) (including solutions or other auxiliary signals). Training minimizes the generalized Jensen–Shannon divergence (JSD, 5 ) between student and teacher distributions: \[\mathcal{L}_{\mathrm{SDPO}}(\theta) = \sum_{t} D_\mathrm{JSD}(\beta) \left( \pi_{\theta}(\cdot \mid x, y_{<t}) \parallel \mathrm{stopgrad}\left(\pi_{\theta}(\cdot \mid x, c, y_{<t})\right) \right).\] This yields dense supervision: instead of a single reward for the whole sequence, SDPO encourages student distribution at each generated token to align with more informed teacher distribution.

3 Token-Level Dense Supervision is Powerful but Sensitive↩︎

First, we claim that SDPO’s benefits depend on the quality of the dense signal. SDPO distills token logits from the teacher model, improving sample efficiency because every generated token contributes supervision. However, it also means that teacher noise, temporal mismatch, and formatting artifacts are reinforced many times within a single response. We therefore begin by asking: how do token-level reward stability (3.1) and density (3.2) affect self-distillation?

3.1 Token-Level Supervision Requires Fresh but Temporally Stable Teachers↩︎

A natural intuition in SDPO is that the teacher should be updated frequently: a fresher teacher should better reflect the student’s latest improvements and therefore provide stronger token-level supervision. To test this intuition, we sweep the EMA update rate \(\alpha \in \{0,0.5\%,1\%,2\%,5\%\}\): \[\Theta_{\text{teacher}}^{(k)} = (1-\alpha)\Theta_{\text{teacher}}^{(k-1)} + \alpha\Theta_{\text{student}}^{(k)} .\] Surprisingly, freshness alone does not predict performance. As shown in 2 (a), no single \(\alpha\) is uniformly optimal: in Math-SDPO, \(\alpha=1\%\) gives the best overall profile, while \(\alpha=5\%\) performs worst; in ToolUse training, the frozen teacher is best, although \(\alpha=5\%\) is competitive. This raises a question: if a larger EMA rate makes the teacher fresher, why can it hurt SDPO so severely?

We argue that EMA entangles two different properties of the teacher: freshness and temporal stability. A larger \(\alpha\) makes the teacher weights closer to the current student, but it also makes the teacher a more rapidly moving target. This is particularly risky for dense distillation: because SDPO applies supervision at every generated token, small teacher fluctuations can be reinforced repeatedly within a single response. The training curves in 2 (b) support this view. Large EMA rates, especially \(\alpha=2\%\) and \(\alpha=5\%\), observe late-stage increases in entropy and JS divergence, indicating growing teacher–student mismatch and less reliable token targets. Smaller or frozen teachers avoid this instability, but they introduce the opposite problem: the teacher can become stale and provide a weakening optimization signal as the student evolves.

To separate freshness from per-step teacher dynamics, we introduce a simple restart-and-freeze strategy named StableSDPO. Instead of updating the teacher continuously, we periodically refresh it from the current student and then keep it frozen within the next interval: \[\Theta_{\text{teacher}} \leftarrow (1-\gamma)\Theta_{\text{teacher}}^{\text{old}} + \gamma\Theta_{\text{student}}^{(k)}, \quad 0 < \gamma \leq 1 .\] This design preserves teacher freshness at refresh points while removing the step-to-step volatility of EMA updates. For comparison with EMA, we match the effective \(k\)-step refresh amount by \(\gamma \approx 1-(1-\alpha)^k\); for example, \(\alpha=5\%\) over \(k=10\) steps corresponds to \(\gamma \approx 0.40\).

\begin{table}[t] 9pt

-.5em \end{table}

The results in ¿tbl:tab:sdpoema:restart? strongly support this decomposition. Replacing continuous EMA with matched restart-and-freeze substantially repairs the destructive behavior of \(\alpha=5\%\): on AIME, performance improves from 34.38% to 55.00%, and the average score improves from 65.34% to 71.84%, a gain of 6.50 points. Importantly, StableSDPO also improves over the frozen teacher: hard restart with \(\gamma=1\) outperforms the frozen teacher on all five benchmarks, improving the average score from 71.88% to 72.94%. Thus, the problem is not that freshness is harmful. Rather, freshness is useful only when it is delivered through a stable teacher signal. The training dynamics in 3 further illustrate this point. The frozen teacher produces smooth curves, but its entropy and JS divergence gradually decline, suggesting that the teacher becomes increasingly outdated. In contrast, restart-and-freeze produces bounded oscillations aligned with refresh events: each restart renews the teacher signal, and avoids noisy per-step target changes. The JS divergence remains bounded, with neither late-stage blow-up nor collapse.

3.2 CoT Distillation: More Supervision Is Not Always Better↩︎

We next consider the supervision density of SDPO, by additionally distilling Chain-of-Thought tokens. This is a stronger test of the dense-supervision hypothesis: if more token-level targets were uniformly beneficial, CoT distillation should consistently improve performance. 4 (a) shows the opposite: CoT helps ToolUse training but hurts Math and Science training.

We attribute this contrast to the reliability of intermediate tokens. CoT distillation does not merely add more supervision; it changes the target from answer-producing behavior to a particular reasoning trajectory. In long-form Math and Science tasks, such trajectories are underdetermined and often contain redundant self-checks, uncertain interpretations, formatting habits, or incorrect intermediate claims. These tokens are weakly tied to final correctness, so distilling them can dilute credit assignment and encourage imitation of reasoning artifacts. This is reflected in Math training by longer responses, higher entropy, and larger JS divergence (4 (b)).

By contrast, ToolUse traces are short and schema-constrained. CoT therefore acts as useful process supervision in ToolUse training, yielding stable dynamics and improved BFCLv4 performance (4 (c)). Additional EMA ablations further show that long-CoT supervision becomes especially harmful when the teacher is updated rapidly (5), consistent with a confirmation-bias loop in which noisy reasoning patterns are copied into the teacher and reinforced again.

4 SDPO Trades Retention for Specialization↩︎

The previous section reveals that SDPO’s dense signal is conditional: it helps only when the teacher is stable and the distilled tokens are reliable. We now ask whether these local benefits translate into broader training advantages. Does token supervision improves out-of-distribution (OOD) generalization (4.1), and mitigates forgetting in continual post-training (4.2)?

4.1 Rethinking OOD Generalization: Intermediate-Distance Interference↩︎

\begin{table}[t] 4pt

-.5em \end{table}

SDPO’s main advantage is specialization, not uniformly better generalization (¿tbl:tab:singledomain:result?). Under Math training, \(\text{SDPO}_0\) is the strongest math specialist, raising AIME from 32.71% to 56.42% and Math500 from 73.85% to 84.29%. Similarly, Coding-\(\text{SDPO}_0\) improves LCBv6 by 13.39%. These results confirm that token-level supervision can strongly exploit source-like evaluation distributions.

However, this specialization does not translate into uniform retention. We decompose relative performance change into positive gain and negative forgetting ([eq:gain,eq:forgetting]), and demonstrate that the SDPO-induced forgetting pattern is non-monotonic, as 5 plots gain and forgetting against the distributional distance measured by Maximum Mean Discrepancy (MMD, [35]). Source-like tasks (e.g., \(\textrm{MMD}\lesssim0.05\)) and very distant tasks (e.g., \(\textrm{MMD}\gtrsim0.4\)) mostly improve, while the largest drops concentrate in intermediate-distance tasks.

GPQA is the clearest intermediate-forget example: it is not the farthest benchmark, but consistently degrades after SDPO training from multiple source domains. We interpret this pattern as evidence that GPQA may lie in an interference region: it is close enough to be affected by source-domain updates, yet insufficiently aligned to benefit from them. More broadly, we observe SDPO degradation on Math-to-Science, Math-to-ZLogic, and Science-to-BFCLv4 (more cases are in 6).

These results suggest that SDPO’s OOD failures are consistent with an intermediate-distance interference pattern, rather than a simple monotonic function of distributional distance. Distillation specializes the model toward the source-domain response manifold. When an OOD task is very close, this specialization transfers. When it is very far, the perturbation may not strongly affect it. The most vulnerable tasks are partially aligned OOD tasks: close enough to be changed, but misaligned enough not to benefit. This non-monotonic pattern is consistent with prior theory of continual learning with linear models [36], [37], and our results show that a similar interference regime emerges in LLM continual post-training,

4.2 Continual Learning Exposes the Accumulated Cost of Dense Supervision↩︎

6pt

Single-domain interference becomes more severe when updates are composed across domains. We therefore train models sequentially on Math \(\rightarrow\) Science \(\rightarrow\) ToolUse \(\rightarrow\) Coding and evaluate after each stage. We report both Current, i.e., performance on the current domain immediately after training it, and Last, i.e., performance after the full training sequence.

The continual setting reverses the single-domain picture. \(\text{SDPO}_0\) is a strong early specializer: it reaches 56.42% on Math and 65.70% on Science, exceeding GRPO’s 44.67% and 56.39%. However, these gains do not survive later domain shifts: after the full sequence, \(\text{SDPO}_0\) drops to 34.38% on Math and 9.93% on ToolUse, and also falls below the base model on Logic and Knowledge. \(\text{SDPO}_0^{\text{CoT}}\) degrades even earlier, consistent with the instability of long-CoT token targets. The most severe case is \(\text{SDPO}_{5\%}\), which collapses after the third stage and remains near zero. By contrast, GRPO adapts more conservatively but retains capabilities more reliably, achieving final improvements over the base model across all evaluated categories ([fig:cl,tab:cl]).

These results argue that single-domain specialization is not evidence of continual learning. SDPO’s dense token-level updates can accelerate local adaptation, but the same updates also accumulate drift, interference, and artifacts across domain shifts. Sequence-level GRPO is less aggressive, but its more selective reward signal better supports long-horizon retention.

5 Understanding SDPO in Continual Post-training↩︎

5.1 Dense Supervision Causes Larger Parameter and Response Drift↩︎

The benchmarks report that SDPO adapts more aggressively than GRPO. To understand why, we look inside the model and ask whether SDPO’s greater behavioral changes correspond to larger parameter shifts. For each LLM weight matrix (e.g., self-attention \(V\)) at training step \(k\): \({\mathbf{W}}^{(k)}\in \mathbb{R}^{m \times n}\), we compute the compact singular value decomposition: \[{\mathbf{W}}^{(k)}= {\mathbf{U}}^{(k)}{\mathbf{\Sigma}}^{(k)}{{\mathbf{V}}^{(k)}}^{\top},\] where \({\mathbf{U}}^{(k)}\in \mathbb{R}^{m \times d}\), \({\mathbf{V}}^{(k)}\in \mathbb{R}^{n \times d}\), and \(d=\min(m,n)\). The SVD decomposes each matrix into input-output directions and their associated spectral strengths, and we use it to quantify how training changes the geometry, spectral magnitude, subspace orientation, and dominant parameter regions of the model. Specifically, we calculate the following metrics between \({\mathbf{W}}^{(0)}\) and \({\mathbf{W}}^{(k)}\):

• Normalized spectral shift: \(\ell_2\)-normalized drift of the singular-value profile in 11 . Larger values indicate substantial strength changes in dominant directions.

• Principal-angle rotation: maximum rotation within top-\(k\) left singular subspaces, as shown in 14 . A small value implies better preservation of dominant subspaces.

• Principal-mask overlap: intersection-over-union of principal masks before and after training per 15 . High values mean dominant entries remain important.

• Principal-mask update sparsity: intersection-over-union of the initial principal mask and the changed-parameter mask in 17 . A high value demonstrates that training directly modifies structurally important parameters.

[fig:svd:math,fig:svd:cl] highlight a consistent drift hierarchy. GRPO changes the model least: its spectral shift and subspace rotation remain small, and its principal-mask overlap stays near \(99.9\%\). SDPO variants move farther, with \(\text{SDPO}_0^{\text{CoT}}\) producing the largest subspace change. This pattern supports an accumulated-drift explanation of continual-learning failure: SDPO produces notably larger spectral shifts and more principal-parameter modifications than GRPO. The ordering \(\text{SDPO}_0^{\text{CoT}}>\text{SDPO}_{5\%}>\text{SDPO}_0>\text{GRPO}\) matches the aggressiveness of the supervision signal: CoT increases token coverage, while fast EMA increases target non-stationarity. This supports the view that denser or less stable token-level targets drive stronger internal updates.

Parameter drift is mirrored in the model’s outputs. Response-embedding similarities remain tightly concentrated under GRPO, whereas SDPO shifts the distribution downward and develops heavier lower tails ([fig:rollout:math,fig:rollout:cl], embedding computed by Qwen3-Embedding-4B, [38]). This suggests that SDPO does not merely improve selected answers; it reshapes the response manifold more strongly than sequence-level GRPO.

5.2 Collapse as Artifact Amplification Under Dense Self-Distillation↩︎

Finally, we diagnose one extreme failure mode as a case of artifact amplification in self-distillation. \(\text{SDPO}_{5\%}\) collapses with a visible generation artifact: endlessly repeated “\boxed” tokens. This behavior coincides with a change in final-answer formatting requirement at the third stage, where ToolUse differs from earlier Math and Science stages (data samples in 9.2). The collapsed model is over-amplifying the high-frequency pattern from earlier on-policy training stages, as JS divergence (8) plummets in the third stage, even to zero within the first few steps of ToolUse training. More experiments show that reordering data or masking tokens only partially mitigates this collapse ([appdx:tab:cl,appdx:tab:cl_collapse]).

This failure is consistent with the central diagnostic observed throughout our analyses. Because supervision is token-level, frequent tokens (e.g., formatting patterns) can receive repeated reinforcement. As the teacher is derived from the student, such artifacts can enter a feedback loop we call confirmation bias: the student emits the artifact, the teacher assigns it token-level support, and the next student update makes the artifact more likely. A frequently refreshed teacher or denser supervision can worsen this process and, in severe cases, lead to near-complete collapse.

5.3 Theoretical View on Forgetting: Excess KL Drift↩︎

Existing studies have revealed that on-policy RL is biased toward solutions that solve the new task while staying close to the base policy in KL [7]. However, SDPO does not inherit this bias: its rollouts are on-policy, but its local target is a teacher distribution conditioned on augmented context, rather than the KL-minimal distribution induced by a sequence-level reward.

Let \(x\sim\mathcal{D}\) be a prompt, \(y\) a response, and \(S_x\subseteq\mathcal{Y}\) the set of successful responses under a binary sequence reward. Let \(P_0(\cdot\mid x)\) be the sequence distribution induced by the base policy \(\pi_0\). For any success level \(\alpha\in(0,1)\), define the matched-success Razor policy \[Q_{\alpha,x}^{\dagger} := \mathop{\mathrm{arg\,min}}_{Q:\,Q(S_x)=\alpha} D_{\mathrm{KL}}\bigl(Q\parallel P_0(\cdot\mid x)\bigr). \label{eq:razor95policy95def}\tag{2}\] Let \(q_{\alpha,x}^{\dagger}(\cdot\mid h_t)\) be its autoregressive token kernel at history \(h_t=(x,y_{<t})\). This is the token distribution induced by the sequence-level reward: it preserves the base policy except for the reweighting needed to change the probability of eventual success. In the exact-success limit \(\alpha\to1\), \(Q_{\alpha,x}^{\dagger}\) becomes the base-conditioned solution \(P_0(\cdot\mid x,y\in S_x)\).

In contrast, SDPO distills a teacher that observes augmented context \(c\), such as a solution, rationale, or auxiliary signal. Its local target is \(T_c(\cdot\mid h_t)=\pi_T(\cdot\mid x,c,y_{<t})\). For a strict distillation divergence \(D\), e.g., generalized JSD, the population token-level target is the teacher projection \[B(\cdot\mid h_t) \in \mathop{\mathrm{arg\,min}}_{u\in\Delta(\mathcal{V})} \mathbb{E}_{c\mid h_t} D\bigl(u,T_c(\cdot\mid h_t)\bigr). \label{eq:teacher95projection}\tag{3}\] When \(c\) is deterministic, this reduces to \(B(\cdot\mid h_t)=T_c(\cdot\mid h_t)\). Let \(P_B(\cdot\mid x)\) be the sequence distribution induced by this SDPO token target and set \(\alpha_x=P_B(S_x\mid x)\).

The no-excess-drift case is therefore restrictive: SDPO must satisfy \(B(\cdot\mid h_t)=q_{\alpha_x,x}^{\dagger}(\cdot\mid h_t)\) at every visited history. This is not implied by teacher–student parameter sharing. Because the teacher conditions on augmented context \(c\), it may select a particular reasoning trace, formatting pattern, or artifact even when many continuations are equally successful. Thus, SDPO’s on-policy sampling controls where supervision is applied, but the distribution selected at each token is the teacher projection, not the KL-minimal sequence-reward projection. Combined with the KL-forgetting law, the excess drift in ?? predicts additional forgetting. Full proofs are in 8.

6 Conclusion↩︎

This paper studied whether on-policy self-distillation improves continual post-training. Our results indicate that SDPO can be an effective specialization method when teacher signals are stable and aligned with the target domain. However, it is not uniformly beneficial for continual learning. SDPO’s token supervision amplifies both useful supervision and harmful artifacts, making it sensitive to teacher quality, update schedules, and token structure. Compared with GRPO, SDPO variants exhibit greater forgetting, instability, parameter drift, and response-manifold shifts.

The main lesson is not that on-policy self-distillation methods are ineffective, but they are not automatically safe. Token supervision can be a rapid specialization signal, but also a potentially dangerous signal: each token can reinforce useful signals and harmful artifacts. Sequence-level methods such as GRPO learn more slowly, but in our experiments, they preserve capabilities more reliably. Thus, the future of on-policy self-distillation for continual learning likely depends not on making the signal denser, but on safeguarding it through token weighting, teacher controls, or data balancing strategies to be more selective, stable, and artifact-aware.

7 Math Notations↩︎

7.0.0.1 Kullback–Leibler (KL) divergence.

For two discrete distributions \(P(\mathcal{X})\) and \(Q(\mathcal{X})\), \[D_{\mathrm{KL}}(P \parallel Q) = \sum_{x\in \mathcal{X}} P(x)\log\!\left(\frac{P(x)}{Q(x)}\right). \label{eq:kl}\tag{4}\]

7.0.0.2 Generalized Jensen–Shannon (JS) divergence.

A weighted average of two KL divergences from \(P\) and \(Q\) to their mixture with \(\beta\in(0,1)\) (\(\beta=0.5\) recovers the standard symmetric JS divergence): \[D_\mathrm{JSD}(\beta)(P\parallel Q) = \beta {D_\mathrm{KL}(P \parallel\beta P + (1 - \beta)Q)}+ (1-\beta) D_\mathrm{KL}(Q \parallel\beta P + (1 - \beta)Q). \label{eq:jsd}\tag{5}\]

7.0.0.3 Performance Change.

We measure absolute (\(\Delta\)) and relative (\(\delta\)) change against the base model: \[\begin{align} \Delta_{m,\mathrm{train},\mathrm{eval}}={P_{m,\mathrm{train},\mathrm{eval}}}-{P_{\mathrm{base},\mathrm{eval}}},\tag{6}\\ \delta_{m,\mathrm{train},\mathrm{eval}}=\frac{\Delta_{m,\mathrm{train},\mathrm{eval}}}{P_{\mathrm{base},\mathrm{eval}}}\times 100\%,\tag{7} \end{align}\] where \(m\) is the training method, \(\mathrm{train}\) is the training domain, and \(\mathrm{eval}\) is the evaluation benchmark. We define gain \(G\) and forgetting \(F\) as the positive and negative part of the relative change: \[\begin{align} G_{m,\mathrm{train},\mathrm{eval}}=\max(0,\delta_{m,\mathrm{train},\mathrm{eval}}),\tag{8}\\ F_{m,\mathrm{train},\mathrm{eval}}=\max(0,-\delta_{m,\mathrm{train},\mathrm{eval}}).\tag{9} \end{align}\]

7.0.0.4 Intersection-over-Union.

For two masks \(A\) and \(B\) (e.g., predicted mask and ground-truth mask), \[\label{eq:iou} \mathop{\mathrm{IoU}}(A,B) = \frac{|A\cap B|}{|A\cup B|},\tag{10}\] where \(|\cdot|\) denotes cardinality. The quantity lies in \([0,1]\), with \(1\) for a perfect overlap.

7.0.0.5 Normalized spectral shift.

It is defined as the \(\ell_2\)-normalized drift of the singular-value profile: \[\label{eq:nss} \mathrm{NSS}\left({\mathbf{W}}^{(0)},{\mathbf{W}}^{(k)}\right) = \tfrac{\left\lVert\sigma\!\left({\mathbf{W}}^{(0)}\right)-\sigma\!\left({\mathbf{W}}^{(k)}\right)\right\rVert_2} {\left\lVert\sigma\!\left({\mathbf{W}}^{(0)}\right)\right\rVert_2}.\tag{11}\]

7.0.0.6 Principal-mask (PM) and Principal-angles (PAs).

Given \({\mathbf{W}}^{(k)}_r = {\mathbf{U}}^{(k)}_r {\mathbf{\Sigma}}^{(k)}_r {{\mathbf{V}}^{(k)}_r}^{\top}\) as the rank-\(r\) approximation, PM is \[\label{eq:pm} {\mathbf{M}_{\mathrm{principal}}}^{(k)}[i,j]= \mathbb{I} \left( \left| {\mathbf{W}}^{(k)}_r[i,j] \right| \geq {\tau}^{(k)}_{\eta} \right),\tag{12}\] where \({\tau}^{(k)}_{\eta}\) is a threshold controlled by the hyperparameter \(\eta\). Principal angles are \[\label{eq:pa} \theta_i({\mathbf{W}}^{(0)},{\mathbf{W}}^{(k)}) = \arccos \sigma_i\left({{\mathbf{U}}^{(0)}_r}^{\top}{\mathbf{U}}^{(k)}_r\right), \qquad i=1,\dots,k,\tag{13}\] where \(\sigma_i(\cdot)\) denotes the \(i\)-th singular value.

7.0.0.7 Maximum principal-angle rotation.

Max PA of the top-\(r\) left singular vectors: \[\label{eq:maxpa} \mathrm{maxPA}\left({\mathbf{W}}^{(0)},{\mathbf{W}}^{(k)}\right) = \max_{i \leq r} \theta_i({\mathbf{W}}^{(0)},{\mathbf{W}}^{(k)}).\tag{14}\]

7.0.0.8 Principal-mask overlap.

Intersection-over-Union between the initial and current PM masks: \[\label{eq:pmoverlap} \mathrm{PMOverlap}\left({\mathbf{W}}^{(0)},{\mathbf{W}}^{(k)}\right) =\mathop{\mathrm{IoU}}\left({\mathbf{M}_{\mathrm{principal}}}^{(0)},{\mathbf{M}_{\mathrm{principal}}}^{(k)}\right).\tag{15}\]

7.0.0.9 Changed-parameter-mask.

Per-parameter mask with change threshold \(\eta>0\): \[\label{eq:cm} {\mathbf{M}_{\mathrm{param}}}^{(k)}[i,j] = \mathbb{I} \left( \left| \left({\mathbf{W}}^{(k)}-{\mathbf{W}}^{(0)}\right)[i,j] \right| \geq \eta \right).\tag{16}\]

7.0.0.10 Principal-mask update sparsity.

Intersection-over-Union between the initial principal mask and the changed-parameter mask: \[\label{eq:cmoverlap} \mathrm{PMUpdateSparsity}\left({\mathbf{W}}^{(0)},{\mathbf{W}}^{(k)}\right) = \mathop{\mathrm{IoU}}{ \left({\mathbf{M}_{\mathrm{principal}}}^{(0)}, {\mathbf{M}_{\mathrm{param}}}^{(k)}\right) }.\tag{17}\]

8 Proofs↩︎

We provide the proofs for results in 5.3. All distributions are over finite spaces. We assume that the base sequence distribution \(P_0(\cdot\mid x)\) has full support. When the prompt \(x\) is fixed, we suppress it from notation.

8.0.0.1 Matched-success Razor policy.

Let \(S=S_x\) be the set of successful responses and let \(z=P_0(S\mid x)\in(0,1)\). For a target success level \(\alpha\in(0,1)\), define \[\mathcal{Q}_{\alpha} = \{Q\in\Delta(\mathcal{Y}): Q(S)=\alpha\}.\]

8.0.0.2 Teacher projection and excess reverse KL.

For a strict distillation divergence \(D\), the population SDPO token target is \[B(\cdot\mid h_t) \in \arg\min_{u\in\Delta(\mathcal{V})} \mathbb{E}_{c\mid h_t} D\bigl(u,T_c(\cdot\mid h_t)\bigr).\] When \(c\) is deterministic and \(D\) is generalized JSD, the minimizer is \(B(\cdot\mid h_t)=T_c(\cdot\mid h_t)\) because generalized JSD is nonnegative and equals zero if and only if its two arguments are identical.

Let \(P_B(\cdot\mid x)\) be the sequence distribution induced by \(B(\cdot\mid h_t)\), and let \[\alpha=P_B(S_x\mid x).\] The corresponding matched-success Razor policy is \(Q_{\alpha,x}^{\dagger}\).

8.0.0.3 Forward KL counterpart.

The previous theorem uses reverse KL because RL’s Razor is naturally stated as a reverse-KL projection. The empirical forgetting law is often measured by forward KL from the base policy to the fine-tuned policy, so we also give the matched-success forward-KL analogue.

8.0.0.4 Connection to forgetting.

We use the KL-forgetting law as an empirical bridge between distributional drift and prior-task forgetting.

9 Training Details↩︎

9.1 Models and Datasets↩︎

We adopt Qwen3 series (Qwen3-4B-Instruct-2507 and Qwen3-4B-Thinking-2507, [3]), Olmo-3 series (Olmo-3-7B-Instruct-DPO and Olmo-3-7B-Think-DPO, [39]) as base models² and train³ on 4 diverse tasks—Math, Science, ToolUse, and Coding, with in-distribution and out-of-distribution⁴ test sets to evaluate model performance measured in averaged accuracy (i.e., Acc@k). Data sources and training hyperparameters (learning rates according to [14]) are disclosed in [appdx:tab:datasets,appdx:tab:hparams], respectively. We visualize the embeddings of each benchmark using Qwen3-Embedding-4B [38] in 9.

9.2 Data Samples↩︎

Every morning Aya goes for a $9$-kilometer-long walk and stops at a coffee shop afterwards. When she walks at a constant speed of $s$ kilometers per hour, the walk takes her 4 hours, including $t$ minutes spent in the coffee shop. When she walks $s+2$ kilometers per hour, the walk takes her 2 hours and 24 minutes, including $t$ minutes spent in the coffee shop. Suppose Aya walks at $s+\frac{1}{2}$ kilometers per hour. Find the number of minutes the walk takes her, including the $t$ minutes spent in the coffee shop.
Please reason step by step, and put your final answer within \boxed{}.

Your task is to answer the user's question using available tools.
You have access to the following tools:
Name: Tools
Description: A set of functions/tools that you can use to answer the user's question.
Documentation:
math.triangle_area_heron: Calculates the area of a triangle using Heron's formula, given the lengths of its three sides.
Parameters: {"side1": "integer. Length of the first side of the triangle.", "side2": "integer. Length of the second side of the triangle.", "side3": "integer. Length of the third side of the triangle."}
Output: Successful response.
 - Format: application/json
math.circle_area: Calculates the area of a circle given its radius.
Parameters: {"radius": "float. The radius of the circle."}
Output: Successful response.
 - Format: application/json
math.triangle_area_base_height: Calculates the area of a triangle using the formula (1/2)base*height.
Parameters: {"base": "float. The base length of the triangle.", "height": "float. The height of the triangle."}
Output: Successful response.
 - Format: application/json

Use the following format:
Thought: you should always think about what to do
Action: the action to take, should be one of the tool names.
Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user.

Begin!
Question: Calculate the area of a triangle, given the lengths of its three sides: 3, 4, and 5.

You are tasked with creating a function that retrieves the details of a specific assignment based on its ID. The function should take a string representing the assignment ID as input and return a dictionary containing the assignment details. For the purpose of this question, you can assume that the assignment details will be represented as a fixed dictionary for demonstration. Implement a function named `get_assignment_details` that takes an assignment ID and returns the corresponding assignment details as a dictionary. If the assignment ID does not exist, return an empty dictionary. The assignment details for valid IDs are predefined as follows: '101': {'title': 'Math Homework', 'due_date': '2023-11-01'}, '102': {'title': 'Science Project', 'due_date': '2023-12-15'}, '103': {'title': 'History Essay', 'due_date': '2023-10-30'}.

Two quantum states with energies E1 and E2 have a lifetime of 10^-9 sec and 10^-8 sec, respectively. We want to clearly distinguish these two energy levels. Which one of the following options could be their energy difference so that they can be clearly resolved?

A. 10^-8 eV
B. 10^-4 eV
C. 10^-11 eV
D. 10^-9 eV
Please reason step by step, and put your final answer within \boxed{}.

In triangle $ABC$, $\sin \angle A = \frac{4}{5}$ and $\angle A < 90^\circ$. Let $D$ be a point outside triangle $ABC$ such that $\angle BAD = \angle DAC$ and $\angle BDC = 90^\circ$. Suppose that $AD = 1$ and that $\frac{BD}{CD} = \frac{3}{2}$. If $AB + AC$ can be expressed in the form $\frac{a\sqrt{b}}{c}$ where $a, b, c$ are pairwise relatively prime integers, find $a + b + c$.
Please reason step by step, and put your final answer within \boxed{}.

Convert the point $(0,3)$ in rectangular coordinates to polar coordinates.  Enter your answer in the form $(r,\theta),$ where $r > 0$ and $0 \le \theta < 2 \pi.$
Please reason step by step, and put your final answer within \boxed{}.

Determine whether the polynomial in Z[x] satisfies an Eisenstein criterion for irreducibility over Q. x^2 - 12
A. Yes, with p=2.
B. Yes, with p=3.
C. Yes, with p=5.
D. No.
Please reason step by step, and put your final answer within \boxed{}.

Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?

A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
Please reason step by step, and put your final answer within \boxed{}.

Your task is to answer the user's question using available tools. 
You have access to the following tools:
Name: Axolotl
Description: Collection of axolotl pictures and facts
Documentation:
getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source.
Parameters: {}
Output: Successful response.
 - Format: application/json
 - Structure: Object{url, source, description}
searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size.
Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."}
Output: Successful response.
 - Format: application/json
 - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}}
getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics.
Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."}
Output: Successful response.
 - Format: application/json
 - Structure: Array[Object{fact, source}]

Use the following format:
Thought: you should always think about what to do
Action: the action to take, should be one of the tool names.
Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user.

Begin!
Question: Hey, can you show me a random picture of an axolotl?

There are 6 houses, numbered 1 to 6 from left to right, as seen from across the street. Each house is occupied by a different person. Each house has a unique attribute for each of the following characteristics:
 - Each person has a unique name: `Arnold`, `Peter`, `Eric`, `Alice`, `Bob`, `Carol`
 - People have unique favorite book genres: `biography`, `science fiction`, `fantasy`, `mystery`, `romance`, `historical fiction`
 - People have unique favorite sports: `baseball`, `basketball`, `swimming`, `volleyball`, `tennis`, `soccer`
 - People own unique car models: `honda civic`, `ford f150`, `tesla model 3`, `chevrolet silverado`, `bmw 3 series`, `toyota camry`

## Clues:
1. Eric is the person who loves mystery books.
2. The person who loves tennis is the person who loves fantasy books.
3. The person who loves soccer is directly left of the person who loves science fiction books.
4. There is one house between the person who owns a Honda Civic and the person who loves biography books.
5. Peter is somewhere to the right of Carol.
6. The person who loves tennis is in the first house.
7. The person who owns a Tesla Model 3 is somewhere to the right of the person who loves baseball.
8. Eric is somewhere to the left of the person who loves romance books.
9. The person who owns a Toyota Camry is somewhere to the right of the person who loves romance books.
10. The person who owns a BMW 3 Series is Peter.
11. The person who owns a BMW 3 Series is the person who loves basketball.
12. The person who owns a Tesla Model 3 is directly left of Arnold.
13. Alice and the person who loves volleyball are next to each other.
14. The person who loves historical fiction books is the person who loves soccer.
15. The person who owns a Chevrolet Silverado is not in the first house.
16. The person who loves science fiction books is directly left of the person who loves swimming.

What is Name of the person who lives in House 5?
A. Eric
B. Bob
C. Alice
D. Peter
E. Carol
F. Arnold
Please reason step by step, and put your final answer within \boxed{}.

9.3 Model Chat Templates↩︎

<|im_start|>user\n{prompt}<|im_end|>
<|im_start|>assistant\n<think>

<|im_start|>user\n{prompt}<|im_end|>
<|im_start|>assistant\n

<|im_start|>system\nYou are a helpful function-calling AI assistant. You do not currently have access to any functions. <functions></functions><|im_end|>
<|im_start|>user\n{prompt}<|im_end|>
<|im_start|>assistant\n

<|im_start|>system\nYou are OLMo, a helpful function-calling AI assistant built by Ai2. Your date cutoff is November 2024, and your model weights are available at https://huggingface.co/allenai. You do not currently have access to any functions. <functions></functions><|im_end|>
<|im_start|>user\n{prompt}<|im_end|>
<|im_start|>assistant\n<think>

9.4 Prompt Templates↩︎

{question}
Please reason step by step, and put your final answer within \boxed{}.

Your task is to answer the user's question using available tools. 
You have access to the following tools:
{tool}

Use the following format:
Thought: you should always think about what to do
Action: the action to take, should be one of the tool names.
Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user.

Begin!
Question: {question}

{question}
Please reason step by step, and put your final answer within \boxed{}.
Correct solution: {previously correct solution}
Correctly solve the original question.

10 Supplementary Results↩︎

7.5pt

4 presents the performance scores used to plot 2 (a), and displays a configuration matrix of EMA teachers \(\alpha\in\{0, 0.5\%, 1\%, 2\%, 5\%\}\) and training datasets Math, ToolUse, and Science. 5 extends 4 for SDPO versus \(\text{SDPO}^{\text{CoT}}\) under more teacher EMA settings and evaluation benchmarks. The large degradation at \(\alpha=1\%, 5\%\) further verifies confirmation bias.

4pt

4pt

For single-domain training, ¿tbl:appdx:tab:singledomain? examines SDPO and GRPO for Olmo-3 series. For continual training, 8 presents detailed scores for [fig:cl,tab:cl] across each stage, with a data-ordering ablation on collapsed \(\text{SDPO}_{5\%}\) in 7. We supplement diagnoistics (7) with additional matrices (10), more training (11), and more benchmarks (12).

2.5pt

11 Generation Samples↩︎

This is a complex or challenging question, and it is difficult to provide a direct and correct answer. I need to think about it.\nWell, let's tackle this problem step by step. So, first, let's make sure I understand what's given and what's being asked. Aya does a 9 km walk every morning, but wait, hold on\u2014wait, no, wait: "a 9-kilometer-long walk"\u2014does that mean the walk itself is 9 km, not including the coffee shop? Wait, the problem says "the walk takes her 4 hours, including t minutes spent in the coffee shop." Hmm, okay, so the "walk" here is the time from when she starts walking until she gets to the coffee shop? Wait, no, maybe better to parse the problem carefully.

"Every morning Aya goes for a 9-kilometer-long walk and stops at a coffee shop afterwards." So, the walk is 9 km long, then she stops at a coffee shop (so the coffee shop is the endpoint of the walk? Maybe the walk is the 9 km journey to the coffee shop, and then she spends t minutes there? Wait, but the problem says "the walk takes her 4 hours, including t minutes spent in the coffee shop." Wait, that's confusing. If the walk is the 9 km, then why is the time including time spent in the coffee shop? Maybe the problem is using "walk" to mean the entire activity? Wait, no, let's check the problem statement again:

"When she walks at a constant speed of s kilometers per hour, the walk takes her 4 hours, including t minutes spent in the coffee shop." Oh! Wait, maybe "the walk" here is a misnomer, but actually, the total time from when she starts walking until she finishes at the coffee shop (i.e., the time to walk the 9 km plus the time she spends at the coffee shop) is 4 hours? Wait, but the problem says "the walk takes her 4 hours, including t minutes spent in the coffee shop." Hmm, maybe the problem is translated or phrased a bit awkwardly, but let's think in terms of variables.

Let's define:

Let\u2019s let the time she spends walking (i.e., the time actually moving) be W hours, and the time spent at the coffee shop be T hours? Wait, no, the problem says t minutes, so maybe better to use minutes for the coffee shop time to avoid decimals, but maybe keep it in hours first.

Wait, let's try to set up equations based on the two scenarios.

First scenario: speed = s km/h, total time (walk + coffee shop) = 4 hours, which includes t minutes at coffee shop. Wait, but hold on\u2014if the walk is 9 km long, then the walking time should be distance over speed, right? So maybe the "walk" here is the 9 km, so the time she spends walking (moving) is 9/s hours, and then she spends t minutes at the coffee shop, so total time from start to finish (including coffee shop) is (9/s) + (t/60) hours. The problem says "the walk takes her 4 hours, including t minutes spent in the coffee shop." Wait, maybe the problem is using "walk" to mean the entire trip? Like, "the walk" is the activity from when she starts walking until she stops at the coffee shop, so the duration of that activity is 4 hours, which consists of the time walking (9 km) plus time at coffee shop (t minutes). That might be the right interpretation.

Yes, that makes sense. So "the walk takes her 4 hours"\u2014the entire walk activity (which includes walking the 9 km and then stopping at coffee shop) takes 4 hours total, with t minutes of that time spent at the coffee shop. So the rest of the time (4 hours minus t minutes) is spent actually walking the 9 km.

Similarly, second scenario: speed = s + 2 km/h, total walk activity time = 2 hours 24 minutes, which includes t minutes at coffee shop, so walking time is 2h24m - t minutes.

Okay, that seems to be the key interpretation. Let's formalize this.

Let me convert all times to hours to make the equations easier, or maybe keep t in minutes and convert the total times to minutes? Let's see. Let's try hours first, then convert t to hours.

First scenario:

Total time (including coffee shop) = 4 hours = 240 minutes.

Time spent at coffee shop = t minutes = t/60 hours.

Therefore, time spent walking = total time - coffee shop time = 240 - t minutes = (240 - t)/60 hours.

But she walks 9 km at speed s km/h, so walking time = 9 / s hours.

Therefore, we have equation 1: 9 / s = (240 - t)/60.

Second scenario:

Total time (including coffee shop) = 2 hours 24 minutes = 2 + 24/60 = 2.4 hours = 144 minutes.

Time spent at coffee shop = still t minutes (the problem says "including t minutes spent in the coffee shop" in both cases, so t is the same in both scenarios\u2014she spends t minutes at coffee shop regardless of walking speed, which is good, that's a constant we need to find or relate).

Therefore, time spent walking = total time - coffee shop time = 144 - t minutes = (144 - t)/60 hours.

She walks 9 km at speed s + 2 km/h, so walking time = 9 / (s + 2) hours.

Therefore, equation 2: 9 / (s + 2) = (144 - t)/60.

Okay, great! So now we have two equations with two variables: s (km/h) and t (minutes). We need to solve for s and t, then find the total time when speed is s + 1/2 km/h, which would be (walking time) + t minutes, so total time in minutes is (9 / (s + 1/2) * 60) + t, right? Wait, let's confirm: walking time at speed s + 1/2 is 9 / (s + 1/2) hours, which is 60*(9 / (s + 1/2)) minutes, then add t minutes for coffee shop, so total time in minutes is 540 / (s + 1/2) + t. Maybe better to keep everything in minutes to avoid confusion later.

Let's write equations 1 and 2 again, but let's multiply both sides by 60 to eliminate denominators, that might make it cleaner.

Equation 1: 9 / s = (240 - t)/60 \u21d2 multiply both sides by 60s: 540 = s(240 - t) \u21d2 540 = 240s - st. Let's call this Eq1.

Equation 2: 9 / (s + 2) = (144 - t)/60 \u21d2 multiply both sides by 60(s + 2): 540 = (s + 2)(144 - t). Let's call this Eq2.

Okay, now we have two linear equations in terms of s and t? Wait, no, actually, Eq1 and Eq2 are both linear in s and t, yes! Let's check:

Eq1: 240s - st = 540 \u21d2 s(240 - t) = 540

Eq2: (s + 2)(144 - t) = 540

Oh, nice! Both equal 540, so we can set them equal to each other to eliminate the 540.

So, s(240 - t) = (s + 2)(144 - t)

Let's expand the right-hand side: s(144 - t) + 2(144 - t) = 144s - st + 288 - 2t

Left-hand side: 240s - st

So set equal: 240s - st = 144s - st + 288 - 2t

Wait, look, we can subtract (-st) from both sides? Wait, let's subtract 144s - st from both sides to simplify.

Left side minus (144s - st) = (240s - st) - 144s + st = 96s

Right side minus (144s - st) = (144s - st + 288 - 2t) - 144s + st = 288 - 2t

So we get 96s = 288 - 2t

Oh, that's a nice simplification! The st terms canceled out, which is good, we didn't want to deal with quadratic equations. Let's write that as:

96s = 288 - 2t \u21d2 let's divide both sides by 2 to simplify: 48s = 144 - t \u21d2 t = 144 - 48s. Let's call this Eq3.

Okay, great, now we can plug Eq3 into one of the original equations to solve for s. Let's pick Eq1: s(240 - t) = 540.

First, compute 240 - t using Eq3: 240 - (144 - 48s) = 240 - 144 + 48s = 96 + 48s.

So Eq1 becomes: s(96 + 48s) = 540.

Wait, hold on: is that right? Wait, Eq1 was s(240 - t) = 540, yes, and we found 240 - t = 96 + 48s, so yes, s*(96 + 48s) = 540.

Let's write that as 48s\u00b2 + 96s - 540 = 0. Maybe we can simplify this equation by dividing all terms by 24? Let's see: 48/24=2, 96/24=4, 540/24=22.5, which is not integer. Divide by 12: 4s\u00b2 + 8s - 45 = 0. Wait, 48\u00f712=4, 96\u00f712=8, 540\u00f712=45, yes! Perfect, that's integer coefficients.

So 4s\u00b2 + 8s - 45 = 0. Now let's solve for s using quadratic formula.

Quadratic in standard form: as\u00b2 + bs + c = 0, solutions s = [-b \u00b1 \u221a(b\u00b2 - 4ac)]/(2a)

Here, a=4, b=8, c=-45.

Discriminant D = b\u00b2 - 4ac = 64 - 4*4*(-45) = 64 + 720 = 784. Oh, nice, 784 is a perfect square! \u221a784 = 28.

So s = [-8 \u00b1 28]/(8). Now, speed can't be negative, so we take the positive root: (-8 + 28)/8 = 20/8 = 5/2 = 2.5 km/h? Wait, hold on, wait: 20 divided by 8 is 2.5, yes, that's 2.5 km/h. Wait, but let's check if that makes sense with the times.

Wait, but hold on, let's make sure we didn't make a mistake in setting up the equations, because if s is 2.5 km/h, then walking time for 9 km is 9 / 2.5 = 3.6 hours = 216 minutes, so total time including coffee shop is 216 + t minutes = 4 hours = 240 minutes, so t = 24 minutes. Let's check the second scenario: speed s + 2 = 4.5 km/h, walking time = 9 / 4.5 = 2 hours = 120 minutes, total time including coffee shop is 120 + t = 120 + 24 = 144 minutes = 2 hours 24 minutes, which matches the problem statement! Oh, that's perfect! So that checks out.

Wait, so why did I doubt? Let's go through the algebra again to make sure I didn't skip a step wrong.

We had Eq1: s(240 - t) = 540 (from 9/s = (240 - t)/60 \u21d2 9*60 = s(240 - t) \u21d2 540 = s(240 - t), yes, that's correct, 9*60 is 540, right).

Eq2: (s + 2)(144 - t) = 540, same way: 9/(s+2) = (144 - t)/60 \u21d2 9*60 = (s+2)(144 - t) \u21d2 540 = (s+2)(144 - t), correct.

Set equal: s(240 - t) = (s + 2)(144 - t). Expanded RHS: s(144 - t) + 2(144 - t) = 144s - st + 288 - 2t. LHS: 240s - st. Subtract 144s - st from both sides: 96s = 288 - 2t \u21d2 48s = 144 - t \u21d2 t = 144 - 48s, correct.

Then plug into Eq1: s(240 - (144 - 48s)) = s(96 + 48s) = 540 \u21d2 48s\u00b2 + 96s - 540 = 0 \u21d2 divide by 12: 4s\u00b2 + 8s - 45 = 0, correct.

Quadratic formula: s = [-8 \u00b1 \u221a(64 + 720)]/8 = [-8 \u00b1 \u221a784]/8 = [-8 \u00b1 28]/8. Positive solution: 20/8 = 2.5, yes, 2.5 km/h. Then t = 144 - 48s = 144 - 48*(2.5) = 144 - 120 = 24 minutes. Perfect, matches the check we did earlier.

Wait, so in the check, when s=2.5, walking time is 9/2.5=3.6 hours=216 minutes, total time=216 +24=240 minutes=4 hours, correct. Second scenario: s+2=4.5, walking time=9/4.5=2 hours=120 minutes, total time=120+24=144 minutes=2h24m, correct. So all checks out.

Now, the problem asks: Suppose Aya walks at s + 1/2 kilometers per hour. Find the number of minutes the walk takes her, including the t minutes spent in the coffee shop.

So first, s + 1/2 = 2.5 + 0.5 = 3 km/h. Wait, that's nice! 2.5 is 5/2, so 5/2 + 1/2 = 6/2 = 3 km/h. Good, so speed is 3 km/h.

Now, walking time at 3 km/h for 9 km: 9 / 3 = 3 hours. Then add t minutes for coffee shop, which we found t=24 minutes.

Wait, hold on! Wait, the problem says "the number of minutes the walk takes her, including the t minutes spent in the coffee shop." So total time in minutes is (walking time in hours * 60) + t.

Walking time at 3 km/h: 9 km / 3 km/h = 3 hours = 180 minutes. Then add t=24 minutes, so total time=180 +24=204 minutes.

Wait, is that it? Let me make sure we didn't misinterpret "the walk takes her...". Wait, in the problem statement, when she walks at s km/h, the walk takes 4 hours including t minutes at coffee shop. So in our interpretation, "the walk" (the activity) takes total time = walking time + coffee time. So yes, for the new speed, it's walking time (at s+1/2) + t minutes, which is what we did.

But let's just go through the problem statement again to be 100
"Every morning Aya goes for a 9-kilometer-long walk and stops at a coffee shop afterwards." So the walk is 9 km, then she stops at coffee shop (so coffee shop is after the walk? Wait, maybe that's the confusion! Maybe "the walk" is 9 km, then she goes to coffee shop, so the time to walk the 9 km is separate from the time at coffee shop, and the problem says "the walk takes her 4 hours, including t minutes spent in the coffee shop"\u2014wait, that would be odd, because if the walk is the 9 km, it shouldn't include coffee shop time. Wait, but in our check, it worked out perfectly, so maybe the problem statement is a bit poorly worded, but the math checks out with the interpretation that total time from start of walk (when she begins walking) until she finishes at coffee shop is 4 hours (which includes the time she's walking the 9 km plus time at coffee shop).

Wait, another way: suppose "the walk" refers to the 9 km journey, so time to walk 9 km is some time, then she spends t minutes at coffee shop, so total time from when she starts walking until she leaves coffee shop is (walking time) + t minutes. The problem says "the walk takes her 4 hours, including t minutes spent in the coffee shop"\u2014maybe "takes her" is the duration of the entire activity (walking + coffee), so yes, that's how we took it, and the check worked, so probably that's the right interpretation.

Let's see, if we had misinterpreted and thought that the coffee shop time is part of the walk time, but no, the problem says "including t minutes spent in the coffee shop", so the t minutes are added to the walk time to get the total time.

Wait, in our check, with s=2.5 km/h, walking time (just walking) is 3.6 hours=216 minutes, total time with coffee is 240 minutes, so t=24 minutes. Second scenario: walking time 2 hours=120 minutes, total time 144 minutes, so t=24 minutes, same t, which the problem states, so that's consistent.

So when speed is 3 km/h, walking time is 9/3=3 hours=180 minutes, total time=180 +24=204 minutes.

Wait, but let's try to do it without plugging in numbers early, to verify.

We had s=2.5, t=24, so s + 1/2=3, walking time=9/3=3 hours=180 min, total=180+24=204.

Alternatively, let's see if we can find the total time without finding s and t individually, maybe as a check.

Let\u2019s denote for a given speed v (km/h), walking time is 9/v hours, total time T(v) = 9/v + t/60 hours (since t minutes is t/60 hours).

In first case, v=s, T(s)=4 hours \u21d2 9/s + t/60 = 4 \u21d2 9/s = 4 - t/60 \u21d2 which is what we had (since 4 - t/60 = (240 - t)/60 hours).

Second case, v=s+2, T(s+2)=2 + 24/60=2.4 hours \u21d2 9/(s+2) + t/60=2.4 \u21d2 9/(s+2)=2.4 - t/60=(144 - t)/60, same as before.

So we can write the two equations as:

9/s = 4 - t/60 ...(1a)

9/(s+2) = 2.4 - t/60 ...(2a)

Let\u2019s let u = t/60, so u is time in hours at coffee shop. Then equations become:

9/s = 4 - u ...(1b)

9/(s+2) = 2.4 - u ...(2b)

Now, subtract equation (2b) from equation (1b):

9/s - 9/(s+2) = (4 - u) - (2.4 - u) = 1.6

Simplify left side: 9[ (s+2 - s)/(s(s+2)) ] = 9[2/(s(s+2))] = 18/(s(s+2))

So 18/(s(s+2)) = 1.6 = 8/5 (converting 1.6 to fraction: 8/5)

Therefore, s(s+2) = 18 * (5/8) = 90/8 = 45/4 = 11.25

Wait, s(s+2)=45/4 \u21d2 s\u00b2 + 2s - 45/4=0 \u21d2 multiply by 4: 4s\u00b2 +8s -45=0, which is exactly the quadratic we had earlier! Good, so that's a good consistency check.

Then from (1b): u=4 - 9/s. We know s=2.5=5/2, so 9/s=9/(5/2)=18/5=3.6, so u=4 - 3.6=0.4 hours=24 minutes, correct.

Now, for speed v=s + 1/2=5/2 +1/2=3 km/h, total time T(v)=9/v + u=9/3 + 0.4=3 + 0.4=3.4 hours. Convert to minutes: 3.4*60=204 minutes. Perfect, same answer.

So that's reassuring. The key was setting up the equations correctly, which we verified with two methods and a check, so the total time is 204 minutes.

Wait, just to make sure there isn't a trick in the problem statement: "the walk takes her 4 hours, including t minutes spent in the coffee shop". Does "the walk" refer to the coffee shop trip? No, the coffee shop is where she stops after the walk, so the walk is the 9 km, then coffee shop is separate. But the problem says the time for the walk (activity) includes coffee shop time, which is a bit confusing, but since the math checks out with the problem's given times (4 hours total with t minutes, 2h24m total with t minutes), and we found consistent t, it's probably the right interpretation.

Another possible misinterpretation: maybe t minutes is the time she spends walking? But no, the problem says "t minutes spent in the coffee shop", so t is coffee time.

Wait, let's suppose for a second that we misread and thought the 9 km is the time or something, but no, it says "9-kilometer-long walk", so distance is 9 km.

Wait, let's see if the problem says "the walk takes her 4 hours"\u2014if "the walk" is the 9 km, then walking time is 4 hours, but then she spends t minutes at coffee shop, so total time is 4h + t min, but the problem says "the walk takes her 4 hours, including t minutes...", so "including" implies that the t minutes are part of the 4 hours, so the walk's duration is 4 hours, which comprises walking and coffee time. So yes, our original interpretation is correct.

I think the key was not overcomplicating it and setting up the equations with total time = walking time + coffee time, which gave us two equations that simplified nicely, and the check confirmed the values of s and t, so we didn't have to worry about extraneous solutions or wrong interpretations.

So to recap:

- Found s=2.5 km/h, t=24 minutes via equations, verified with time calculations.

- New speed: 2.5 + 0.5=3 km/h.

- Walking time for 9 km at 3 km/h: 3 hours=180 minutes.

- Total time including coffee: 180 +24=204 minutes.

All steps check out, multiple methods confirm the answer. I think that's solid.

I think sometimes when problems say "the walk takes X time including Y time", it's easy to get confused about what's included, but by testing with the numbers we got (which matched the problem's time descriptions), we could be confident in the setup.

Just to write the final answer clearly: 204 minutes.

**Final Answer**\n\boxed{204}\n</think>

We are given that Aya walks a **9-kilometer-long walk** and stops at a coffee shop after. The problem states:

- When she walks at a constant speed of $ s $ km/h, the **total time** (including $ t $ minutes spent in the coffee shop) is **4 hours**.\n- When she walks at $ s + 2 $ km/h, the **total time** (including $ t $ minutes at the coffee shop) is **2 hours and 24 minutes**.

We are to find how many **minutes** the walk takes her when she walks at $ s + \frac{1}{2} $ km/h, including the $ t $ minutes spent at the coffee shop.

---

### Step 1: Convert all times into minutes for consistency

Let\u2019s define:\n- Total time in minutes for first scenario: $ 4 \times 60 = 240 $ minutes\n- Total time in minutes for second scenario: $ 2 \times 60 + 24 = 144 $ minutes

Let:\n- $ t $ be the number of minutes spent at the coffee shop (same in both scenarios)\n- Then the **walking time** in each case is:\n  - First scenario: $ 240 - t $ minutes\n  - Second scenario: $ 144 - t $ minutes

Since she walks 9 km in each case, we can write equations based on speed and time:

$$\n\text{Speed} = \frac{\text{Distance}}{\text{Time}} \Rightarrow \text{Time} = \frac{\text{Distance}}{\text{Speed}}\n$$

So for the first scenario:

$$\n\frac{9}{s} = \frac{240 - t}{60}\n\quad \Rightarrow \quad 540 = s(240 - t) \tag{1}\n$$

For the second scenario:

$$\n\frac{9}{s + 2} = \frac{144 - t}{60}\n\quad \Rightarrow \quad 540 = (s + 2)(144 - t) \tag{2}\n$$

---

### Step 2: Solve the system of equations

From (1):  \n$$\ns(240 - t) = 540\n$$

From (2):  \n$$\n(s + 2)(144 - t) = 540\n$$

Set them equal:

$$\ns(240 - t) = (s + 2)(144 - t)\n$$

Expand both sides:

Left: $ 240s - st $

Right: $ s(144 - t) + 2(144 - t) = 144s - st + 288 - 2t $

Subtract $ 144s - st $ from both sides:

$$\n96s = 288 - 2t\n\quad \Rightarrow \quad 48s = 144 - t\n\quad \Rightarrow \quad t = 144 - 48s \tag{3}\n$$

Now plug (3) into equation (1):

$$\ns(240 - (144 - 48s)) = 540\n\quad \Rightarrow \quad s(96 + 48s) = 540\n\quad \Rightarrow \quad 48s^2 + 96s - 540 = 0\n$$

Divide by 12:

$$\n4s^2 + 8s - 45 = 0\n$$

Use quadratic formula:

$$\ns = \frac{-8 \pm \sqrt{64 + 720}}{8} = \frac{-8 \pm \sqrt{784}}{8} = \frac{-8 \pm 28}{8}\n$$

Take positive root:

$$\ns = \frac{20}{8} = 2.5 \text{ km/h}\n$$

Now compute $ t $ using (3):

$$\nt = 144 - 48 \cdot 2.5 = 144 - 120 = 24 \text{ minutes}\n$$

---

### Step 3: Compute total time at new speed

New speed: $ s + \frac{1}{2} = 2.5 + 0.5 = 3 $ km/h

Walking time for 9 km:

$$\n\frac{9}{3} = 3 \text{ hours} = 180 \text{ minutes}\n$$

Add coffee shop time $ t = 24 $ minutes:

$$\n\text{Total time} = 180 + 24 = \boxed{204} text{ minutes}\n$$

---

### Final Answer

$$\n\boxed{204}\n$$

Okay, the user is asking for a random picture of an axolotl. Let me check the available tools.

First, there's the Axolotl tool with two main functions: getRandomAxolotlImage and searchAxolotlImages. Since the user wants a random picture, the getRandomAxolotlImage tool seems perfect here. The search tool requires parameters like color, gender, size, which the user didn't specify, so that's not needed.

The getRandomAxolotlImage has no parameters, so the Action Input should be an empty JSON object. I need to make sure the response is in the correct JSON format as per the documentation. The output should be an object with url, source, and description.

I should call getRandomAxolotlImage with no parameters. Let me structure the Action correctly. The user didn't mention any specific criteria, so no need for the search tool. Just use the random one.
</think>

Thought: The user is asking for a random picture of an axolotl. The appropriate tool for this is getRandomAxolotlImage, which requires no parameters.

Action: getRandomAxolotlImage
Action Input: {}

References↩︎