Learning While Staying Curious: Entropy-Preserving Supervised Fine-Tuning via Adaptive Self-Distillation for Large Reasoning Models
February 02, 2026
Abstract
The standard post-training recipe for large reasoning models, supervised fine-tuning followed by reinforcement learning (SFT-then-RL), may limit the benefits of the RL stage: while SFT imitates expert demonstrations, it often causes overconfidence and reduces generation diversity, leaving RL with a narrowed solution space to explore. Adding entropy regularization during SFT is not a cure-all; it tends to flatten token distributions toward uniformity, increasing entropy without improving meaningful exploration capability. In this paper, we propose CurioSFT, an entropy-preserving SFT method designed to enhance exploration capabilities through intrinsic curiosity. It consists of (a) Self-Exploratory Distillation, which distills the model toward a self-generated, temperature-scaled teacher to encourage exploration within its capability; and (b) Entropy-Guided Temperature Selection, which adaptively adjusts distillation strength to mitigate knowledge forgetting by amplifying exploration at reasoning tokens while stabilizing factual tokens. Extensive experiments on mathematical reasoning tasks demonstrate that, in SFT stage, CurioSFT outperforms the vanilla SFT by 2.5 points on in-distribution tasks and 2.9 points on out-of-distribution tasks. We also verify that exploration capabilities preserved during SFT successfully translate into concrete gains in RL stage, yielding an average improvement of 5.0 points. Code is available at https://anonymous.4open.science/r/CurioSFT.
1 Introduction↩︎
Recent breakthroughs [1], [2] establish "SFT-then-RL" as the de-facto paradigm for enhancing large reasoning models on automatically verifiable tasks, such as mathematical reasoning [3], [4], code generation [5], and agentic search [6], [7]. In this paradigm, the Supervised Fine-tuning (SFT) stage aligns the model with domain-specific patterns and required knowledge, thereby providing a superior initialization for the subsequent Reinforcement Learning (RL) stage.
However, this paradigm faces a critical challenge: the Cross Entropy loss in SFT rigidly maximizes the likelihood of expert tokens, which inevitably drives the model toward overconfidence [8], [9] and constricts the exploration space as training progresses. As shown in Figure 1, we use token entropy to quantify exploration capability and observe a rapid collapse over the SFT stage. Counter-intuitively, the SFT stage locks the model into a low-diversity mode, severely constraining the search space for the subsequent RL stage. This limitation often leads to marginal gains or even degradation compared to direct RL, aligning with recent findings [10], [11].
A straightforward approach is to regularize the SFT stage with entropy loss [12] on each token. However, trivially maximizing entropy will indiscriminately smooth the token probability and introduce ungrounded entropy, damaging exploration capability and leading to unsatisfactory or degraded performance. Concretely, it fails to distinguish token roles: forcing entropy on factual tokens disrupts knowledge retention, while neglecting critical reasoning tokens (e.g., “wait”) where exploration is truly beneficial. This discrepancy highlights the need for a method that substantially enhances exploration capabilities without compromising the model’s intrinsic knowledge.
To achieve this, we introduce CurioSFT, a novel entropy-preserving SFT method designed to enhance exploration with knowledge retention. This method consists of two key components: Self-Exploratory Distillation and Entropy-Guided Temperature Selection. Building on self-distillation [13], [14], Self-Exploratory Distillation exploits the monotonic relationship between token entropy and sampling temperature to construct a higher-entropy “teacher distribution” via an increased temperature. Aligning with this high-entropy teacher allows the model to selectively expand its search space under the guidance of its own curiosity. Crucially, to account for the distinct roles of tokens during reasoning, Entropy-Guided Temperature Selection dynamically modulates the temperature based on token-level uncertainty. This mechanism selectively encourages exploration at critical reasoning tokens while maintaining deterministic targets for factual tokens, thereby effectively mitigating the risk of knowledge forgetting.
Extensive experiments on mathematical reasoning benchmarks demonstrate that, CurioSFT not only effectively preserves entropy but also achieves superior performance across both in-distribution and out-of-distribution (OOD) tasks, outperforming vanilla SFT by an average of 2.5 points and 2.9 points, respectively. We empirically verify that the exploration capabilities preserved during SFT successfully translate into concrete gains in the RL stage. To this end, our contributions are three-fold:
We empirically analyze the drawbacks of entropy loss in SFT, including exploration degradation and knowledge forgetting. To address these, we propose CurioSFT, which preserves entropy while improving overall performance during the SFT stage.
We propose Self-Exploratory Distillation to preserve entropy while improving effective exploration by aligning with a self-generated, temperature-scaled teacher. We further introduce Entropy-Guided Temperature Selection to adapt token-level temperatures, selectively encouraging exploration and mitigating knowledge forgetting.
Extensive experiments on mathematical reasoning benchmarks demonstrate that, CurioSFT not only improves performance in SFT but also enhances the exploration capability, significantly improving the performance of RL stage. We also verify the robustness of CurioSFT across models and hyperparameters.
Figure 2: Expert token entropy and evaluation accuracy during the SFT stage. Compared to vanilla entropy loss, which uniformly encourages entropy across tokens, CurioSFT selectively increases entropy on high-entropy tokens while preserving low-entropy ones. We further observe that the increased token entropy induced by entropy loss does not translate into actual improvements in Pass@32 performance in our experiments.. a — Expert Token Entropy (Top 20%), b — Expert Token Entropy (Bottom 80%), c — Pass@32 for AIME 2024
2 Preliminaries↩︎
2.0.0.1 SFT Loss.
Let \(\mathcal{D}\) denote the SFT dataset, which contains multiple questions \({\mathbf{x}}\) and corresponding expert responses \(\mathbf{y}\). The optimization objective during the SFT stage is to minimize the cross-entropy loss between the model distribution and a one-hot target distribution induced by expert tokens, as: \[\begin{align} \mathcal{L}_{\text{SFT}}(\theta)=- \log \pi_\theta\big(y_t \mid \mathbf{s}_t\big), \end{align}\] where \(\pi_\theta\) is the fine-tuned model and \(\mathbf{s}_t\) is the concatenation of the question \(\mathbf{x}\) and the previously generated tokens \(\mathbf{y}_{<t}\).
2.0.0.2 Entropy Loss.
To encourage output diversity and prevent over-confidence, prior works [3], [15] introduce an entropy loss term as a regularizer during the RL stage, as: \[\begin{align} \mathcal{L}_{\text{entropy}}(\theta) &= \alpha \cdot -H(\pi_\theta(\cdot \mid \mathbf{s}_t)) \\ &= \alpha \cdot \sum_{y \in \mathcal{V}} \pi_\theta(y \mid \mathbf{s}_t) \log \pi_\theta(y \mid \mathbf{s}_t), \end{align}\] where \(\mathcal{V}\) denotes the vocabulary of the fine-tuned model, \(\alpha\) is the loss weight. However, applying entropy regularization solely at the RL stage often yields marginal benefits, as the preceding SFT stage has already driven the model into a low-entropy mode. Consequently, it is important to preserve entropy and encourage exploration during the SFT stage itself. Yet, deploying entropy loss in the SFT stage presents a fundamental challenge: unlike the online nature of RL, SFT is an offline process where the model cannot judge whether the increased entropy leads to valid reasoning paths or merely introduces noise. In the following section, we discuss two key limitations arising from this “blind” regularization through empirical observations.
3 The Pitfall of Entropy Loss in SFT↩︎
| Method | GPQA | MMLU-Pro | ARC-C | Avg.\(\uparrow\) |
|---|---|---|---|---|
| Vanilla SFT | 27.7 | 47.5 | 78.8 | 51.3 |
| SFT w/ Entropy | 28.0 | 45.9 | 77.2 | 50.4 |
| SFT w/ Entropy (Top 20%) | 29.6 | 47.9 | 79.3 | 52.3 |
6pt
3.0.0.1 Ungrounded entropy degrades exploration capability.
In the SFT stage, many tokens in the dataset are relatively unfamiliar to the current model, reflected by their lower output probability compared to online sampling tokens (offline 71% vs.online 76%). As a result, the entropy loss becomes highly sensitive to its weight \(\alpha\): as shown in Figure 2 (a) and Figure 2 (b), when \(\alpha\) is too small, entropy barely increases; when \(\alpha\) is too large (e.g., \(\alpha \ge 0.08\) in our setting), the objective can push some token distributions toward near-uniformity, causing an “entropy explosion” that destabilizes training. Even with a seemingly reasonable choice (e.g., \(\alpha=0.06\)), entropy loss does not reliably improve performance (Figure 2 (c)). The key reason is that the entropy loss indiscriminately pushes the token distribution toward higher entropy, without distinguishing between expanding a valid reasoning path and merely injecting noise. Consequently, increased token entropy does not translate into better reasoning performance and may even harm effective exploration.
3.0.0.2 Token-agnostic regularization amplifies knowledge forgetting.
Recent works suggest that exploration in LLMs is driven by a relatively small subset of high-entropy tokens, while most tokens remain low-entropy to preserve knowledge [16]. As shown in the Figure 2 (a) and Figure 2 (b), when we partition tokens by entropy (e.g., top 20% vs.the remaining 80%), naive entropy loss increases entropy in both groups. This is because maximizing entropy is equivalent to minimizing the KL divergence to a uniform distribution for all tokens (detailed proof in Appendix 8). Such token-agnostic regularization is detrimental to the model’s original knowledge and reasoning behavior. As shown in Table 1, restricting entropy loss to the top 20% high-entropy tokens yields significantly better performance on knowledge-intensive OOD tasks. Empirically, low-entropy tokens often correspond to deterministic factual content (e.g., nouns and numbers), where stability is crucial; forcing entropy at these positions weakens factual consistency and can induce knowledge forgetting. In contrast, high-entropy tokens tend to act as reasoning connectors (e.g., “wait”, “alternatively”), which are the natural targets for exploration.
4 Proposed Solution: CurioSFT↩︎
To address the limitations of vanilla entropy loss, we introduce CurioSFT, an entropy-preserving SFT method that enables models to learn expert behaviors while maintaining exploration capability. As shown in Figure 3, our method consists of two key components: Self-Exploratory Distillation (Section 4.1) and Entropy-Guided Temperature Selection (Section 4.2).
4.1 Self-Exploratory Distillation↩︎
Frontier LLMs internalize extensive world knowledge during pre-training, and exhibit implicit exploration capabilities that enable the generation of diverse trajectories [17], [18]. Motivated by self-distillation [19], we exploit this intrinsic capacity by minimizing the divergence between the policy of current model and a self-generated, higher-entropy teacher distribution. Formally, an LLM policy is obtained by applying the softmax function to the output logits \(z_\theta(\cdot\mid\mathbf{s}_t)\), as: \[\textcolor{SC}{\pi_\theta(y \mid \mathbf{s}_t; \textcolor{SC}{\tau})}=\frac{\exp\big(z_\theta(y \mid \mathbf{s}_t) / \textcolor{SC}{\tau}\big)}{\sum_{y' \in \mathcal{V}} \exp\big(z_\theta(y' \mid \mathbf{s}_t) / \textcolor{SC}{\tau}\big)},\] where \(\textcolor{SC}{\tau}\) denotes the sampling temperature and is typically fixed at \(1.0\) in standard SFT training. Leveraging the property that the entropy is monotonically increasing with respect to \(\tau\) (see proof in Appendix 9), we can construct a higher-entropy teacher distribution \(\pi^{\text{tch}}\) by simply rescaling the logits with a larger temperature \(\textcolor{TC}{\hat{\tau}} > \textcolor{SC}{\tau}\), which satisfies \(H\big(\textcolor{TC}{{\pi^{\text{tch}}}(\cdot \mid \mathbf{s}_t; \hat{\tau})}\big) > H\big(\textcolor{SC}{\pi_\theta(\cdot \mid \mathbf{s}_t; \tau)}\big)\). Subsequently, we can introduce a regularization loss that aligns the model with this higher-entropy teacher to achieve entropy preservation.
We theoretically prove that the constructed teacher distribution is the unique higher-entropy distribution that minimizes the KL divergence to the current policy under the entropy-increase constraint (see proof in Appendix 10). Adopting this teacher offers two key advantages: (a) Curiosity-driven exploration. The temperature-scaled teacher strictly preserves the relative order of token probabilities. Distilling the student toward this teacher therefore encourages exploration only over tokens that lie within the model’s valid exploration space, rather than injecting uninformative entropy to all tokens. (b) Reduced Knowledge forgetting. Prior works suggest that training data with lower divergence from the current model is associated with less knowledge forgetting [20]. By distilling toward a higher-entropy teacher that remains close to the current policy, the model can enhance its exploration ability while mitigating knowledge forgetting.
To ensure the stability of the teacher model, we deploy a separate teacher model parameterized by \(\phi\), and the teacher distribution is denoted as: \[\textcolor{TC}{\pi_\phi^{\text{tch}}(y \mid \mathbf{s}_t;\hat{\tau})} = \frac{\exp(z^{\text{tch}}_\phi(y \mid \mathbf{s}_t) / \textcolor{TC}{\hat{\tau}})}{\sum_{y' \in \mathcal{V}} \exp(z_\phi^{\text{tch}}(y' \mid \mathbf{s}_t) / \textcolor{TC}{\hat{\tau}})},\] where \(\textcolor{TC}{\hat{\tau}}\) is the teacher sampling temperature. Using a separate teacher updated more slowly than the student stabilizes the distillation target, preventing rapid fluctuations in the teacher distribution during training. The parameters of the teacher model \(\phi\) are synchronized with the current policy \(\theta\) every \(n\) steps using an exponential moving average with a decay factor of \(\mu\) (Algo. 4 Line 17). Finally, we formulate the self-exploratory distillation objective using the K2-loss [21], defined as: \[\label{eq:loss95sd} \mathcal{L}_{\text{SED}}(\theta) = \frac{1}{2} \sum_{t=1}^{T} \left( \log \frac{\textcolor{SC}{\pi_\theta(y \mid \mathbf{s}_t;\tau)}}{\textcolor{TC}{\pi_\phi^{\text{tch}}(y \mid \mathbf{s}_t;\hat{\tau})}} \right)^2 .\tag{1}\] Finally, the overall optimization objective is defined as a combination of the SFT loss and the self-distillation loss, with a coefficient \(\alpha\): \[\label{eq:loss95all} \mathcal{L}_{\text{total}}(\theta) = \mathcal{L}_{\text{SFT}}(\theta) + \alpha \cdot \mathcal{L}_{\text{SED}}(\theta).\tag{2}\]
4.2 Entropy-Guided Temperature Selection↩︎
The sampling temperature \(\textcolor{TC}{\hat{\tau}}\) for the teacher distribution is a crucial parameter in CurioSFT. A higher value encourages the model to align with a higher-entropy teacher distribution, whereas a lower value keeps the update closer to standard SFT. As discussed in Section 3, high-entropy tokens typically act as branching points that benefit from exploration, while low-entropy tokens encode deterministic facts, where stability is preferred. To respect such heterogeneity, we adaptively assign temperatures based on the token uncertainty. We first compute a token-level entropy increment \(\Delta_t\), then determine the temperature required via binary search. Finally, we use these specialized temperatures to construct the teacher distribution in Eq. (1 ).
Specifically, given the current token entropy \(H_t = H(\pi_\phi^{\text{tch}}(\cdot \mid \mathbf{s}_t))\), we compute the entropy increment \(\Delta_t\) via: \[\label{eq:entropy95increment} \Delta_t = \Delta_{\max} \cdot \text{Sigmoid}\big( \gamma \cdot (H_t - H_{\text{pivot}}) \big),\tag{3}\] where \(\Delta_{\max}\) is the maximum allowable entropy increase, \(\gamma\) is a scaling factor, and \(H_{\text{pivot}}\) is the entropy pivot that decides the activation threshold of exploration: increasing \(H_{\text{pivot}}\) makes exploration more selective, while decreasing it expands the set of tokens receiving substantial entropy increase. We adopt soft-gating rather than a hard mask to avoid brittle thresholding and introduce a smooth, adaptive margin: for high-entropy tokens (\(H_t \gg H_{\text{pivot}}\)), the sigmoid term approaches \(1\), pushing the target entropy toward \(H_t + \Delta_{\max}\) and thus strongly encouraging diversity at those positions. Conversely, for low-entropy tokens (\(H_t \ll H_{\text{pivot}}\)), the sigmoid term approaches \(0\), keeping the target entropy close to \(H_t\) and thereby minimizing interference with the model’s established knowledge.
Given the entropy increment \(\Delta_t\), our goal is to find a temperature for teacher distribution that matches the desired entropy target, as: \[\min_{\textcolor{TC}{\hat{\tau}_t}} \left| H\big(\pi_{\phi}^{\text{tch}}(\cdot \mid \mathbf{s}_t; \textcolor{TC}{\hat{\tau}_t})\big) - \big( H_t + \Delta_t \big) \right| < \epsilon,\] where \(\epsilon\) is a small constant. Given that entropy is monotonically increasing with respect to temperature \(\tau\) (see proof in Appendix 9), we can efficiently solve for \(\textcolor{TC}{\hat{\tau}_t} \in [\hat{\tau}_{\min}, \hat{\tau}_{\max}]\) using a binary search. To minimize computational overhead during training, we implement the temperature search as a fully vectorized operation and approximate the entropy using only the top-\(k\) logits, which focuses computation on the most influential tokens while significantly accelerating the calculation. Details of binary search are provided in Appendix 11.
5 Experiments↩︎
3.5pt
lccccc>cc|ccc>cc|cc & & &
(lr)2-7(lr)8-11(lr)12-13 & AIME25/24 & AMC23 & MATH. & Miner. & Olymp. & Avg.\(\uparrow\) & GPQA & MMLU. & ARC-C & Avg.\(\uparrow\) & Entropy\(\uparrow\) & Speed\(\downarrow\)
Qwen2.5-Math-7B & 4.6/8.3 & 35.5 & 50.1 & 12.1 & 16.5 & 21.2 & 26.2 & 32.4 & 63.2 & 40.6 & 0.15 & –
Vanilla SFT & 22.9/26.7 & 59.6 & 85.8 & 45.5 & 50.4 & 48.5 & 27.7& 47.5 & 78.8 & 51.3 & 0.31& 43.2
SFT with Entropy & 23.3/25.4 & 60.3 & 86.1 & 44.2 & 49.2 & 48.1 & 28.0& 45.9 & 77.2 & 50.4 & 0.36& 44.6
SFT with KL & 21.6/24.6 & 58.2 & 83.9 & 45.2 & 46.3 & 46.6 & 27.4& 47.8 & 78.0 & 51.1 & 0.30 & 50.2
GEM [22] & 24.6/26.7 & 60.2 & 85.7 & 47.7 & 50.9 & 49.3 & 30.6 & 49.0 & 81.4 & 53.7 &
0.66& 44.5
DFT [23] & 23.3/25.0 & 59.3 & 86.6 & 46.4 & 49.9 & 48.4 & 31.1 & 48.8 & 79.0 & 53.0 & 0.29 & 43.7
PSFT [24]& 25.0/28.8 & 60.4 & 86.9 & 48.3 & 52.6 & 50.3 & 29.9& 47.8 & 76.7 & 51.5 & 0.32&
45.2
& 26.3/29.6 & 59.9 & 87.0 & 49.8 & 53.2 & 51.0 & 31.7 & 49.5 & 81.3 & 54.2 &
0.43 & 53.9
Impr. vs SFT & +3.4/+2.9 & +0.3 & +1.2 & +4.3 & +2.8 & +2.5
& +4.0& +2.0 & +2.5 & +2.9 & +0.12 & +10.7
w/o Adaptive Temp. & 24.6/26.7 & 59.0 & 86.4 & 48.8 & 53.1 & 49.8 & 29.5 &47.3 & 79.8& 52.2 & 0.45 & 51.9
w/o Separate Teacher & 25.0/28.8 & 58.2 & 85.4 & 49.3 & 52.8 & 49.9 & 30.8 & 48.2 &80.2 & 53.0 & 0.39 & 45.5
5.0.0.1 Training.
We use OpenR1-Math-46K [25] as the training dataset for both SFT and RL stages, which contains 46K mathematics problems and corresponding answers generated by DeepSeek-R1 [2]. We adopt Qwen2.5-Math-7B [26] as the base model, except in Section 5.3, where we study robustness across different models. For the SFT stage, we train for 3 epochs, and for the RL stage, we train for 500 steps using GRPO [27]. We set the entropy pivot \(H_{\text{pivot}} = 1.2\) nats, the scaling factor \(\gamma = 2.0\), the maximum entropy increment \(\Delta_{\max} = 0.5\) nats, and the loss weight \(\alpha = 1\). The temperature clip range is \(\hat{\tau}_{\min}=1.1,\hat{\tau}_{\max}=1.5\). Further training details and hyperparameters are provided in Appendix 12.
5.0.0.2 Evaluation.
To evaluate the model’s performance, we utilize six challenging and widely used mathematical reasoning benchmarks, including: AIME 2024, AIME 2025, AMC [28], Math-500 [29], Olympiad Bench [30], and Minerva [31]. Furthermore, to assess the extent of knowledge retention and generalization ability, we evaluate the model on three OOD benchmarks: ARC-Challenge [32], GPQA-Diamond [33], and MMLU-Pro [34]. We set the sampling temperature to \(0.6\) and \(\text{Top\_p}=0.95\), and keep other settings consistent with the training. Due to the large number of questions in MMLU-Pro, we generate a single response per question for MMLU-Pro, while using \(8\) responses per question for all other benchmarks, and compute the average accuracy as the final reported metric.
5.1 Effectiveness of CurioSFT↩︎
5.1.0.1 Baselines.
We compare CurioSFT against two categories of baselines: (a) Vanilla SFT with Regularization, which includes adding entropy loss and KL divergence constraints relative to the base model. (b) Advanced SFT Variants, which are designed to mitigate over-confidence and encourage diversity. Specifically, PSFT [24] employs trust-region constraints to limit policy shift; DFT [23] re-weights token updates based on model internal knowledge; and GEM [22] maintains diversity by encouraging the model to diverge from over-confident distributions.
4pt
lcccccc>cc|ccc>cc & &
(lr)2-8 (lr)9-12 & AIME25 & AIME24 & AMC23 & MATH. & Miner. & Olymp. & Avg. &
GPQA & MMLU. & ARC-C & Avg.
Vanilla GRPO & 18.8 & 20.8 & 62.8 & 84.7 & 46.7 & 50.0 & 47.3 & 40.3 & 50.1 & 84.1 & 58.2
LUFFY [25] & 29.4 & 23.1 & 65.6 & 87.6 & 49.5 & 57.2 & 52.1 & 39.9 & 53.0 & 80.5 & 57.8
Prefix-RFT [35] & 26.4 & 31.8 & 68.2 & 88.4 & 50.9 & 55.7 & 53.6 & 39.1 & 52.1 & 84.0 & 58.4
RL-PLUS [36] & 25.9 & 33.4 & 68.1 & 90.2 & 52.3 & 58.8 & 54.8 & 40.4 & 54.7 & 82.3 & 59.1
SFT + RL & 24.6 & 32.1 & 68.1 & 88.2 & 51.9 & 57.1 & 53.7 & 41.1 & 53.3 & 83.5 & 59.3
GEM [22] + RL & 27.5 & 34.6 & 71.1 & 90.8 & 52.1 & 61.3 & 56.2 & 40.3 & 54.1 & 83.7 & 59.4
DFT [23] + RL & 24.2 & 31.3 & 69.8 & 91.3 & 50.5 & 59.0 & 54.4 & 40.4 & 54.0 & 84.9 & 59.8
PSFT [24] + RL & 26.7 & 36.7 & 71.5 & 91.2 & 52.0 & 62.7 & 56.8 & 42.8 & 55.4 & 85.1 & 61.1
+ RL & 30.4 & 39.2 & 72.7 & 91.7 & 54.9 & 63.2 & 58.7 & 43.2 & 56.0
& 85.9 & 61.7
Impr. vs SFT+RL & +5.8 & +7.1 & +4.6 & +3.5 & +3.0 & +6.1 & +5.0&
+2.1 & +2.7 & +2.4 & +2.4
5.1.0.2 Results.
Table ¿tbl:tab:sft95results? shows that CurioSFT achieves the best overall performance on both in-distribution and OOD benchmarks. Compared to vanilla SFT, CurioSFT improves the ID average from 48.5% to 51.0% (+2.5 points) and the OOD average from 51.3% to 54.2% (+2.9 points), while preserving higher token entropy (0.31 \(\rightarrow\) 0.43, +0.12 nats). Enforcing a KL constraint slows the training while offering little benefit for entropy preservation. Among SFT variants, GEM achieves the highest entropy by keeping away from overconfident distributions. However, this occurs at the cost of ungrounded entropy, leading it to underperform CurioSFT on in-distribution tasks. PSFT and DFT improve training stability, but since they do not explicitly target exploration preservation, their overall improvements remain limited. As shown in Figure 5, we also report N-gram diversity (1 minus N-gram similarity) and confirm that entropy preservation consistently increases generation diversity. Overall, these results indicate that CurioSFT successfully preserves effective entropy while mitigating knowledge forgetting during the SFT stage.
5.1.0.3 Ablation Study.
We ablate key components of CurioSFT to quantify their contributions. Removing the Entropy-Guided Temperature Selection module (Section 4.2) and using a fixed temperature \(\hat{\tau}=1.3\) leads to a clear OOD drop (54.2% \(\rightarrow\) 52.2%, -2.0 points), highlighting the importance of token-level adaptivity for retaining knowledge. Next, we remove the separate teacher model and directly use the current model as the teacher, which causes a modest performance degradation, supporting the role of a stable teacher signal in providing reliable entropy-preserving guidance.
5.1.0.4 Complexity.
CurioSFT introduces additional computation due to an extra forward pass and token-wise temperature search, but the overhead remains within a practical and acceptable range (43.2 \(\rightarrow\) 53.9 seconds per step). Moreover, most of the cost can be reduced by removing the separate teacher, at the risk of a small performance drop.
5.2 Unlocking the Potential of the SFT-then-RL Paradigm↩︎
5.2.0.1 Baselines.
We next examine whether the preserved entropy during SFT leads to meaningful gains in the RL stage. We compare our solution against two families of baselines. First, we consider single-stage hybrid SFT+RL methods that fuse offline demonstrations with on-policy exploration into one single stage. Specifically, LUFFY [25] optimizes an RL objective on a mixture of online rollouts and offline demonstrations; Prefix-RFT [35] injects expert prefixes to steer exploration; and RL-PLUS [36] reuses expert examples during RL through multiple importance sampling. Second, we consider the standard two-stage SFT-then-RL paradigm, where we run GRPO from the SFT checkpoints in Section 5.1.
5.2.0.2 Results.
Table ¿tbl:tab:rl95results? summarizes the results after the RL stage. CurioSFT + GRPO pipeline achieves the best overall performance, improving the two-stage baseline SFT+RL from 53.7% to 58.7% on the in-distribution tasks (+5.0 points) and from 59.3% to 61.7% on the OOD tasks (+2.4 points). The gains are most pronounced on the challenging AIME benchmarks, where CurioSFT+GRPO reach 39.2% on AIME24 (vs.% for SFT+RL) indicating that CurioSFT provides a substantially better initialization for RL exploration. Moreover, CurioSFT + GRPO consistently outperforms single-stage hybrid methods, demonstrating that a well-designed SFT stage that preserves effective exploration can unlock a higher RL performance ceiling.
4pt
lccccc>cc Method & AIME24/25 & AMC & MATH. & Miner. & Olymp. & Avg.
Base model & 6.3 / 4.2 & 42.0 & 53.0 & 17.9 & 20.8 & 24.0
SFT & 20.4 / 19.6 & 53.0 & 83.5 & 47.4 & 47.7 & 45.3
SFT + GRPO & 27.5 / 22.9 & 62.9 & 89.0 & 50.2 & 57.9 & 51.7
& 21.3 / 25.0 & 54.4 & 84.2 & 48.3 & 48.2 & 46.9
+ GRPO & 28.8 / 27.9 & 65.0 & 89.7 & 53.6 & 59.1 & 54.0
Base model & 2.1 / 2.5 & 19.3 & 43.2 & 26.3 & 14.8 & 18.0
SFT & 8.3 / 11.7 & 38.3 & 68.1 & 32.7 & 35.6 & 32.5
SFT + GRPO & 9.6 / 12.1 & 40.3 & 74.1 & 35.5 & 38.9 & 35.1
& 10.0 / 10.4 & 38.9 & 69.0 & 33.0 & 36.6 & 33.0
+ GRPO & 14.2 / 11.7 & 42.9 & 74.3 & 38.6 & 40.9 & 37.1
5.3 Algorithm Robustness↩︎
We further evaluate CurioSFT on Qwen3-4B-Base [37] and Llama-3.1-8B-Instruct [38]. As shown in Table ¿tbl:tab:robustness?, CurioSFT consistently improves over vanilla SFT on both backbones, improving the averaged accuracy from 45.3% to 46.9% on Qwen3-4B (+1.6 points) and from 32.5% to 33.0% on Llama-3.1-8B-Instruct (+0.5 points). These results validate that CurioSFT generalizes across model families and sizes.
5.4 Hyperparameter Tuning↩︎
Unlike entropy loss, which typically requires careful tuning of the loss weight, CurioSFT adaptively computes a token-wise temperature and thus avoids introducing an additional sensitive coefficient. In practice, the key hyperparameter is the entropy pivot \(H_{\text{pivot}}\) in Eq. (3 ), which controls the overall strength of the entropy regularization. Increasing \(H_{\text{pivot}}\) weakens the overall encouragement (fewer tokens receive a large entropy increment), while decreasing it makes the entropy increment larger for more tokens, thereby preserving more entropy. We sweep \(H_{\text{pivot}}\) from 0.4 to 1.4. As shown in Figure 6, performance peaks around \(H_{\text{pivot}}=1.2\). Importantly, across the entire range, CurioSFT consistently maintains performance gains over the baseline, indicating that the proposed method is robust to the choice of \(H_{\text{pivot}}\).
6 Related Work↩︎
6.0.0.1 SFT in Post-Training.
SFT plays an irreplaceable role in enhancing model capabilities during post-training [39], [40]. Generally, SFT serves two primary functions: (a) serving as a large-scale knowledge injection [41], which significantly enhances zero-shot performance on OOD tasks [42]; and (b) serving as a cold-start initialization that enables the model to rapidly adapt to specific response patterns, thereby raising the performance ceiling for subsequent post-training stages (typically RL). For example, DeepSeek-R1 [2] utilizes a small set of high-quality reasoning data to activate the model’s inherent chain-of-thought capabilities before RL. ReTool [43] constructs a diverse, high-quality tool-use dataset to instruct the model on when to invoke specific tools. In this paper, we focus on the latter role, investigating how SFT can be optimized to provide a superior initialization point for the subsequent RL stage.
6.0.0.2 SFT-then-RL vs. Hybrid SFT with RL.
Several studies argue that the two-stage “SFT-then-RL” paradigm may underperform compared to applying RL directly to the base model [10], [11], [25], [44], [45]. Hence, a line of research has explored fusing SFT and RL into a single-stage hybrid paradigm. For example, LUFFY [25] updates the model using both expert demonstrations and self-exploration rollouts via a weighted RL loss. RL-PLUS [36] introduces an exploration-based advantage function to balance SFT and RL losses. HPT [44] integrates the two objectives through a unified theoretical perspective. In contrast to these works, we empirically demonstrate that when intrinsic exploration capabilities are preserved during the SFT phase, the SFT-then-RL paradigm achieves a higher performance ceiling than hybrid approaches.
6.0.0.3 Diversity Regularization in SFT.
The central challenge in SFT lies in its inherent susceptibility to overconfidence and diversity collapse. To address this, several works have explored regularization techniques. DFT [23] re-weights token updates based on generation probabilities; GEM [22] employs reverse KL divergence to prevent the distribution from converging to a collapsed mode; and PSFT [24] imposes trust-region constraints to limit policy drift. However, none of these approaches address the problem from the perspective of entropy collapse, which is the key for effective exploration in the “SFT-then-RL” paradigm.
7 Conclusion↩︎
In this paper, we address a critical bottleneck in the SFT-then-RL paradigm: standard SFT induces entropy collapse that severely constricts downstream exploration. We propose CurioSFT, an entropy-preserving SFT method utilizing adaptive self-distillation to maintain diverse yet valid exploration spaces. Experiments demonstrate that CurioSFT consistently outperforms vanilla SFT in both in-distribution and out-of-distribution tasks. More importantly, we verify that the preserved exploration effectively transfers to the RL stage, unlocking a significantly higher performance ceiling.
Limitations↩︎
While CurioSFT effectively preserves exploration during SFT, we acknowledge two limitations. First, our method incurs additional training overhead compared to vanilla SFT, as it requires an extra forward pass to compute the teacher distribution and token-wise temperature selection. Nevertheless, the added cost remains within a practical range in our experiments. Second, our approach is bounded by the base model’s intrinsic capabilities. Since we rely on self-distillation to preserve and amplify the model’s latent exploration, the benefit may be smaller when the base model lacks sufficient prior knowledge. Future work could incorporate external signals to further enhance the exploration capability beyond what self-distillation alone can provide.
8 Derivation of Entropy Loss↩︎
Let \(\mathcal{U}\) be the uniform distribution, i.e., \(\mathcal{U}(y)=\frac{1}{|\mathcal{V}|}\). The KL divergence from \(\pi_\theta(\cdot\mid\mathbf{s})\) to \(\mathcal{U}\) is: \[\begin{align} D_{\mathrm{KL}}\!&\Big(\pi_\theta(\cdot\mid\mathbf{s}) \,\Big\|\, \mathcal{U}\Big) \\ &= \sum_{y\in\mathcal{V}} \pi_\theta(y\mid\mathbf{s})\log\frac{\pi_\theta(y\mid\mathbf{s})}{\mathcal{U}(y)} \\ &= \sum_{y\in\mathcal{V}} \pi_\theta(y\mid\mathbf{s})\Big(\log \pi_\theta(y\mid\mathbf{s}) - \log \tfrac{1}{|\mathcal{V}|}\Big) \\ &= \sum_{y\in\mathcal{V}} \pi_\theta(y\mid\mathbf{s})\log \pi_\theta(y\mid\mathbf{s}) + \log|\mathcal{V}| \\ &= -H\!\big(\pi_\theta(\cdot\mid\mathbf{s})\big) + \log|\mathcal{V}|. \end{align}\] Since \(\log|\mathcal{V}|\) is constant w.r.t.\(\theta\), minimizing \(D_{\mathrm{KL}}\!\big(\pi_\theta(\cdot\mid\mathbf{s}) \,\|\, \mathcal{U}\big)\) is equivalent to maximizing \(H\!\big(\pi_\theta(\cdot\mid\mathbf{s})\big)\). Therefore, naive entropy regularization implicitly encourages the model distribution to move toward the uniform distribution over \(\mathcal{V}\).
9 Proof of Monotonicity↩︎
Theorem 1. Let \(\pi_\theta(\cdot \mid \mathbf{s};\tau)=\mathrm{softmax}\!\big(z_\theta(\cdot\mid\mathbf{s})/\tau\big)\) be the distribution derived from logits \(z_\theta(\cdot\mid\mathbf{s})\) with temperature \(\tau\). Then the entropy \(H\!\big(\pi_\theta(\cdot \mid \mathbf{s};\tau)\big)\) is non-decreasing with respect to \(\tau\).
Proof. For brevity, let \(\pi_y\) denote \(\pi_\theta(y \mid \mathbf{s};\tau)\), and let \(z_y\) denote the logit for token \(y\) (i.e., \(z_y := z_\theta(y\mid\mathbf{s})\)). Recall that: \[\log \pi_y = \frac{z_y}{\tau} - \log Z(\tau),\] where \(Z(\tau)\) is the partition function. The derivative of the entropy \(H(\pi) = -\mathbb{E}_{\pi}[\log \pi]\) with respect to \(\tau\) is derived as follows: \[\begin{align} \frac{\partial H}{\partial \tau} &= - \sum_{y} \frac{\partial \pi_y}{\partial \tau} \log \pi_y - \sum_{y} \pi_y \frac{\partial \log \pi_y}{\partial \tau} \nonumber \\ &= - \sum_{y} \frac{\partial \pi_y}{\partial \tau} \log \pi_y \quad (\text{since } \sum_y \pi_y = 1) \nonumber \\ &= - \sum_{y} \frac{\partial \pi_y}{\partial \tau} \left( \frac{z_y}{\tau} - \log Z \right) \nonumber \\ &= - \frac{1}{\tau} \sum_{y} z_y \frac{\partial \pi_y}{\partial \tau}. \label{eq:short95deriv} \end{align}\tag{4}\] Using the standard derivative of the softmax function \(\frac{\partial \pi_y}{\partial \tau} = \frac{\pi_y}{\tau^2}\Big(\mathbb{E}_{\pi}[z] - z_y\Big)\) and substituting this into Eq. 4 , we have: \[\begin{align} \frac{\partial H}{\partial \tau} &= - \frac{1}{\tau^3} \sum_{y} \pi_y z_y \Big(\mathbb{E}_{\pi}[z] - z_y\Big) \nonumber \\ &= \frac{1}{\tau^3} \left[ \sum_{y} \pi_y z_y^2 - \left(\sum_{y} \pi_y z_y\right)^2 \right] \nonumber \\ &= \frac{1}{\tau^3} \left( \mathbb{E}_{\pi}[z^2] - (\mathbb{E}_{\pi}[z])^2 \right) \nonumber \\ &= \frac{1}{\tau^3} \mathrm{Var}_{\pi}[z]. \end{align}\] Since \(\mathrm{Var}_{\pi}[z] \ge 0\) and \(\tau > 0\), the derivative is always non-negative. Notably, the variance \(\mathrm{Var}_{\pi}[z]\) vanishes if and only if the distribution \(\pi\) is uniform. Thus, \(H\!\big(\pi_\theta(\cdot \mid \mathbf{s};\tau)\big)\) is strictly non-decreasing in \(\tau\), except in the case of a uniform distribution. ◻
10 Temperature scaling as the KL-closest higher-entropy teacher↩︎
To reduce overconfidence while staying close to \(\pi\), we aim to construct a teacher distribution \(\pi^{\text{tch}}\) that satisfies two requirements: (i) it has higher entropy than the current policy, so it encourages exploration; and (ii) it remains KL-close to \(\pi\), so the supervision signal is stable and capability-aware. This naturally leads to the following constrained optimization problem: \[\begin{align} \label{eq:entropy95proj} \mathbf{P1:}\,\,\min_{\pi^{\text{tch}}}\quad & D_{\mathrm{KL}}\!\Big(\pi^{\text{tch}}(\cdot\mid\mathbf{s}) \,\Big\|\, \pi(\cdot\mid\mathbf{s})\Big) \\ \text{s.t.}\quad & H\!\big(\pi^{\text{tch}}(\cdot\mid\mathbf{s})\big) \;\ge\; H\!\big(\pi(\cdot\mid\mathbf{s})\big)+\Delta. \nonumber \end{align}\tag{5}\]
Theorem 2 (Temperature scaling is the optimum of P1). Given an entropy increment \(\Delta\), the unique optimum \(\pi_*^{\text{tch}}\) of 5 is a temperature-scaled distribution: there exists a unique \(\hat{\tau}>1\) such that \[\label{eq:q95star95temp} \begin{align} \pi_*^{\text{tch}}(y\mid\mathbf{s}) &=\pi(y\mid\mathbf{s};\hat{\tau}) \\ &=\frac{\exp\!\big(z(y\mid\mathbf{s})/\hat{\tau}\big)}{\sum_{y'\in\mathcal{V}} \exp\!\big(z(y'\mid\mathbf{s})/\hat{\tau}\big)}, \end{align}\qquad{(1)}\] and it satisfies \(H\!\big(\pi_*^{\text{tch}}(\cdot\mid\mathbf{s})\big)=H\!\big(\pi(\cdot\mid\mathbf{s})\big)+\Delta.\)
Proof. We solve P1 by forming its Lagrangian and applying the KKT conditions. For convenience, denote the target entropy as \(H^{\text{tar}}=H\!\big(\pi(\cdot\mid\mathbf{s})\big)+\Delta.\) Rewrite the inequality constraint in the standard KKT form: \[g(\pi^{\text{tch}}):=H^{\text{tar}}-H\!\big(\pi^{\text{tch}}(\cdot\mid\mathbf{s})\big)\le 0,\] with KKT multiplier \(\lambda_H\ge 0\). Since \(H(\pi^{\text{tch}})=-\sum_{y\in\mathcal{V}}\pi^{\text{tch}}(y\mid\mathbf{s})\log \pi^{\text{tch}}(y\mid\mathbf{s})\), we have \(g(\pi^{\text{tch}})=H^{\text{tar}}+\sum_{y\in\mathcal{V}}\pi^{\text{tch}}(y\mid\mathbf{s})\log \pi^{\text{tch}}(y\mid\mathbf{s})\). The Lagrangian of P1 is: \[\label{eq:lagrangian} \begin{align} \mathcal{L}&(\pi^{\text{tch}},\lambda,\lambda_H)=\\ &\sum_{y\in\mathcal{V}} \pi^{\text{tch}}(y\mid\mathbf{s}) \log\frac{\pi^{\text{tch}}(y\mid\mathbf{s})}{\pi(y\mid\mathbf{s})} \\ &+\lambda\Big(\sum_{y\in\mathcal{V}}\pi^{\text{tch}}(y\mid\mathbf{s})-1\Big) \\ &+\lambda_H\Big( H^{\text{tar}} +\sum_{y\in\mathcal{V}} \pi^{\text{tch}}(y\mid\mathbf{s}) \log \pi^{\text{tch}}(y\mid\mathbf{s}) \Big), \end{align}\tag{6}\] where \(\lambda\) is the multiplier for normalization and \(\lambda_H\ge 0\) is the KKT multiplier for \(g(\pi^{\text{tch}})\le 0\). Taking the derivative w.r.t.\(\pi^{\text{tch}}(y\mid\mathbf{s})\) and setting it to zero: \[\label{eq:kkt95step} \begin{align} &(1+\log \pi^{\text{tch}}(y\mid\mathbf{s})) -\log \pi(y\mid\mathbf{s}) +\lambda \\ &\qquad+\lambda_H\big(1+\log \pi^{\text{tch}}(y\mid\mathbf{s})\big)=0. \end{align}\tag{7}\] Simplifying the above equation gives: \[\label{eq:q95prop95p} \log \pi^{\text{tch}}(y\mid\mathbf{s}) = \frac{1}{1+\lambda_H}\log \pi(y\mid\mathbf{s}) + C,\tag{8}\] where \(C\) is a normalization constant independent of \(y\). Define \(\hat{\tau}:=1+\lambda_H\), then we obtain a power-form solution: \[\label{eq:power95form} \pi^{\text{tch}}(y\mid\mathbf{s}) \propto \pi(y\mid\mathbf{s})^{1/\hat{\tau}}.\tag{9}\] When \(\Delta>0\), the entropy constraint must be active at optimum; otherwise one could move \(\pi^{\text{tch}}\) closer to \(\pi\) and strictly decrease \(D_{\mathrm{KL}}(\pi^{\text{tch}}\|\pi)\) while remaining feasible. Thus, by complementary slackness, \(\lambda_H>0\) and hence \(\hat{\tau}>1\). Using \(\pi(y\mid\mathbf{s})\propto \exp(z(y\mid\mathbf{s}))\), we have: \[\label{eq:power95to95temp} \begin{align} \pi^{\text{tch}}(y\mid\mathbf{s}) &\propto \pi(y\mid\mathbf{s})^{1/\hat{\tau}} \;\propto\; \Big(\exp(z(y\mid\mathbf{s}))\Big)^{1/\hat{\tau}} \\ &\propto\; \exp\!\big(z(y\mid\mathbf{s})/\hat{\tau}\big), \end{align}\tag{10}\] which is exactly the temperature-scaled softmax form in ?? after normalization. Finally, by Appendix 9, under the full-support assumption the entropy \(H\!\big(\pi(\cdot\mid\mathbf{s};\tau)\big)\) is continuous and strictly increasing in \(\tau\), so there exists a unique \(\hat{\tau}>1\) such that \(H\!\big(\pi(\cdot\mid\mathbf{s};\hat{\tau})\big)=H^{\text{tar}}\). Since \(D_{\mathrm{KL}}(\cdot\|\pi)\) is convex in its first argument and the feasible set is convex because entropy is concave, P1 is a convex optimization problem. Therefore, any distribution that satisfies the KKT conditions is globally optimal. Consequently, the teacher distribution constructed by temperature scaling is the KL-closest distribution to \(\pi\) among all distributions whose entropy is increased by at least \(\Delta\). ◻
11 Efficient Implementation of Entropy-Guided Temperature Search↩︎
Leveraging the monotonic relationship between entropy and the sampling temperature, we can find a unique temperature \(\hat{\tau_t}\) whose entropy matches a desired target via binary search. To make this procedure efficient in large-scale LLM training, we implement it with two key design choices:
Vectorized binary search. Instead of searching for \(\hat{\tau_t}\) token by token, we perform a batched binary search over all tokens in a mini-batch using vectorized PyTorch operations.
Top-\(k\) entropy approximation. The softmax distribution over the vocabulary is typically heavy-tailed, so the entropy is dominated by the highest-probability tokens. We therefore approximate the full entropy using only the top-\(k\) logits: \[H\big(\pi_\theta(\cdot\mid\mathbf{s}_t)\big) \approx -\sum_{y \in \mathcal{V}_{:k}} \hat{\pi}_\theta(y\mid\mathbf{s}_t) \log \hat{\pi}_\theta(y\mid\mathbf{s}_t),\] where \(\mathcal{V}_{:k}\) denotes the top-\(k\) tokens and \(\hat{\pi}_\theta\) is the distribution renormalized over this subset.
In all experiments, we set \(k = 512\), which reduces the complexity of the entropy computation from \(O(|\mathcal{V}|)\) to \(O(k)\) and leads to only a modest training overhead.
| Parameter Name | Value |
|---|---|
| Epochs | 3 |
| Batch Size | 256 |
| Max Response Length | 8192 |
| Learning Rate | 1e-5 |
| Warm Up Style | cosine |
| Warm Up Steps | 60 |
| Parameter Name | Value |
|---|---|
| Training Steps | 500 |
| Batch Size | 128 |
| Mini Batch Size | 64 |
| Max Response Length | 8192 |
| Learning Rate | 1e-6 |
| Clip Higher | 0.28 |
| Clip Lower | 0.2 |
| KL coefficient | 0.0 |
| Rollout Numbers | 8 |
| Rollout Temperature | 1.0 |
| Rollout Top_p | 0.95 |
12 Experiment Setting↩︎
12.0.0.1 Training.
For both SFT and RL, we use Verl [46] as the training framework and vLLM [47] as the inference engine. All experiments are conducted on \(8\times\) NVIDIA H800 GPUs. We update the teacher model every \(n=5\) steps with a decay
\(\mu=0.99\). The full SFT hyperparameters are provided in Table 2. For the RL stage, we generate \(N=8\) candidate responses per question to
estimate the advantage. We use a binary reward: a response receives reward 1 if it matches the ground truth (verified by Math-Verify [48]) and follows the required format, and 0 otherwise. The full RL hyperparameters are listed in Table 3. We use the prompt template in Table 7 for both SFT and RL training.
12.0.0.2 Dataset Processing.
We use OpenR1-Math [25] for both SFT and RL training. We observe that a large portion of the questions contain irrelevant or distracting information. To reduce such noise, we rewrite the questions using DeepSeek-V3 [49] and keep the original ground-truth answers unchanged. Example rephrase prompts are shown in Table 8.
Table 7: Training Prompt <|im_start|>system
You are an exceptional mathematician. Your task is to solve mathematical questions through a systematic and thorough reasoning process. This involves careful analysis, exploration of possible approaches, verification of intermediate steps, critical
reassessment, and iterative refinement of your reasoning process.
Structure your response in two distinct sections: “Thought” and “Solution”.
In the “Thought” section, present your detailed reasoning process in the following format:
<think>
Your detailed reasoning, including brainstorming, logical deductions, verification, and refinement of ideas.
</think>
This section must conclude with “</think>”, and should reflect deep, reflective, and self-correcting thinking process.
In the “Solution” section, following the “</think>”, concisely draw the final, logical, and accurate answer from your reasoning.
Please output your final answer within \\boxed{}.<|im_end|>
<|im_start|>user
Here is the question:{question}<|im_end|>
<|im_start|>assistant
<think>
Table 8: Question Rephrase Prompt I will provide a post from a math-related forum that contains a math problem. Your task is to extract only the math problem statement and remove any irrelevant
or noisy content (e.g., commentary, solutions, chat, metadata). Keep the original wording and question type intact, and present the extracted problem clearly and concisely.
Remove any redundant context, personal commentary, anecdotes, or unrelated information. But make sure not to change the meaning of the problem and keep all necessary mathematical or technical details.
Here are a few examples.
Example 1:
Input:
What is the remainder of \(8^6+7^7+6^8\) is divided by \(5\)?
no calculator, of course, paper isn’t needed either, but sure.
Output:
What is the remainder of \(8^6+7^7+6^8\) when divided by 5?
Example 2:
Input:
(20 points) Let \(x, y\) be non-zero real numbers, and satisfy \(\frac{x \sin \frac{\pi}{5} + y \cos \frac{\pi}{5}}{x \cos \frac{\pi}{5} - y \sin \frac{\pi}{5}} = \tan \frac{9 \pi}{20}\).(1)
Find the value of \(\frac{y}{x}\);
Output:
Let \(x, y\) be non-zero real numbers, and satisfy \(\frac{x \sin \frac{\pi}{5} + y \cos \frac{\pi}{5}}{x \cos \frac{\pi}{5} - y \sin \frac{\pi}{5}} = \tan \frac{9 \pi}{20}\). Find the value
of \(\frac{y}{x}\).
Now, here is the text you need to extract the problem.
Input:
{question}
Output: