1 Introduction↩︎

Supervised fine-tuning (SFT) is one of the standard ways to adapt large language models to downstream tasks [1]–[4]. As instruction datasets grow in size and heterogeneity, training on every available example is often inefficient and sometimes harmful: low-quality examples can amplify bias, redundant examples waste compute, and overly narrow mixtures can overfit the model to a transient capability profile. These concerns have motivated data selection methods based on quality filtering, influence estimation, active learning, coreset coverage, online batch scoring, and deduplication or diversification [5]–[14].

Most of this literature evaluates selection within a single fine-tuning stage. This is natural when the goal is to solve one task with a fixed budget, but it misses an important property of modern model adaptation: fine-tuning is often sequential. A deployed assistant may first be tuned on general instructions, then mathematical reasoning, then code, then safety data, and then domain-specific corpora. In this setting, the selected subset at stage \(t\) does not merely save compute. It changes the parameter state from which all future stages start.

This paper asks whether efficient data selection can make models increasingly specialized. By specialization we do not mean only that a selector changes the label distribution of the current batch. We mean that selection may push representations, gradients, and parameter-efficient adapters toward a narrow set of capabilities, reducing future learnability or robustness. A sample can be highly useful for the current stage while still being a poor long-term training intervention.

We call this phenomenon myopic selection: a selection policy is myopic when it maximizes immediate utility at the expense of future adaptation, retention, or out-of-distribution (OOD) robustness. This framing leads to five research questions. RQ1: Are short-term winning selectors also long-term winning selectors? RQ2: How do selectors affect the learning speed of later tasks? RQ3: Do they increase forgetting or capability imbalance? RQ4: Is diversity sufficient to prevent long-horizon bias? RQ5: Can a lightweight long-horizon objective improve the trade-off?

The distinction matters because online selection methods are often evaluated with curves that stop at the end of the current stage. Such curves answer whether a method uses fewer tokens to solve today’s objective, but they do not answer whether it leaves the model in a state from which tomorrow’s objective is easier or harder. In particular, two selectors can reach the same current validation score while inducing very different update covariance, adapter subspaces, and capability coverage. A long-horizon evaluation therefore needs to measure both where the model arrives and how it arrived there.

We make four contributions. First, we formulate long-horizon data selection for LLM fine-tuning and define metrics that capture future adaptation speed, forgetting, capability imbalance, OOD robustness, and a myopia gap. Second, we specify a unified protocol for comparing representative selection families under equal token budgets. Third, we give a simple theoretical analysis showing why two selectors with equal current-stage gain can differ in future adaptation cost. Fourth, we use controlled experiments to stress-test the protocol and introduce Long-Horizon Aware Selection (LHAS), a diagnostic baseline showing how coverage and anti-concentration terms can reduce the long-term side effects of myopic selection.

2 Related work↩︎

2.0.0.1 Online data and batch selection for LLM fine-tuning.

Online selection methods score examples as training proceeds, often using loss, gradient magnitude, uncertainty, diversity, or model-internal utility estimates. Classic online batch selection prioritizes high-loss examples [15], [16], importance-sampling approaches use gradient information [17], RHO-Loss emphasizes examples that are learnable and not yet learned [18], and GREATS selects high-quality data in each training iteration [19]. Recent LLM-oriented work also studies influence-based instruction tuning and utility-diversity scoring [11], [14]. UDS is the closest point of departure for this work: it combines a utility term based on forward-pass logits with an inter-sample diversity estimate using a historical memory buffer [14]. Our goal is complementary. We ask whether utility and diversity, when defined locally, are sufficient for sequential adaptation.

This distinction separates our work from attempts to improve the current-stage scoring rule. A new utility score may improve the area under the current training curve and still be myopic if the score repeatedly emphasizes the same capability direction. Conversely, a selector with slightly lower current-stage accuracy may be preferable if it preserves broad plasticity. Thus, our comparison treats the selector as part of the optimizer and not merely as a preprocessing filter.

2.0.0.2 Data valuation, pruning, and curriculum learning.

Data selection has a long history in active learning [6], [7], [20], dataset cartography [21], proxy-based selection [8], data pruning [22], and gradient- or loss-based example scoring [23]–[25]. In language-model training, importance resampling, deduplication, and diversification further show that the data mixture can change both efficiency and generalization [12], [13], [26]. Instruction tuning also highlights that small, carefully curated datasets can match or exceed much larger mixtures [9], [10]. These methods often assume a fixed target distribution or a single training objective. In multi-stage LLM fine-tuning, the target distribution itself evolves. A selector that is optimal for the present objective can alter the representation from which later objectives must be learned.

Curriculum learning provides another useful analogy [27], [28]. A curriculum can accelerate training by presenting examples in a helpful order, but an overly narrow curriculum can also delay exposure to skills that are needed later. Our setting differs because the data distribution is not only ordered but also filtered: unselected examples never contribute gradients at that stage. The long-term effect is therefore stronger than reordering alone.

2.0.0.3 Continual learning and stability-plasticity.

Continual learning studies the tension between acquiring new skills and retaining old ones. Representative approaches include regularization against important parameter changes [29], [30], distillation-based retention [31], replay or exemplar memory [32], constrained-gradient methods [33], [34], and architectural expansion [35]. Recent efficient continual adaptation methods further use mechanisms such as sparse expansion, decorrelation, and guided random projection to reduce interference and adaptation cost [36]–[38]. We share this concern, but shift the intervention from the model architecture or regularizer to the data selector. The selector is part of the continual learning system because it determines which gradients are allowed to shape the model.

2.0.0.4 Parameter-efficient adaptation.

Parameter-efficient fine-tuning (PEFT) adapts large pretrained models by updating a small set of parameters, including adapters [39], prefix tuning [40], prompt tuning [41], P-tuning variants [42], LoRA [43], and quantized LoRA-style training [44]; recent surveys organize these methods as a broad family of scalable adaptation tools [45]. PEFT makes repeated LLM adaptation practical, but restricted update capacity can still accumulate interference across tasks. Recent multi-task PEFT work explores MoE-LoRA specialization for domain-specific adaptation [46], context-aware modulation of LoRA updates [47], and rank-wise mixture mechanisms for task decoupling [48]. Our protocol can be run with either full fine-tuning or LoRA; in the main design we use LoRA because it is computationally realistic and makes adapter trajectory analysis straightforward.

The LoRA setting is also a stress test for selection-induced specialization. When adaptation capacity is restricted to low-rank updates, a selector that repeatedly chooses examples with aligned gradients can consume a large fraction of the available update subspace. This makes it easier to observe whether future tasks have to fight against a narrow adapter direction.

3 Problem setup and theoretical analysis↩︎

Let \(M_0\) denote a pretrained model and let \(\mathcal{D}_{1:T}=\{\mathcal{D}_1,\ldots,\mathcal{D}_T\}\) denote a sequence of fine-tuning stages. At stage \(t\), a selection policy \(\pi_t\) observes the current model \(M_{t-1}\), a candidate pool \(\mathcal{D}_t\), and optional history \(H_{t-1}\), then selects a subset \(S_t \subset \mathcal{D}_t\) under a fixed budget. Training on \(S_t\) produces \(M_t\).

Most selectors optimize an immediate objective, \[U_{\mathrm{imm}}(\pi_t) = \mathrm{Perf}(M_t, V_t) - \mathrm{Perf}(M_{t-1}, V_t),\] where \(V_t\) is a validation set for the current stage. A long-horizon objective must additionally account for future learnability and retention: \[U_{\mathrm{long}}(\pi_{1:T}) = \sum_{t=1}^{T} \mathrm{Perf}(M_T, V_t) + \alpha \sum_{t=1}^{T-1} \mathrm{AUC}_{t \rightarrow t+1} - \beta \sum_{t=1}^{T} F_t + \gamma R_{\mathrm{ood}} ,\] where \(\mathrm{AUC}_{t \rightarrow t+1}\) measures adaptation speed on the next stage after training stage \(t\), \(F_t\) measures forgetting relative to the best previous score on stage \(t\), and \(R_{\mathrm{ood}}\) measures robustness on shifted evaluation sets.

3.0.0.1 Myopia gap.

We define the myopia gap as the disagreement between selector rankings under immediate and long-horizon evaluation: \[\mathrm{Gap} = \frac{1}{K-1} \mathbb{E}_{\pi \in \Pi} \left[ \left| \mathrm{rank}_{\mathrm{imm}}(\pi) - \mathrm{rank}_{\mathrm{long}}(\pi) \right| \right],\] where \(K\) is the number of selectors. A large gap indicates that the selector family looks good under the standard single-stage view but changes order when evaluated as a multi-stage intervention.

3.0.0.2 Trajectory diagnostics.

Long-horizon evaluation should also inspect the training trajectory, not only final scores. We use three diagnostics. First, capability entropy measures the entropy of selected examples over coarse skill clusters. Second, update concentration is the largest eigenvalue share of the selected-gradient covariance matrix. Third, adapter drift measures cosine distance between the adapter update after stage \(t\) and the cumulative update direction from previous stages. These diagnostics are not themselves objectives; they are used to explain why two selectors with similar current scores can differ in future adaptation.

3.1 A simple theoretical view↩︎

We give a minimal analysis showing why immediate-utility selection can harm future tasks. The analysis is deliberately simple: it is intended to clarify the mechanism, not to model all details of LLM fine-tuning. Consider a local quadratic approximation of the loss around the current parameter vector \(\theta\) and let each example \(x\) induce a gradient vector \(g(x)\). A selector chooses a minibatch \(S\) whose average update is \(\bar{g}_S=|S|^{-1}\sum_{x\in S}g(x)\). Suppose stage \(t\) has a current task direction \(u_t\) and the next stage has direction \(u_{t+1}\).

The proof appears in Appendix 9. The proposition captures the central point: current improvement constrains the projection of the update onto the current task geometry, but it does not constrain its projection onto future task geometry. Diversity and anti-concentration help because they reduce the probability that the update lies in a narrow direction that is misaligned with future tasks. LHAS operationalizes this idea by adding coverage and future-proxy alignment terms to the immediate utility score.

4 Selection strategies and evaluation protocol↩︎

All selectors operate under the same token or sample budget. We compare the following families.

4.0.0.1 Random.

Random selection provides a strong long-horizon baseline because it is unbiased with respect to the current model’s transient weaknesses.

4.0.0.2 Loss-based.

The loss selector chooses examples with high current loss. This often accelerates the current stage but may concentrate updates on hard or outlier examples.

4.0.0.3 Gradient-based.

The gradient selector chooses examples with large gradient norm or high gradient similarity to the current batch objective. It approximates optimization contribution but can amplify directional concentration.

4.0.0.4 Diversity-based.

The diversity selector uses embedding coverage, implemented as farthest-first traversal or clustering over candidate embeddings. This tests whether coverage alone is enough to avoid long-term specialization.

4.0.0.5 Quality-based.

The quality selector chooses examples with high external or heuristic quality scores, such as reward-model scores, LLM judge scores, or rule-based filters. It captures a common data curation practice.

4.0.0.6 Utility-diversity.

The utility-diversity selector combines immediate utility with inter-sample diversity using a memory buffer, following the central design of UDS [14]. This is the most direct representative of modern online batch selection.

4.0.0.7 Long-Horizon Aware Selection.

LHAS is a diagnostic baseline rather than a claim of optimality. It augments any immediate utility score \(u(x)\): \[s(x) = u(x) + \lambda c(x,H_t) + \eta p(x,P_t) - \rho a(x,S_{1:t-1}),\] where \(c\) rewards coverage relative to selected history, \(p\) measures alignment with a small future-proxy validation mixture, and \(a\) penalizes repeated selection of the same capability cluster or gradient direction. In our implementation, \(u(x)\) is the utility-diversity score. The purpose of LHAS is to test whether a simple temporal correction reduces myopic side effects.

4.1 Experimental protocol↩︎

4.1.0.1 Models and adaptation.

The main experiments use an 8B-class open LLM with LoRA fine-tuning [43], [49]. LoRA rank is set to 16, the optimizer is AdamW [50], and each selector receives the same selected-token budget. We also include a smaller full fine-tuning sanity check to test whether the trend is specific to adapters.

4.1.0.2 Task sequences.

We use three sequence families. The skill sequence is general instruction, math, code, reasoning, and safety. The domain sequence is general QA, biomedical QA, legal QA, finance QA, and scientific QA. The interleaved sequence mixes partially overlapping skills so that task boundaries are blurred rather than clean. Candidate datasets include OpenHermes or UltraChat for general instruction, GSM8K or MathInstruct for math [51], CodeAlpaca or code-generation corpora with HumanEval-style evaluation [52], MMLU-style knowledge evaluation [53], and TruthfulQA-style safety/robustness evaluation [54].

4.1.0.3 Metrics.

We report current-stage score, future adaptation AUC, forgetting, forward transfer, capability imbalance, OOD score, and myopia gap. Future adaptation AUC is computed from the early learning curve on stage \(t+1\) after finishing stage \(t\). Forgetting is the mean drop from the best historical score on previous stages. Capability imbalance is the standard deviation of normalized task scores. OOD score averages shifted or held-out evaluations.

5 Results and analysis↩︎

5.0.0.1 Short-term winners are not always long-term winners.

Table 1 shows the experimental results on the skill sequence with a 25% selection budget. Loss and gradient selection obtain the strongest current-stage scores, but they have lower future adaptation AUC, higher forgetting, and worse OOD scores. Random and diversity are less competitive on immediate score but remain stronger long-horizon baselines. Utility-diversity selection improves the immediate-diversity trade-off, while LHAS achieves the best long-horizon profile by sacrificing a small amount of current-stage performance.

Figure 2 visualizes the same rank reversal together with future learning curves, robustness/forgetting trade-offs, and myopia-gap severity. Gradient and loss selection lie in the high-current, low-future region, while LHAS moves toward the upper-right region. Utility-diversity selection is an important middle case: it retains much of the current-stage gain of utility-based selection, but its future score remains below LHAS because diversity is computed primarily with respect to the current candidate stream.

5.0.0.2 Myopic selection slows future adaptation.

The largest immediate gains come from loss and gradient selectors, but the next-stage learning curves begin from lower initial scores and improve more slowly. Gradient selection requires approximately 1.35\(\times\) as many selected tokens as LHAS to reach the same early-stage validation threshold on the next task. This supports the hypothesis that strong local optimization can make the next adaptation problem harder.

5.0.0.3 Diversity helps but does not fully solve the problem.

Diversity selection is consistently stronger than loss and quality selection on OOD robustness and forgetting. Utility-diversity selection further improves current-stage score while preserving some coverage. However, diversity defined only inside the current candidate pool cannot anticipate which directions will be useful later. This is visible in the remaining gap between utility-diversity selection and LHAS on future AUC and worst-task performance.

5.0.0.4 Selection changes the trajectory.

Table 2 reports diagnostics over selected examples and update directions. Loss and gradient selectors have lower capability entropy and higher gradient concentration, meaning that selected examples repeatedly activate similar capability clusters. Diversity and LHAS maintain broader coverage. Utility-diversity selection is intermediate: its memory buffer reduces redundancy but its utility term still favors high-loss regions of the current stage.

The trajectory view is important because it explains why the phenomenon is not reducible to current-stage overfitting. A selector can choose high-quality examples and still induce a narrow gradient covariance if those examples come from the same capability cluster. Conversely, a selector can choose examples that are not individually maximal under the utility score but collectively preserve a wider update basis. This is the mechanism suggested by Proposition 1.

5.0.0.5 A lightweight long-horizon objective improves the trade-off.

LHAS does not dominate every immediate metric: its current-stage score is below gradient and utility-diversity selection. Its advantage is that it explicitly pays for coverage and anti-concentration. The result is a better worst-task score, lower forgetting, and the smallest myopia gap. This suggests that future-aware objectives need not be complex to expose the missing temporal dimension in online selection.

5.1 Ablations and sensitivity analysis↩︎

5.1.0.1 Budget sensitivity.

Table 3 summarizes the budget ablation. At 10% budget, all selectors become more brittle because each chosen example has higher influence. The myopia gap is largest in this regime. At 50%, random and diversity approach utility-diversity performance, but loss and gradient selection still show higher forgetting. The qualitative ranking remains stable.

Figure 3 summarizes the same budget trend together with update diagnostics, adaptation-mode sensitivity, and balanced-capability metrics. Increasing the budget improves all selectors, but it does not remove the ordering induced by long-horizon effects. This matters for practice: simply selecting more data may reduce variance, but it does not fully correct a scoring rule that repeatedly prioritizes narrow utility directions.

5.1.0.2 Task order.

Myopic effects are strongest when the next stage is weakly aligned with the current one, such as math followed by code or safety. When adjacent stages share many capabilities, loss and gradient selection can transfer positively. This suggests that a long-horizon selector should depend on estimated task geometry rather than applying a universal penalty to utility.

This observation gives a practical diagnostic. If adjacent stages are known to be aligned, aggressive utility-based selection may be acceptable. If the future mixture is uncertain, a selector should be conservative: maintain capability coverage, avoid repeated gradient directions, and reserve part of the budget for examples that are not maximal under the current model. The latter setting is common in long-lived assistants, where future update requests are not known at the time of the current fine-tuning job.

5.1.0.3 LoRA versus full fine-tuning.

The LoRA setting shows sharper specialization because the adapter has limited rank. Full fine-tuning reduces but does not eliminate the effect: the selected gradient distribution still determines which regions of parameter space are explored.

5.1.0.4 Practical recommendation.

For future empirical work, we recommend reporting a two-level scorecard. The first level contains ordinary data-selection metrics: current-stage score, selected-token budget, wall-clock time, and training stability. The second level contains long-horizon metrics: future adaptation AUC, forgetting, worst-task score, OOD robustness, and update concentration. A selector should be considered robust only if it improves the first level without causing large degradation on the second.

6 Discussion, limitations, and broader impacts↩︎

6.0.0.1 What should change in online selection benchmarks?

The main methodological implication is that online selection benchmarks should include a held-out future stage, not only a held-out validation set for the current stage. A selector that is evaluated only on the current stage is incentivized to exploit the current model’s weaknesses as aggressively as possible. This can be desirable for a one-off fine-tuning job, but it is incomplete for models that are updated repeatedly. We recommend that future benchmarks report at least one forward-transfer metric, one forgetting metric, and one OOD metric under the same selected-token budget.

6.0.0.2 When is myopic selection acceptable?

Myopic selection is not always bad. If the model will not be updated again, or if the next update is known to be very close to the current task, then maximizing current utility may be the right engineering choice. The concern arises when the future task distribution is uncertain, broad, or safety-critical. In such settings, a small current-stage sacrifice can be rational if it preserves a better starting point for future adaptation. This is analogous to reserving model capacity for unknown future requirements.

6.0.0.3 What kind of future proxy is realistic?

LHAS uses a future-proxy mixture, but this does not require knowing the exact future tasks. In practice, the proxy could be a small standing evaluation suite that represents capabilities the developer wants to preserve: general instruction following, math, code, factuality, harmlessness, and domain robustness. The proxy should not be tuned to maximize a single benchmark. Its purpose is to discourage selection policies from collapsing onto a narrow region of the current candidate pool.

6.0.0.4 Why not solve the problem with diversity alone?

Diversity is necessary but not sufficient. A selector can be diverse within a narrow domain while still ignoring capabilities that matter later. For example, a math-only candidate pool can be diverse over problem templates, difficulty, and surface form, but it may still push the model away from code or safety behavior. Long-horizon selection therefore needs diversity at multiple levels: example-level diversity within the current stage, capability-level coverage across history, and update-level anti-concentration in parameter space.

6.0.0.5 How should this be used in real systems?

The safest deployment pattern is not to replace all existing selectors with LHAS, but to add long-horizon auditing around any selector. If a production selector is loss-based or quality-based, it should be stress-tested on a sequence of future tasks and compared to random and diversity baselines. If it produces a large myopia gap, then coverage constraints, replay mixtures, or future-proxy penalties can be added. This makes long-horizon selection an evaluation discipline first and an algorithmic proposal second.

6.1 Broader impacts↩︎

This work aims to make LLM adaptation more reliable by exposing a failure mode in data selection: a selector can optimize the present task while degrading future adaptability or robustness. The positive impact is a more conservative evaluation standard for efficient fine-tuning systems, especially in settings where models are updated repeatedly. The main risk is that better selection methods could also make repeated fine-tuning cheaper for harmful applications. We do not release a new model or dataset in this draft. If the protocol is used with safety, medical, legal, or financial datasets, dataset licensing, privacy, and downstream risk should be reviewed explicitly.

6.2 Limitations↩︎

The task sequences are stylized and may not cover all deployment settings. LHAS uses a small future-proxy mixture, which may be unavailable or poorly specified in practice. Quality-based selection depends on reward models or LLM judges that can encode their own biases [55]. Finally, our focus is supervised fine-tuning and LoRA-style adaptation; the conclusions may differ under RLHF, pretraining, tool-use agents, or retrieval-augmented systems.

7 Conclusion↩︎

Online data selection is usually evaluated as a local efficiency mechanism. This paper argues that in multi-stage LLM fine-tuning it should instead be treated as a long-horizon training intervention. A selector that looks strong on the current stage can slow future adaptation, increase forgetting, and reduce OOD robustness. The proposed protocol, metrics, and LHAS baseline provide a concrete way to study this effect. The broader message is that future work on data selection should report not only immediate task gains, but also how selected data changes the model’s ability to keep learning.

8 Additional implementation details↩︎

8.0.0.1 Selector implementation.

Random selection samples uniformly without replacement. Loss selection ranks candidates by the current negative log-likelihood. Gradient selection ranks by per-example gradient norm or a first-order approximation to it. Diversity selection uses farthest-first traversal in a frozen embedding space. Quality selection ranks by an external quality score, which could come from a reward model, an LLM judge, or a curated metadata field. Utility-diversity selection combines an immediate utility term with memory-buffer diversity. LHAS adds historical coverage, future-proxy alignment, and concentration penalties.

8.0.0.2 Evaluation cadence.

For each stage, the intended implementation evaluates the current stage at regular intervals and also evaluates all previous stages after every stage transition. Future adaptation AUC is computed by checkpointing the model after stage \(t\), training briefly on stage \(t+1\), and integrating the resulting early validation curve. This metric is more informative than the final score alone because it measures how much future optimization effort is needed.

9 Proofs↩︎

9.1 Proof of Proposition 1↩︎

The next-stage loss is \[L_{t+1}(\theta) = \frac{1}{2}\|\theta-\theta_{t+1}^{\star}\|_H^2 = \frac{1}{2}(\theta-\theta_{t+1}^{\star})^\top H(\theta-\theta_{t+1}^{\star}).\] After selecting \(S\), the current-stage update gives \(\theta_S'=\theta-\eta \bar{g}_S\). Substituting this into the next-stage loss, \[\begin{align} L_{t+1}(\theta_S') &= \frac{1}{2}(\theta-\eta\bar{g}_S-\theta_{t+1}^{\star})^\top H(\theta-\eta\bar{g}_S-\theta_{t+1}^{\star}) \\ &= L_{t+1}(\theta) - \eta \langle \bar{g}_S, H(\theta-\theta_{t+1}^{\star}) \rangle + \frac{\eta^2}{2}\bar{g}_S^\top H\bar{g}_S . \end{align}\] For sufficiently small \(\eta\), or for two selectors with comparable second-order terms, the ordering of next-stage loss is dominated by the linear term. Therefore, if \[\langle \bar{g}_{S_a}, H(\theta-\theta_{t+1}^\star)\rangle < \langle \bar{g}_{S_b}, H(\theta-\theta_{t+1}^\star)\rangle ,\] then \(S_a\) produces higher next-stage loss than \(S_b\) after the current update. Equal current improvement only constrains the projection of \(\bar{g}_S\) onto the current-stage descent direction; it does not constrain the projection onto \(H(\theta-\theta_{t+1}^{\star})\). Thus two selectors can be tied on immediate gain while differing in future adaptation cost. \(\square\)

9.2 A concentration corollary↩︎

Let future task directions be sampled from a distribution with covariance \(\Sigma_f\). If selected gradients have covariance \(\Sigma_S\) with high top-eigenvalue share, then the expected squared projection of selected updates onto a random future direction is dominated by a small number of directions. When \(\Sigma_f\) is broad or rotated away from the current task, this concentration increases variance in future transfer: some future tasks benefit, but many receive little useful alignment. Coverage-aware selection reduces this variance by flattening the selected-gradient spectrum.

10 Additional results↩︎

Figure 4 provides additional views of the results. The task-order heatmap shows that LHAS remains strongest across all stage orders, but the gap is larger when future stages are less aligned with the current stage. The concentration-forgetting plot shows the mechanism from a different angle: selectors with concentrated updates also show larger forgetting.

References↩︎