Multilingual Fine-Tuning via
Localized Gradient Conflict Resolution
June 04, 2026
Abstract
The rapid evolution of Large Language Models (LLMs) has established cross-lingual versatility as a defining feature of modern systems. However, fine-tuning these models frequently induces negative interference across languages. To address this, we reformulate multilingual fine-tuning as a multi-objective optimization (MOO) problem. Specifically, we introduce Bucket-Level MOO, a scalable distributed framework that applies gradient-based MOO algorithms locally on parameter buckets. This enables conflict-aware updates without the prohibitive communication overhead of reconstructing full gradient vectors. Theoretically, we prove this localized resolution natively enforces Refined Pareto Stationarity, a strictly tighter necessary condition for Pareto optimality. Empirically, Bucket-Level MOO mitigates interference by driving LLMs to construct distinct language-specific dimensions, improving representational separability. Extensive experiments across four base LLMs demonstrate that our method significantly improves both seen and unseen multilingual performance over standard fine-tuning paradigms.1
1 Introduction↩︎
The multilingual capabilities of Large Language Models (LLMs) have advanced rapidly, driven largely by data-centric strategies such as translating English corpora or distilling knowledge from high-resource languages [1]–[3]. While these approaches improve average cross-lingual performance, they implicitly assume that a single set of shared parameters can accommodate diverse and often incompatible linguistic patterns. In practice, this assumption frequently breaks down during fine-tuning: updates that benefit one language can adversely affect others, leading to negative interference, a phenomenon where optimizing for one language degrades performance on another [4], [5]. As multilingual training scales to more languages and tasks, mitigating such interference becomes a central challenge for achieving robust and balanced cross-lingual performance.
From an optimization perspective, negative interference is fundamentally driven by conflicting gradients across language-specific objectives [6]–[8]. As illustrated in Figure 1 (Left), models with limited initial multilingual alignment (e.g.,
Meta-Llama-3-8B) exhibit severe, sustained gradient volatility throughout fine-tuning. Their pairwise cosine similarities frequently plunge into negative regimes with massive variance, severely constraining downstream progress. Conversely,
stronger base multilingual models (e.g., Qwen3-8B-Base) are significantly more stable and maintain a largely positive average alignment. However, they still experience occasional, sharp gradient conflicts. These observations suggest the
inherent instability of optimizing for diverse languages simultaneously.
A natural approach to mitigating such conflicts is to cast multilingual fine-tuning as a Multi-Objective Optimization (MOO) problem [9], [10]. In this framework, each language defines a separate objective, and the ultimate goal is to achieve Pareto optimality—a state where no language’s performance can be improved without degrading another’s. To move toward this optimal state, conventional MOO algorithms seek a common descent direction that guarantees Pareto stationarity, the widely accepted first-order necessary condition for optimality.
However, directly applying these standard methods (e.g., MGDA [6] and CAGrad [11], which ensure Pareto stationarity, or heuristics like PCGrad [7]) to LLMs is challenging. These methods require access to full gradient vectors across all objectives, incurring substantial communication and memory overhead in distributed training settings. Beyond scalability, they also treat the model as a monolithic parameter space, implicitly assuming conflicts are globally uniform. In practice, linguistic processing within LLMs is structurally heterogeneous, e.g., various layers perform distinct roles [12], [13]. As shown in Figure 1 (Right), gradient conflicts are highly localized, often concentrating in specific layers (e.g., early or late layers). This mismatch implies that global gradient aggregation can obscure severe local conflicts, motivating the need for more fine-grained, structure-aware optimization strategies.
To address these challenges, we introduce Bucket-Level MOO, a scalable framework that applies gradient-based MOO at the level of parameter partitions. Concretely, instead of forming and manipulating full-model gradients, we intercept the backward pass in distributed training (e.g., ZeRO [14] or FSDP [15]) and apply MOO algorithms independently within each parameter bucket before gradient reduction between GPUs. This design enables conflict-aware updates using only locally available gradients, avoiding the need for expensive global aggregation. By resolving conflicts at the same granularity at which gradients are partitioned and communicated, our method aligns naturally with the structural and systems properties of modern LLM training. Theoretically, we show that this localized optimization is not merely a heuristic approximation: it enforces Refined Pareto Stationarity [16], a stricter necessary condition than standard Pareto stationarity, thereby providing stronger guarantees on the quality of the obtained solutions.
Our main contributions are summarized as follows:
Practically, we reveal the layer-wise heterogeneity of negative interference in modern LLMs, motivating the critical need for localized, rather than global, conflict resolution.
Methodologically, we propose Bucket-Level MOO, an optimization framework that seamlessly integrates into distributed training paradigms, enabling structure-aware conflict resolution while maintaining memory and communication efficiency during large-scale fine-tuning.
Empirically, we demonstrate through extensive experiments that Bucket-Level MOO significantly improves both in-distribution and out-of-distribution multilingual capabilities, successfully carving out highly separable, language-specific representation dimensions.
2 Preliminaries↩︎
This section frames multilingual fine-tuning as a multi-objective optimization problem and discusses conventional training approaches. Formal definitions and theorems are introduced in Appendix 7.
2.1 Negative Interference in Multilingual Fine-Tuning↩︎
Given a set of target languages denoted by \(\mathcal{T} := \{1, 2, \dots, T\}\), we formulate the multilingual fine-tuning of a language model parameterized by \(\boldsymbol{\theta}\) as a Multi-Objective Optimization (MOO) problem. Specifically, the goal is to simultaneously minimize potentially conflicting objectives across all languages. This is formally expressed as: \[\begin{align} \min_{\boldsymbol{\theta}}\; \mathbf{L}(\boldsymbol{\theta}), \quad \text{where} \quad \mathbf{L}(\boldsymbol{\theta}) = \left[ \mathcal{L}_1(\boldsymbol{\theta}), \dots, \mathcal{L}_T(\boldsymbol{\theta})\right]^\top,\label{equ:main} \end{align}\tag{1}\] where \(\mathcal{L}_t(\boldsymbol{\theta})\) denotes the loss for language \(t \in \mathcal{T}\) when trained on its corresponding dataset \(\mathcal{D}_{t}\).
However, when optimizing the objective defined in Equation 1 , a prevalent challenge is the emergence of conflicting gradients. Specifically, two gradients \(\nabla_{\boldsymbol{\theta}}\mathcal{L}_i\) and \(\nabla_{\boldsymbol{\theta}}\mathcal{L}_j\) are considered conflicting if their cosine similarity is negative, or equivalently, if their inner product is negative: \(\langle \nabla_{\boldsymbol{\theta}}\mathcal{L}_i, \nabla_{\boldsymbol{\theta}}\mathcal{L}_j \rangle < 0\). This conflict leads to negative interference. Consider the first-order Taylor expansion approximating the change in loss \(\mathcal{L}_j\) following a gradient update for language \(i\): \[\begin{align} \Delta \mathcal{L}_j \approx \mathcal{L}_j\left(\boldsymbol{\theta} - \eta\nabla_{\boldsymbol{\theta}}\mathcal{L}_i\right) - \mathcal{L}_j\left(\boldsymbol{\theta}\right) \approx -\eta \langle \nabla_{\boldsymbol{\theta}}\mathcal{L}_j, \nabla_{\boldsymbol{\theta}}\mathcal{L}_i \rangle, \end{align}\] where \(\eta\) denotes the learning rate. As shown, negative interference (\(\Delta \mathcal{L}_j > 0\)) occurs precisely when the gradients conflict (\(\langle \nabla_{\boldsymbol{\theta}}\mathcal{L}_j, \nabla_{\boldsymbol{\theta}}\mathcal{L}_i \rangle < 0\)), meaning that an update step for language \(i\) inadvertently degrades the performance on language \(j\).
Consequently, naive optimization via minimizing the average loss (\(\frac{1}{T}\sum_{t}\mathcal{L}_t\)) risks improving dominant languages at the expense of others, as illustrated in Figure 2 (a).
2.2 Gradient-Based Conflict Resolution↩︎
To mitigate negative interference, various gradient manipulation algorithms identify an optimal descent direction that balances competing objectives better than naive averaging.
2.2.0.1 Multiple Gradient Descent Algorithm (MGDA).
MGDA [17], [18] seeks the minimum-norm vector within the convex hull of the language-specific gradients by solving a quadratic program for combination weights \(\boldsymbol{\alpha}\): \[\begin{align} \min_{\alpha_1,\dots, \alpha_T} \left\{ \left\|\sum_{t=1}^T\alpha_t \nabla_{\boldsymbol{\theta}}\mathcal{L}_t(\boldsymbol{\theta}) \right\|_2^2 \;\Bigg|\; \sum_{t=1}^T\alpha_t=1, \alpha_t \geq 0 \;\forall t \right\}. \label{eq:mgda} \end{align}\tag{2}\] The resulting update vector ensures a common descent direction that decreases the loss for all objectives simultaneously, preventing negative interference.
2.2.0.2 Conflict-Averse Gradient Descent (CAGrad).
CAGrad [11] bridges the gap between the strict conflict avoidance of MGDA and the fast convergence of average gradient descent. It maximizes the worst-case local improvement among all objectives, constrained within a trust region around the average gradient \(\mathbf{g}_{\text{avg}}\): \[\begin{align} \mathbf{d}^{\text{CAGrad}} = \mathop{\mathrm{arg\,max}}_{\mathbf{d}} \min_{t \in \{1,\dots,T\}} \langle \nabla_{\boldsymbol{\theta}}\mathcal{L}_t, \mathbf{d} \rangle \quad \text{s.t.} \quad \|\mathbf{d} - \mathbf{g}_{\text{avg}}\| \leq c \|\mathbf{g}_{\text{avg}}\|, \label{eq:cagrad} \end{align}\tag{3}\] where \(c \in [0, 1)\) controls the degree of conflict aversion.
2.2.0.3 Projected Conflicting Gradients (PCGrad).
PCGrad [7] uses direct “gradient surgery”. For conflicting gradients \(g_i = \nabla_{\boldsymbol{\theta}}\mathcal{L}_i(\boldsymbol{\theta})\) and \(g_j = \nabla_{\boldsymbol{\theta}}\mathcal{L}_j(\boldsymbol{\theta})\) (i.e., \(\langle g_i, g_j \rangle < 0\)), PCGrad replaces \(g_i\) with its projection \(g_i^{PC}\) onto the orthogonal complement of \(g_j\): \[\begin{align} \mathbf{d}^{\text{PCGrad}} = \sum_{t=1}^T g_t^{PC}, \quad g_i^{PC} = g_i - \frac{\langle g_i, g_j \rangle}{\|g_j\|^2} g_j. \label{eq:pcgrad95proj} \end{align}\tag{4}\] This neutralizes the conflict by ensuring \(\langle g_i^{PC}, g_j \rangle = 0\).
3 Bucket-Level Multi-Objective Optimization↩︎
3.1 Motivation↩︎
In multilingual fine-tuning, we seek Pareto optimality (PO), where no language improves without degrading another. Since exact optimality is computationally intractable, conventional algorithms target Pareto stationarity (PS), a necessary condition where the zero vector lies within the global convex hull of task gradients: \[\mathbf{0} \in \text{conv}\left\{ \nabla_{\boldsymbol{\theta}} \mathcal{L}_1(\boldsymbol{\theta}^*), \dots, \nabla_{\boldsymbol{\theta}}\mathcal{L}_T(\boldsymbol{\theta}^*)\right\}.\] This implies a global weight \(\boldsymbol{\alpha} \in \Delta\) exists such that \(\sum_{t=1}^T \alpha_t\nabla_{\boldsymbol{\theta}}\mathcal{L}_t(\boldsymbol{\theta}^*) = \mathbf{0}\), where \(\Delta\) is a \(T\)-dimensional simplex. Moreover, [16] show that exploiting network structure yields a tighter criterion. By partitioning \(\boldsymbol{\theta}\) into \(K\) disjoint blocks \(\mathcal{P} = \{ \boldsymbol{\theta}_{(1)},\dots, \boldsymbol{\theta}_{(K)}\}\), a solution \(\boldsymbol{\theta}^*\) achieves Refined Pareto stationarity (RPS) if: \[\mathbf{0} \in \prod_{k=1}^{K} \text{conv}\left\{ \nabla_{\boldsymbol{\theta}_{(k)}}\mathbf{L}(\boldsymbol{\theta}^*)\right\}.\] Equivalently, each block \(k\) has a local weight \(\boldsymbol{\alpha}^k \in \Delta\) satisfying \(\sum_{t=1}^T \alpha^k_t\nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_t(\boldsymbol{\theta}^*) = \mathbf{0}\).
Figure 3: RPS is a stronger candidate for PO than PS..
As established by [16], refined Pareto stationarity is a strictly tighter necessary condition for Pareto optimality (Figure 3). Indeed, relying strictly on global Pareto stationarity can obscure valid descent directions within specific layers. As shown in Figure 1, this occurs because aggregating gradients across the entire model frequently dilutes and hides severe localized conflicts, particularly within the early and late layers during multilingual fine-tuning.
Beyond the aforementioned theoretical limitations, applying global multi-objective optimization to LLMs introduces severe scalability bottlenecks. In distributed training frameworks such as DeepSpeed ZeRO [14], gradients are partitioned across multiple GPUs, meaning that a single device never holds the
complete gradient vector \(\nabla_{\boldsymbol{\theta}}\mathcal{L}_t\). Consequently, applying standard global MOO becomes computationally prohibitive due to both communication and memory constraints. First, computing
global optimization objectives incurs substantial communication overhead, as it requires gathering full gradient vectors via expensive All-Gather operations at every training step, effectively undermining ZeRO’s communication efficiency.
Second, the approach imposes severe memory limitations, because storing \(T\) distinct copies of the full parameter gradient to compute the optimal update direction easily exceeds the memory capacity of individual
accelerators.
3.2 Bucket-Level Multi-Objective Optimization↩︎
Inspired by the motivations above, we offer localized gradient conflict resolution. Rather than treating the model as a monolithic entity where global aggregation obscures distinct, layer-wise gradient interference, we apply gradient-based MOO directly at the bucket level—resolving conflicts strictly locally where they occur. Let the global parameters \(\boldsymbol{\theta}\) be partitioned into \(K\) disjoint buckets \(\{\boldsymbol{\theta}_{(k)}\}_{k=1}^K\). In a distributed setup with \(T\) GPUs, assume each GPU \(t \in \{1, \dots, T\}\) processes a language-specific micro-batch for the corresponding language \(t\).
During the backward pass, before the standard gradient reduction occurs, each GPU \(t\) computes the local gradients of bucket \(k\) with respect to its assigned language objective \(\mathcal{L}_t\), denoted as \(\nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_t\). We then aggregate these bucket-level gradients across all \(T\) GPUs to form a local gradient set: \[\mathbf{G}_{(k)} = \left\{ \nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_1, \dots, \nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_T \right\},\] and apply the multi-objective optimization algorithm locally to \(\mathbf{G}_{(k)}\). By resolving conflicts within each bucket independently, we achieve a refined optimization trajectory while seamlessly integrating with modern distributed training systems.
3.2.0.1 Bucket-Level MGDA.
We solve the local quadratic program for each bucket \(k\): \[\begin{align} \min_{\boldsymbol{\alpha}^{(k)}} \left\| \sum_{t=1}^T \alpha^{(k)}_t \nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_t \right\|_2^2 \quad \text{s.t.} \quad \sum \alpha^{(k)}_t = 1. \label{eq:shard-mgda} \end{align}\tag{5}\] This yields local weights \(\boldsymbol{\alpha}^{(k)}\) and the corresponding update \(\mathbf{d}^{\text{MGDA}}_{(k)}\) specific to bucket \(k\).
3.2.0.2 Bucket-Level CAGrad.
We locally solve Equation 3 . With \(\mathbf{g}^{avg}_{(k)}\) as the local average gradient, we maximize the worst local improvement: \[\begin{align} \mathbf{d}^{\text{CAGrad}}_{(k)} = \mathop{\mathrm{arg\,max}}_{\mathbf{d}_{(k)}} \min_{t} \langle \nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_t, \mathbf{d}_{(k)} \rangle \quad \text{s.t.} \quad \|\mathbf{d}_{(k)} - \mathbf{g}^{avg}_{(k)}\| \leq c \|\mathbf{g}^{avg}_{(k)}\|. \end{align}\] This enforces "Conflict Aversion" per bucket, keeping updates close to the local average while resolving resident conflicts.
3.2.0.3 Bucket-Level PCGrad.
Conflict detection and projection rely solely on local inner products: \[\begin{align} \text{if } \langle \nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_i, \nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_j \rangle < 0 \implies \text{Project locally on rank } k. \end{align}\]
The final global update \(\mathbf{d} = [\mathbf{d}_{(1)}, \dots, \mathbf{d}_{(K)}]\) concatenates the resolved local gradients. The illustration and pseudocode for Bucket-Level MOO are shown in Figure 4 and Appendix 10, respectively.
3.3 Bucket-Level MOO Achieves Refined Pareto Stationarity↩︎
Crucially, this distributed implementation is not merely a heuristic designed for computational efficiency; it provides a stricter theoretical condition for the optimization process. Conventional global MOO algorithms, such as Global MGDA and Global CAGrad, are mathematically guaranteed to converge only to standard Pareto stationarity (Theorems 4, 5). As discussed in Section 3.1, this global equilibrium may be suboptimal for heterogeneous networks, as it can obscure localized gradient conflicts across different layers.
In contrast, our proposed Bucket-Level variants naturally align with the structural framework of Refined Pareto Stationarity. By enforcing multi-objective optimization conditions independently on each bucket \(\boldsymbol{\theta}_{(k)}\) defined by the distributed partition \(\mathcal{P} = \{\boldsymbol{\theta}_{(1)}, \dots, \boldsymbol{\theta}_{(K)}\}\), the algorithm enforces Pareto stationarity strictly at the local level. Because this local equilibrium must be satisfied across all disjoint buckets simultaneously, the model inherently achieves Refined Pareto Stationarity. Consequently, Bucket-Level MOO enforces a strictly tighter necessary condition for Pareto optimality than conventional global MOO. This allows the optimizer to uncover valid common descent directions and exert fine-grained, layer-specific control over language interference.
Theorem 1 (Bucket-Level MGDA Convergence). Let \(\boldsymbol{\alpha}^{(k)}\) be the solution to the MGDA problem (Equation 5 ) solved locally on bucket \(k\), and let \(\mathbf{d}^{\text{MGDA}}_{(k)} = \sum_{t=1}^T\alpha^{(k)}_t \nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_t\) be the local update vector. Define the global update as \(\mathbf{d}^{\text{MGDA}} = [\mathbf{d}^{\text{MGDA}}_{(1)}, \dots, \mathbf{d}^{\text{MGDA}}_{(K)}]\). Then, one of the following holds:
\(\|\mathbf{d}^{\text{MGDA}}\|_2=0\), implying \(\|\mathbf{d}^{\text{MGDA}}_{(k)}\|_2=0\) for all \(k\), and the current parameters are Refined Pareto Stationary with respect to partition \(\mathcal{P}\).
\(-\mathbf{d}^{\text{MGDA}}\) is a valid descent direction that decreases all objectives simultaneously (i.e., \(\langle \nabla_{\boldsymbol{\theta}} \mathcal{L}_t, \mathbf{d}^{\text{MGDA}} \rangle > 0\) for all \(t\) where \(\mathbf{d}^{\text{MGDA}} \neq 0\)).
Theorem 1 guarantees that Bucket-Level MGDA preserves rigorous optimization dynamics. Concatenating locally resolved vectors produces a global update that decreases the loss for all languages simultaneously, ultimately converging to the superior candidate of Refined Pareto Stationarity.
Theorem 2 (Bucket-Level CAGrad Convergence). Let \(\mathbf{d}^{\text{CAGrad}}_{(k)}\) be the solution to the CAGrad problem solved locally on bucket \(k\) with conflict parameter \(0 \le c < 1\) and local average gradient \(\mathbf{g}^{avg}_{(k)}\). Define the global update as \(\mathbf{d}^{\text{CAGrad}} = [\mathbf{d}^{\text{CAGrad}}_{(1)}, \dots, \mathbf{d}^{\text{CAGrad}}_{(K)}]\). Then:
Global Descent: The global update \(-\mathbf{d}^{\text{CAGrad}}\) is a descent direction for the global average loss \(\mathcal{L}_{avg}\), satisfying: \(\langle \nabla_{\boldsymbol{\theta}} \mathcal{L}_{avg}, \mathbf{d}^{\text{CAGrad}} \rangle \geq \frac{1-c^2}{2} \|\nabla_{\boldsymbol{\theta}} \mathcal{L}_{avg}\|_2^2.\)
Stationarity: If \(\|\mathbf{d}^{\text{CAGrad}}\|_2 = 0\), the current parameters satisfy Refined Pareto Stationarity with uniform weights (i.e., \(\frac{1}{T}\sum \nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_t = \mathbf{0}\) for all \(k\)).
Similarly, Theorem 2 extends this theoretical safety to Bucket-Level CAGrad. While CAGrad relaxes MGDA for faster convergence, the bucket-wise application still guarantees descent for the average loss, bounded by \(c\). Crucially, optimization halts strictly when the average gradient within every single bucket vanishes, safely resolving local cross-lingual interference while driving the model toward an average-loss minimum that satisfies the refined stationarity criterion.
4 Experiments↩︎
4.1 Experimental Setup↩︎
4.1.0.1 Training Data Preparation
We construct an English corpus comprising 1,000 LIMA conversational pairs [19] and 630 s1k-1.1 reasoning samples [20] distilled from DeepSeek-R1.2 Gemini 2.0 Flash
is employed to translate this corpus into eight diverse languages (en, zh, it, ar, ko, id, bn, sw) covering the high- to low-resource spectrum.
4.1.0.2 Evaluation
We assess model performance using a comprehensive suite of benchmarks covering three core competencies:
Natural Language Understanding: We use BELEBELE [21] to evaluate parallel machine reading comprehension capabilities.
Technical Reasoning: We employ Multilingual ARC-E [22] and PolyMath (low and medium subsets) [23] to assess reasoning skills.
Knowledge-Based QA: We utilize Global-MMLU-Lite [24] to measure world knowledge retention and transfer.
To ensure a standardized comparison across all tasks, we evaluate the models on the 13 languages that intersect across these four benchmarks. These are categorized into the 8 seen languages from our training phase and 5 unseen
languages (de, es, fr, ja, pt) to evaluate cross-lingual generalization. Further implementation and computational details are in Appendices 11 and 12, respectively.
4.2 Main Results↩︎
| Method | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 2-6 (r)7-11 | BELE | ARC-E | PolyMath | MMLU | Avg. | BELE | ARC-E | PolyMath | MMLU | Avg. |
| Meta-Llama-3-8B | ||||||||||
| Base | \(51.8_{\phantom{\scriptscriptstyle +0.0}}\) | \(54.2_{\phantom{\scriptscriptstyle +0.0}}\) | \(12.0_{\phantom{\scriptscriptstyle +0.0}}\) | \(37.2_{\phantom{\scriptscriptstyle +0.0}}\) | \(38.8_{\phantom{\scriptscriptstyle +0.0}}\) | \(55.9_{\phantom{\scriptscriptstyle +0.0}}\) | \(65.2_{\phantom{\scriptscriptstyle +0.0}}\) | \(13.5_{\phantom{\scriptscriptstyle +0.0}}\) | \(42.6_{\phantom{\scriptscriptstyle +0.0}}\) | \(44.3_{\phantom{\scriptscriptstyle +0.0}}\) |
| V. SFT | \(53.5_{\phantom{\scriptscriptstyle +0.0}}\) | \(61.5_{\phantom{\scriptscriptstyle +0.0}}\) | \(18.3_{\phantom{\scriptscriptstyle +0.0}}\) | \(42.5_{\phantom{\scriptscriptstyle +0.0}}\) | \(44.0_{\phantom{\scriptscriptstyle +0.0}}\) | \(60.7_{\phantom{\scriptscriptstyle +0.0}}\) | \(71.5_{\phantom{\scriptscriptstyle +0.0}}\) | \(21.5_{\phantom{\scriptscriptstyle +0.0}}\) | \(48.9_{\phantom{\scriptscriptstyle +0.0}}\) | \(50.7_{\phantom{\scriptscriptstyle +0.0}}\) |
| MGDA | \(56.4_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.9}}\) | \(63.7_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.2}}\) | \(21.6_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.3}}\) | \(45.5_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.0}}\) | \(46.8_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.8}}\) | \(63.4_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.7}}\) | \(72.7_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.2}}\) | \(24.3_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.8}}\) | \(50.6_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.7}}\) | \(52.8_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.1}}\) |
| CAGrad | \(56.6_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.1}}\) | \(63.9_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.4}}\) | \(20.2_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.9}}\) | \(43.9_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.4}}\) | \(46.2_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.2}}\) | \(62.4_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.7}}\) | \(72.8_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.3}}\) | \(23.8_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.3}}\) | \(50.5_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.6}}\) | \(52.4_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.7}}\) |
| PCGrad | \(56.8_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.3}}\) | \(64.3_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.8}}\) | \(21.4_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.1}}\) | \(45.2_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.7}}\) | \(46.9_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.9}}\) | \(63.7_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.0}}\) | \(73.6_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.1}}\) | \(24.5_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.0}}\) | \(49.0_{\scriptscriptstyle \textcolor[HTML]{008000}{+0.1}}\) | \(52.7_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.0}}\) |
| Llama-3.1-8B | ||||||||||
| Base | \(59.1_{\phantom{\scriptscriptstyle +0.0}}\) | \(60.0_{\phantom{\scriptscriptstyle +0.0}}\) | \(13.2_{\phantom{\scriptscriptstyle +0.0}}\) | \(41.8_{\phantom{\scriptscriptstyle +0.0}}\) | \(43.5_{\phantom{\scriptscriptstyle +0.0}}\) | \(67.8_{\phantom{\scriptscriptstyle +0.0}}\) | \(70.0_{\phantom{\scriptscriptstyle +0.0}}\) | \(13.8_{\phantom{\scriptscriptstyle +0.0}}\) | \(48.9_{\phantom{\scriptscriptstyle +0.0}}\) | \(50.1_{\phantom{\scriptscriptstyle +0.0}}\) |
| V. SFT | \(53.1_{\phantom{\scriptscriptstyle +0.0}}\) | \(61.3_{\phantom{\scriptscriptstyle +0.0}}\) | \(20.3_{\phantom{\scriptscriptstyle +0.0}}\) | \(44.2_{\phantom{\scriptscriptstyle +0.0}}\) | \(44.7_{\phantom{\scriptscriptstyle +0.0}}\) | \(61.3_{\phantom{\scriptscriptstyle +0.0}}\) | \(72.6_{\phantom{\scriptscriptstyle +0.0}}\) | \(22.8_{\phantom{\scriptscriptstyle +0.0}}\) | \(50.5_{\phantom{\scriptscriptstyle +0.0}}\) | \(51.8_{\phantom{\scriptscriptstyle +0.0}}\) |
| MGDA | \(57.1_{\scriptscriptstyle \textcolor[HTML]{008000}{+4.0}}\) | \(63.9_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.6}}\) | \(22.4_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.1}}\) | \(45.2_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.0}}\) | \(47.2_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.5}}\) | \(65.0_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.7}}\) | \(74.1_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.5}}\) | \(25.8_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.0}}\) | \(51.7_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.2}}\) | \(54.2_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.4}}\) |
| CAGrad | \(57.2_{\scriptscriptstyle \textcolor[HTML]{008000}{+4.1}}\) | \(63.7_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.4}}\) | \(22.9_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.6}}\) | \(45.6_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.4}}\) | \(47.3_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.6}}\) | \(63.9_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.6}}\) | \(74.4_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.8}}\) | \(26.6_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.8}}\) | \(52.1_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.6}}\) | \(54.3_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.5}}\) |
| PCGrad | \(57.0_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.9}}\) | \(63.5_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.2}}\) | \(22.4_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.1}}\) | \(44.8_{\scriptscriptstyle \textcolor[HTML]{008000}{+0.6}}\) | \(46.9_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.2}}\) | \(63.2_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.9}}\) | \(74.4_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.8}}\) | \(26.1_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.3}}\) | \(52.6_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.1}}\) | \(54.1_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.3}}\) |
| Qwen3-4B-Base | ||||||||||
| Base | \(75.5_{\phantom{\scriptscriptstyle +0.0}}\) | \(81.7_{\phantom{\scriptscriptstyle +0.0}}\) | \(39.4_{\phantom{\scriptscriptstyle +0.0}}\) | \(56.9_{\phantom{\scriptscriptstyle +0.0}}\) | \(63.4_{\phantom{\scriptscriptstyle +0.0}}\) | \(83.8_{\phantom{\scriptscriptstyle +0.0}}\) | \(93.3_{\phantom{\scriptscriptstyle +0.0}}\) | \(44.0_{\phantom{\scriptscriptstyle +0.0}}\) | \(63.6_{\phantom{\scriptscriptstyle +0.0}}\) | \(71.2_{\phantom{\scriptscriptstyle +0.0}}\) |
| V. SFT | \(69.8_{\phantom{\scriptscriptstyle +0.0}}\) | \(78.9_{\phantom{\scriptscriptstyle +0.0}}\) | \(43.4_{\phantom{\scriptscriptstyle +0.0}}\) | \(55.9_{\phantom{\scriptscriptstyle +0.0}}\) | \(62.0_{\phantom{\scriptscriptstyle +0.0}}\) | \(76.0_{\phantom{\scriptscriptstyle +0.0}}\) | \(88.8_{\phantom{\scriptscriptstyle +0.0}}\) | \(45.8_{\phantom{\scriptscriptstyle +0.0}}\) | \(62.3_{\phantom{\scriptscriptstyle +0.0}}\) | \(68.2_{\phantom{\scriptscriptstyle +0.0}}\) |
| MGDA | \(72.8_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.0}}\) | \(80.2_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.3}}\) | \(44.7_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.3}}\) | \(58.1_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.2}}\) | \(63.9_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.9}}\) | \(79.6_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.6}}\) | \(90.8_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.0}}\) | \(47.4_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.6}}\) | \(66.0_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.7}}\) | \(70.9_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.7}}\) |
| CAGrad | \(72.6_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.8}}\) | \(80.7_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.8}}\) | \(44.9_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.5}}\) | \(58.0_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.1}}\) | \(64.0_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.0}}\) | \(79.4_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.4}}\) | \(90.6_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.8}}\) | \(47.2_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.4}}\) | \(65.5_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.2}}\) | \(70.7_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.5}}\) |
| PCGrad | \(73.0_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.2}}\) | \(80.1_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.2}}\) | \(44.6_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.2}}\) | \(58.4_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.5}}\) | \(64.0_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.0}}\) | \(79.4_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.4}}\) | \(90.6_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.8}}\) | \(46.1_{\scriptscriptstyle \textcolor[HTML]{008000}{+0.3}}\) | \(64.0_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.7}}\) | \(70.0_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.8}}\) |
| Qwen3-8B-Base | ||||||||||
| Base | \(81.8_{\phantom{\scriptscriptstyle +0.0}}\) | \(85.7_{\phantom{\scriptscriptstyle +0.0}}\) | \(44.5_{\phantom{\scriptscriptstyle +0.0}}\) | \(61.7_{\phantom{\scriptscriptstyle +0.0}}\) | \(68.4_{\phantom{\scriptscriptstyle +0.0}}\) | \(85.0_{\phantom{\scriptscriptstyle +0.0}}\) | \(95.1_{\phantom{\scriptscriptstyle +0.0}}\) | \(47.6_{\phantom{\scriptscriptstyle +0.0}}\) | \(70.0_{\phantom{\scriptscriptstyle +0.0}}\) | \(74.4_{\phantom{\scriptscriptstyle +0.0}}\) |
| V. SFT | \(75.3_{\phantom{\scriptscriptstyle +0.0}}\) | \(84.8_{\phantom{\scriptscriptstyle +0.0}}\) | \(46.8_{\phantom{\scriptscriptstyle +0.0}}\) | \(63.4_{\phantom{\scriptscriptstyle +0.0}}\) | \(67.6_{\phantom{\scriptscriptstyle +0.0}}\) | \(80.6_{\phantom{\scriptscriptstyle +0.0}}\) | \(93.4_{\phantom{\scriptscriptstyle +0.0}}\) | \(49.0_{\phantom{\scriptscriptstyle +0.0}}\) | \(69.1_{\phantom{\scriptscriptstyle +0.0}}\) | \(73.0_{\phantom{\scriptscriptstyle +0.0}}\) |
| MGDA | \(77.9_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.6}}\) | \(85.3_{\scriptscriptstyle \textcolor[HTML]{008000}{+0.5}}\) | \(50.0_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.2}}\) | \(64.8_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.4}}\) | \(69.5_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.9}}\) | \(83.9_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.3}}\) | \(94.0_{\scriptscriptstyle \textcolor[HTML]{008000}{+0.6}}\) | \(51.3_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.3}}\) | \(71.0_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.9}}\) | \(75.1_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.1}}\) |
| CAGrad | \(78.4_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.1}}\) | \(85.2_{\scriptscriptstyle \textcolor[HTML]{008000}{+0.4}}\) | \(49.0_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.2}}\) | \(65.2_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.8}}\) | \(69.5_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.9}}\) | \(83.6_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.0}}\) | \(94.0_{\scriptscriptstyle \textcolor[HTML]{008000}{+0.6}}\) | \(49.8_{\scriptscriptstyle \textcolor[HTML]{008000}{+0.8}}\) | \(71.9_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.8}}\) | \(74.8_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.8}}\) |
| PCGrad | \(78.2_{\scriptscriptstyle \textcolor[HTML]{008000}{+2.9}}\) | \(85.4_{\scriptscriptstyle \textcolor[HTML]{008000}{+0.6}}\) | \(48.4_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.6}}\) | \(64.6_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.2}}\) | \(69.2_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.6}}\) | \(83.8_{\scriptscriptstyle \textcolor[HTML]{008000}{+3.2}}\) | \(94.0_{\scriptscriptstyle \textcolor[HTML]{008000}{+0.6}}\) | \(49.1_{\scriptscriptstyle \textcolor[HTML]{008000}{+0.1}}\) | \(70.3_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.2}}\) | \(74.3_{\scriptscriptstyle \textcolor[HTML]{008000}{+1.3}}\) |
4.2.0.1 Performance on Seen Languages
Table 1 demonstrates that reformulating MLF as an MOO problem systematically and comprehensively mitigates negative interference. Unlike
conventional fine-tuning, which forces compromises between languages, all Bucket-Level MOO strategies yield strictly positive improvements over Vanilla SFT across all four base models. The gains are substantial, increasing the average seen performance by
robust margins of +1.6 to +2.9. These improvements span both reading comprehension (e.g., CAGrad drives a +4.1 BELE improvement on Llama-3.1-8B) and complex technical reasoning (e.g., MGDA yields a +3.2 PolyMath boost on
Qwen3-8B-Base).
4.2.0.2 Generalization to Unseen Languages
The structural advantages of gradient-based MOO are even more pronounced under zero-shot cross-lingual transfer to the 5 unseen languages. By resolving conflicts locally, the model is prevented from severely overfitting to the dominant distributions of
the 8 seen languages. Consequently, Bucket-Level MOO achieves massive out-of-distribution average gains, reaching up to +2.5 on Llama-3.1-8B (CAGrad) and +2.7 on Qwen3-4B-Base (MGDA). These improvements showcase robust boosts in
both world knowledge (MMLU +3.7 on Qwen3-4B-Base) and reasoning (PolyMath +3.8 on Llama-3.1-8B). This confirms that mitigating gradient conflicts fosters a highly generalized cross-lingual representation space.
4.2.0.3 Mitigating Catastrophic Forgetting
Our results reveal a systematic “alignment tax” where Vanilla SFT aggressively overwrites and degrades specific base-model capabilities. This is particularly evident in the BELE benchmark, where standard fine-tuning collapses the model’s pre-trained
reading comprehension capacity. Crucially, gradient-based MOO acts as a powerful implicit regularizer that buffers this catastrophic forgetting across both seen and unseen distributions. On seen BELE, Vanilla SFT drastically drops Llama-3.1-8B
from 59.1 down to 53.1; however, Bucket-Level CAGrad successfully recovers it to 57.2. Similarly, on unseen BELE, Vanilla SFT induces a severe drop for Qwen3-8B-Base (falling from 85.0 down to 80.6), which MGDA systematically limits by
recovering the score to 83.9. Furthermore, while Vanilla SFT drags the overall average performance of inherently strong models below their pre-trained baselines (e.g., the Qwen3-8B-Base seen average drops from 68.4 to 67.6), MOO methods
completely arrest this decline and propel the models to new peaks (e.g., MGDA reaching 69.5). This structural preservation confirms that resolving gradient conflicts directly prevents the destructive overwriting of shared pre-trained representations.
4.3 Conventional MOO vs. Bucket-Level MOO↩︎
Theoretically, Bucket-Level MOO naturally generalizes conventional MOO (the trivial global partition \(\mathcal{P} = \{\boldsymbol{\theta}\}\)). However, as established in Section 3.1, applying MOO globally across an LLM introduces prohibitive memory and communication bottlenecks. We empirically validate this by post-training Qwen3-4B-Base on 8 NVIDIA H200 GPUs. Systemically,
conventional MOO requires materializing full global gradient vectors, driving peak VRAM consumption to an unsustainable 123 GB. Conversely, Bucket-Level MOO seamlessly leverages distributed memory partitioning, strictly operating within
the highly efficient 72 GB footprint of Vanilla SFT.
r0.63
| Metric | MGDA | CAGrad | PCGrad | |||
|---|---|---|---|---|---|---|
| 2-3 (lr)4-5 (lr)6-7 | w/o BK | w/ BK | w/o BK | w/ BK | w/o BK | w/ BK |
| Seen Avg. | 63.7 | 63.9 | 63.1 | 64.0 | 63.6 | 64.0 |
| Unseen Avg. | 70.6 | 70.9 | 70.0 | 70.7 | 70.8 | 70.0 |
Beyond hardware efficiency, Table 2 demonstrates that Bucket-Level MOO yields superior representation quality. Across nearly all configurations, bucket-level gradient manipulation drives stronger performance, prominently seen with CAGrad securing a +0.9 improvement on seen languages (63.1 to 64.0) and a +0.7 gain on unseen generalization (70.0 to 70.7). Similarly, Bucket-Level MGDA consistently edges out its global counterpart. This empirical advantage corroborates our core theoretical claim: by enforcing Refined Pareto Stationarity, Bucket-Level MOO natively isolates and resolves the severe, layer-wise gradient conflicts that are otherwise diluted during global aggregation, ultimately producing a more refined cross-lingual alignment.
4.4 Mechanistic Interpretation↩︎
To evaluate how Bucket-Level MOO reshapes the internal representation space, we extract the hidden states of the final input tokens3 across all 13,040 post-training samples of the 8 seen languages. We utilize the Silhouette score [25] to quantify the clustering quality of these embeddings with respect to their source languages. This metric penalizes high intra-class variance and rewards large inter-class margins, making it ideal for evaluating language separability. Furthermore, following recent interpretability studies [26]–[28], we identify critical neurons activated during language processing to calculate the ratio of language-specific neurons—those highly specialized to process a single language (details in Appendix 14).
As shown in Figure 5 (Left), Bucket-Level MOO methods induce significantly higher Silhouette scores than Vanilla SFT, particularly in the earlier or later layers of the network. This divergence indicates a much more
compact and separable clustering of languages. Furthermore, Figure 5 (Right) reveals that gradient-based MOO consistently increases the proportion of language-specific neurons across all evaluated models. Notably, inherently
stronger multilingual models (e.g., the Qwen3 family) naturally allocate a higher baseline ratio of these specialized neurons (\(\sim\)3-4%), a capacity that MOO methods further expand. This structural shift
implies that mitigating gradient conflicts actively drives the model to carve out orthogonal, language-specific dimensions. By isolating these linguistic properties rather than forcing competing features into a shared representation bottleneck, the model
disentangles the representation space. Consequently, it can safely project language-agnostic semantics into shared subspaces without incurring negative interference, ultimately yielding a more refined and capable multilingual representation.
Figure 5: How Bucket-Level MOO affects the internal structure of LLMs. Left: Layer-wise Silhouette scores measuring the representational clustering of languages in Qwen3-8B-Base. Right: The
percentage of language-specific neurons (neurons activated solely by a single language) across different model architectures..
5 Related Work↩︎
5.0.0.1 Multilingual LLMs and Negative Interference
Modern LLMs project diverse languages into a shared parameter space [27], [29], where these languages occupy distinct representational regions [30], [31]. Structurally, models often exhibit a division of labor: early and late layers handle multilingual specializations, while middle layers serve as language-universal mechanisms [12], [13]. While targeted activation interventions improve reasoning [32], cross-lingual generalization is predominantly driven by data-centric strategies like translating instruction corpora [2], [3], self-distillation [33], and multi-task tuning [1]. However, forcing disparate linguistic distributions into a static parameter space inherently triggers negative interference, where optimizing for one language disproportionately degrades others [4], [5].
5.0.0.2 Gradient Conflicts and Scalable Optimization in Distributed LLMs
From an optimization perspective, negative interference fundamentally stems from conflicting gradients across language-specific objectives [8], [9]. Beyond architectural modifications [34] or multi-stage fine-tuning [35], gradient-based Multi-Objective Optimization (MOO)—such as MGDA [6], [18], PCGrad [7], and CAGrad [11]—offers a principled solution.
MOO mitigates conflicts by guaranteeing a common descent direction without altering the underlying model architecture [9],
[10]. However, applying global MOO to large-scale LLMs is severely bottlenecked by distributed training frameworks like FSDP [15] and DeepSpeed ZeRO [14]. Because these paradigms partition gradients across GPUs to conserve memory, reconstructing the full global gradient vector at every step introduces prohibitive
All-Gather communication overhead, rendering standard global MOO computationally unscalable.
6 Conclusion↩︎
In this work, we addressed the pervasive issue of negative interference in multilingual LLM fine-tuning by reformulating it as an MOO problem. Recognizing that global gradient aggregation both obscures layer-specific linguistic conflicts and introduces prohibitive communication overhead in distributed training, we introduced Bucket-Level MOO. By intercepting and resolving gradient conflicts locally within parameter memory buckets, our framework bypasses the scalability bottlenecks of conventional algorithms like MGDA, CAGrad, and PCGrad to modern architectures with billions of parameters.
Theoretically, we prove this method natively enforces Refined Pareto Stationarity, a strictly tighter necessary condition for Pareto optimality. Empirically, by disentangling language-specific dimensions, Bucket-Level MOO curbs negative interference and drives robust cross-lingual generalization across both seen and unseen distributions. Ultimately, this work bridges the gap between theoretical optimization and practical systems engineering, demonstrating that structural awareness of distributed training paradigms can unlock highly scalable and interference-free multilingual alignment.
7 Formal Definitions and Theorems for Related MOO Concepts↩︎
Definition 1 (Pareto Optimality). A solution \(\boldsymbol{\theta}^*\) is termed Pareto optimal if there does not exist any \(\boldsymbol{\theta}\) such that \(\mathcal{L}_t(\boldsymbol{\theta}) \leq \mathcal{L}_t(\boldsymbol{\theta}^*)\) for all \(t \in \{1, \dots, T\}\), and \(\mathbf{L}(\boldsymbol{\theta}) \neq \mathbf{L}(\boldsymbol{\theta}^*)\).
Definition 2 (Pareto Stationarity). A solution \(\boldsymbol{\theta}^*\) is Pareto stationary if and only if: \[\begin{align} \mathbf{0} \in \text{conv}\left\{ \nabla_{\boldsymbol{\theta}} \mathcal{L}_1(\boldsymbol{\theta}^*), \dots, \nabla_{\boldsymbol{\theta}}\mathcal{L}_T(\boldsymbol{\theta}^*)\right\}, \end{align}\] meaning there exists a vector \(\boldsymbol{\alpha} \in \Delta\) such that \(\sum_{t=1}^T \alpha_t\nabla_{\boldsymbol{\theta}}\mathcal{L}_t(\boldsymbol{\theta}^*)=\mathbf{0}\).
Here, \(\Delta=\left\{ \boldsymbol{\alpha} \in \mathbb{R}^T \mid \boldsymbol{\alpha} \geq \mathbf{0}, \sum_{t=1}^T \alpha_t = 1\right\}\) denotes the standard simplex in \(T\)-dimensional space.
7.0.0.1 Refined Pareto Stationarity.
While standard stationarity treats \(\boldsymbol{\theta}\) globally, exploiting variable structure yields tighter optimality criteria [16]. Assume \(\boldsymbol{\theta}\) is partitioned into \(K\) disjoint blocks \(\mathcal{P} = \left\{ \boldsymbol{\theta}_{(1)},\dots, \boldsymbol{\theta}_{(K)}\right\}\) (e.g., layers). Let \(\nabla_{\boldsymbol{\theta}_{(k)}}\mathbf{L}(\boldsymbol{\theta})\) be the collection of objective gradients with respect to block \(\boldsymbol{\theta}_{(k)}\).
Definition 3 (Refined Pareto Stationarity). A solution \(\boldsymbol{\theta}^*\) is refined Pareto stationary with respect to partition \(\mathcal{P}\) if and only if: \[\begin{align} \mathbf{0} \in \prod_{k=1}^{K} \text{conv}\left\{ \nabla_{\boldsymbol{\theta}_{(k)}}\mathbf{L}(\boldsymbol{\theta}^*)\right\} \quad \text{(Cartesian product)}, \end{align}\] i.e., for all blocks \(k \in [K]\), there exists a local weight \(\boldsymbol{\alpha}^k \in \Delta\) such that \(\sum_{t=1}^T \alpha^k_t\nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_t(\boldsymbol{\theta}^*)=\mathbf{0}\).
For the trivial partition \(\mathcal{P} = \{\boldsymbol{\theta}\}\), this reduces to standard Pareto stationarity.
Theorem 3 ([16]). Any Pareto optimal solution is a refined Pareto stationary solution. Furthermore, any refined Pareto stationary solution is a Pareto stationary solution.
Definition 4 (Conflicting Gradients). Two gradients \(\nabla_{\boldsymbol{\theta}}\mathcal{L}_i\) and \(\nabla_{\boldsymbol{\theta}}\mathcal{L}_j\) are defined as conflicting if their cosine similarity is negative, i.e., \(\cos\left(\nabla_{\boldsymbol{\theta}}\mathcal{L}_i, \nabla_{\boldsymbol{\theta}}\mathcal{L}_j\right) < 0\), or equivalently, \(\langle \nabla_{\boldsymbol{\theta}}\mathcal{L}_i, \nabla_{\boldsymbol{\theta}}\mathcal{L}_j \rangle < 0\).
Theorem 4 ([18]). Let \(\alpha_1,\dots,\alpha_T\) be the solution to Problem 2 , and let \(\mathbf{d}^{\text{MGDA}} = \sum_{t=1}^T\alpha_t \nabla_{\boldsymbol{\theta}}\mathcal{L}_t\) be the resulting update vector. Then, one of the following is true:
\(\|\mathbf{d}^{\text{MGDA}}\|_2=0\) and the current parameters are Pareto stationary.
\(-\mathbf{d}^{\text{MGDA}}\) is a descent direction that decreases all objectives.
Theorem 5 ([11]). Assume the loss functions \(\mathcal{L}_t\) are differentiable and have Lipschitz continuous gradients. If the step size \(\eta\) is sufficiently small, the CAGrad update direction \(\mathbf{d}^{\text{CAGrad}}\) ensures monotonic decrease of the average loss \(\mathcal{L}_{avg} = \frac{1}{T}\sum_{t=1}^T \mathcal{L}_t\). Specifically, the algorithm converges to a Pareto stationary point \(\boldsymbol{\theta}^*\) such that \(\|\nabla \mathcal{L}_{avg}(\boldsymbol{\theta}^*)\| = 0\).
8 Proof of Theorem 1↩︎
In this section, we provide the formal proof that applying Multiple Gradient Descent Algorithm (MGDA) at the shard level guarantees a global descent direction that satisfies the stricter condition of Refined Pareto Stationarity.
Theorem 1 Let the model parameters \(\boldsymbol{\theta}\) be partitioned into \(K\) disjoint buckets \(\{\boldsymbol{\theta}_{(k)}\}_{k=1}^K\). Let \(\boldsymbol{\alpha}^{(k)}\) be the solution to the local MGDA problem on bucket \(k\): \[\begin{align} \min_{\boldsymbol{\alpha}^{(k)} \in \Delta} \left\| \sum_{t=1}^T \alpha^{(k)}_t \nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_t(\boldsymbol{\theta}) \right\|_2^2, \end{align}\] and let \(\mathbf{d}^{\text{MGDA}}_{(k)} = \sum_{t=1}^T \alpha^{(k)}_t \nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_t(\boldsymbol{\theta})\) be the resulting local update direction. Define the global update direction as the concatenation \(\mathbf{d}^{\text{MGDA}} = [\mathbf{d}^{\text{MGDA}}_{(1)}, \dots, \mathbf{d}^{\text{MGDA}}_{(K)}]\). Then:
If \(\mathbf{d}^{\text{MGDA}} = \mathbf{0}\), the solution is Refined Pareto Stationary.
If \(\mathbf{d}^{\text{MGDA}} \neq \mathbf{0}\), then \(-\mathbf{d}^{\text{MGDA}}\) is a common descent direction for all objectives, i.e., \(\langle \nabla_{\boldsymbol{\theta}}\mathcal{L}_t, \mathbf{d}^{\text{MGDA}} \rangle \geq 0\) for all \(t=1,\dots,T\).
Proof. The proof relies on the decomposition of the global inner product over the parameter buckets and the properties of the minimum-norm element in a convex hull.
8.0.0.1 1. Decomposition of Directional Derivatives.
Consider the directional derivative of the loss \(\mathcal{L}_t\) with respect to the global parameter vector \(\boldsymbol{\theta}\) along the direction \(\mathbf{d}^{\text{MGDA}}\). Since the parameter space is the Cartesian product of the buckets, the inner product decomposes additively: \[\begin{align} \langle \nabla_{\boldsymbol{\theta}} \mathcal{L}_t, \mathbf{d}^{\text{MGDA}} \rangle &= \sum_{k=1}^K \langle \nabla_{\boldsymbol{\theta}_{(k)}} \mathcal{L}_t, \mathbf{d}^{\text{MGDA}}_{(k)} \rangle. \label{eq:decomp} \end{align}\tag{6}\]
8.0.0.2 2. Local Descent Property.
On each bucket \(k\), the update direction \(\mathbf{d}^{\text{MGDA}}_{(k)}\) is derived from the minimum-norm point within the convex hull of the local gradients. According to the properties of the minimum-norm point in convex geometry [18], the following inequality holds for every constituent vector in the convex hull: \[\begin{align} \forall t \in \{1, \dots, T\}: \quad \langle \nabla_{\boldsymbol{\theta}_{(k)}} \mathcal{L}_t, \mathbf{d}^{\text{MGDA}}_{(k)} \rangle \geq \|\mathbf{d}^{\text{MGDA}}_{(k)}\|_2^2. \label{eq:local95prop} \end{align}\tag{7}\] This inequality guarantees that for a specific bucket, the MGDA direction is a descent direction for all tasks simultaneously, unless the bucket is already at a stationary point (in which case \(\|\mathbf{d}^{\text{MGDA}}_{(k)}\|=0\)).
8.0.0.3 3. Aggregation.
Substituting the local inequality (7 ) into the global decomposition (6 ): \[\begin{align} \langle \nabla_{\boldsymbol{\theta}} \mathcal{L}_t, \mathbf{d}^{\text{MGDA}} \rangle &= \sum_{k=1}^K \langle \nabla_{\boldsymbol{\theta}_{(k)}} \mathcal{L}_t, \mathbf{d}^{\text{MGDA}}_{(k)} \rangle \geq \sum_{k=1}^K \|\mathbf{d}^{\text{MGDA}}_{(k)}\|_2^2 = \|\mathbf{d}^{\text{MGDA}}\|_2^2. \label{eq:global95bound} \end{align}\tag{8}\] Thus, for every task \(t\), the directional derivative is upper-bounded by the negative squared norm of the global update vector: \[\begin{align} \langle \nabla_{\boldsymbol{\theta}} \mathcal{L}_t, \mathbf{d}^{\text{MGDA}} \rangle \geq \|\mathbf{d}^{\text{MGDA}}\|_2^2. \end{align}\]
8.0.0.4 4. Case Analysis.
We now analyze the two possible cases for the value of \(\|\mathbf{d}^{\text{MGDA}}\|_2^2\):
Case (a): \(\mathbf{d}^{\text{MGDA}} = \mathbf{0}\).
If the global update vector is zero, then \(\|\mathbf{d}^{\text{MGDA}}\|_2^2 = \sum_{k=1}^K \|\mathbf{d}^{\text{MGDA}}_{(k)}\|_2^2 = 0\). Since norms are non-negative, this implies \(\|\mathbf{d}^{\text{MGDA}}_{(k)}\|_2 = 0\) for all \(k=1,\dots,K\). By the definition of MGDA, \(\|\mathbf{d}^{\text{MGDA}}_{(k)}\|_2 = 0\) implies that \(\mathbf{0}\) lies within the convex hull of the local gradients for shard \(k\): \[\begin{align} \mathbf{0} \in \text{conv}\left\{
\nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_1, \dots, \nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_T \right\} \quad \forall k.
\end{align}\] Therefore: \[\begin{align} \mathbf{0} \in \prod_{k=1}^K \text{conv}\left\{ \nabla_{\boldsymbol{\theta}_{(k)}}\mathbf{L}(\boldsymbol{\theta}) \right\}.
\end{align}\] This is precisely the definition of Refined Pareto Stationarity with respect to the partition \(\mathcal{P} = \{\boldsymbol{\theta}_{(1)}, \dots, \boldsymbol{\theta}_{(K)}\}\).
Case (b): \(\mathbf{d}^{\text{MGDA}} \neq \mathbf{0}\).
If the update vector is non-zero, then \(\|\mathbf{d}^{\text{MGDA}}\|_2^2 > 0\). From inequality (8 ), we have: \[\begin{align} \langle
\nabla_{\boldsymbol{\theta}} \mathcal{L}_t, \mathbf{d}^{\text{MGDA}} \rangle \geq \|\mathbf{d}^{\text{MGDA}}\|_2^2 > 0 \quad \forall t.
\end{align}\] Since the directional derivative is strictly negative for all \(t\), \(-\mathbf{d}^{\text{MGDA}}\) is a valid descent direction that simultaneously decreases the loss
for all languages. ◻
9 Proof of Theorem 2↩︎
In this section, we provide the formal proof that applying Conflict-Averse Gradient Descent (CAGrad) at the bucket level guarantees a global descent direction for the average loss function, satisfying the condition of Refined Pareto Stationarity under convergence.
Theorem 2. Let the model parameters \(\boldsymbol{\theta}\) be partitioned into \(K\) disjoint buckets \(\{\boldsymbol{\theta}_{(k)}\}_{k=1}^K\). Let \(\mathbf{g}^{avg}_{(k)} = \frac{1}{T}\sum_{t=1}^T \nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_t\) denote the local average gradient on bucket \(k\). Let \(\mathbf{d}^{\text{CAGrad}}_{(k)}\) be the solution to the local CAGrad optimization problem: \[\begin{align} \mathbf{d}^{\text{CAGrad}}_{(k)} = \mathop{\mathrm{arg\,max}}_{\mathbf{d}_{(k)}} \min_{t} \langle \nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_t, \mathbf{d}_{(k)} \rangle \quad \text{s.t.} \quad \|\mathbf{d}_{(k)} - \mathbf{g}^{avg}_{(k)}\| \leq c \|\mathbf{g}^{avg}_{(k)}\|, \label{eq:local95cagrad95opt} \end{align}\tag{9}\] where \(0 \leq c < 1\). Define the global update direction as \(\mathbf{d}^{\text{CAGrad}} = [\mathbf{d}^{\text{CAGrad}}_{(1)}, \dots, \mathbf{d}^{\text{CAGrad}}_{(K)}]\). Then:
The global update \(-\mathbf{d}^{\text{CAGrad}}\) is a descent direction for the global average loss \(\mathcal{L}_{avg}(\boldsymbol{\theta}) = \frac{1}{T}\sum \mathcal{L}_t(\boldsymbol{\theta})\), satisfying: \[\begin{align} \langle \nabla_{\boldsymbol{\theta}} \mathcal{L}_{avg}, \mathbf{d} \rangle \geq \frac{1-c^2}{2} \|\nabla_{\boldsymbol{\theta}} \mathcal{L}_{avg}\|_2^2. \end{align}\]
If \(\|\mathbf{d}^{\text{CAGrad}}\|_2 = 0\), the current parameters satisfy Refined Pareto Stationarity with uniform weights (i.e., \(\mathbf{g}^{avg}_{(k)} = \mathbf{0}\) for all \(k\)).
Proof. The proof relies on the decomposition of the global inner product over disjoint parameter shards and the geometric properties of the Euclidean ball constraint used in CAGrad.
9.0.0.1 1. Decomposition of the Average Gradient.
Let \(\mathbf{g}^{avg} = \nabla_{\boldsymbol{\theta}} \mathcal{L}_{avg}(\boldsymbol{\theta})\) be the global average gradient. Since the parameter space is the Cartesian product of the shards, the global gradient is the concatenation of local average gradients: \(\mathbf{g}^{avg} = [(\mathbf{g}^{avg}_{(1)}), \dots, (\mathbf{g}^{avg}_{(k)})]\). Consequently, the inner product between the global average gradient and the global update direction \(\mathbf{d}\) decomposes additively: \[\begin{align} \langle \nabla_{\boldsymbol{\theta}} \mathcal{L}_{avg}, \mathbf{d}^{\text{CAGrad}} \rangle = \sum_{k=1}^K \langle \mathbf{g}^{avg}_{(k)}, \mathbf{d}^{\text{CAGrad}}_{(k)} \rangle. \label{eq:cagrad95decomp} \end{align}\tag{10}\]
9.0.0.2 2. Local Descent Property.
Consider the optimization problem on a specific bucket \(k\). The constraint requires the update direction \(\mathbf{d}^{\text{CAGrad}}_{(k)}\) to lie within a ball of radius \(c\|\mathbf{g}^{avg}_{(k)}\|\) centered at \(\mathbf{g}^{avg}_{(k)}\): \[\begin{align} \|\mathbf{d}^{\text{CAGrad}}_{(k)} - \mathbf{g}^{avg}_{(k)}\|^2 \leq c^2 \|\mathbf{g}^{avg}_{(k)}\|^2. \end{align}\] Expanding the squared norm on the left-hand side: \[\begin{align} \|\mathbf{d}^{\text{CAGrad}}_{(k)}\|^2 - 2\langle \mathbf{d}^{\text{CAGrad}}_{(k)}, \mathbf{g}^{avg}_{(k)} \rangle + \|\mathbf{g}^{avg}_{(k)}\|^2 \leq c^2 \|\mathbf{g}^{avg}_{(k)}\|^2. \end{align}\] Rearranging terms to isolate the inner product \(\langle \mathbf{d}^{\text{CAGrad}}_{(k)}, \mathbf{g}^{avg}_{(k)} \rangle\): \[\begin{align} 2\langle \mathbf{d}^{\text{CAGrad}}_{(k)}, \mathbf{g}^{avg}_{(k)} \rangle &\geq \|\mathbf{d}^{\text{CAGrad}}_{(k)}\|^2 + (1-c^2)\|\mathbf{g}^{avg}_{(k)}\|^2. \end{align}\] Since \(\|\mathbf{d}^{\text{CAGrad}}_{(k)}\|^2 \geq 0\), we can lower bound the inner product as: \[\begin{align} \langle \mathbf{d}^{\text{CAGrad}}_{(k)}, \mathbf{g}^{avg}_{(k)} \rangle \geq \frac{1-c^2}{2} \|\mathbf{g}^{avg}_{(k)}\|^2. \label{eq:local95cagrad95bound} \end{align}\tag{11}\] This inequality ensures that as long as \(c < 1\) and the local gradient is non-zero, the update direction \(\mathbf{d}^{\text{CAGrad}}_{(k)}\) always has a positive projection onto the local average gradient \(\mathbf{g}^{avg}_{(k)}\).
9.0.0.3 3. Global Aggregation (Proof of Part a).
Substituting the local bound (11 ) into the global decomposition (10 ): \[\begin{align} \langle \nabla_{\boldsymbol{\theta}} \mathcal{L}_{avg}, \mathbf{d}^{\text{CAGrad}} \rangle &= \sum_{k=1}^K \langle \mathbf{g}^{avg}_{(k)}, \mathbf{d}^{\text{CAGrad}}_{(k)} \rangle \\ & \geq \sum_{k=1}^K \frac{1-c^2}{2} \|\mathbf{g}^{avg}_{(k)}\|^2 = \frac{1-c^2}{2} \sum_{k=1}^K \|\mathbf{g}^{avg}_{(k)}\|^2 = \frac{1-c^2}{2} \|\mathbf{g}^{avg}\|_2^2. \end{align}\] Given that \(0 \le c < 1\), the term \(\frac{1-c^2}{2}\) is strictly positive. Thus, \(-\mathbf{d}\) is a valid descent direction for the average loss.
9.0.0.4 4. Stationarity (Proof of Part b).
Assume the global update vector is zero, i.e., \(\|\mathbf{d}^{\text{CAGrad}}\|_2 = 0\). This implies \(\|\mathbf{d}^{\text{CAGrad}}_{(k)}\|_2 = 0\) for all buckets \(k=1,\dots,K\). Substituting \(\mathbf{d}^{\text{CAGrad}}_{(k)} = \mathbf{0}\) into the local constraint inequality: \[\begin{align} \|\mathbf{0} - \mathbf{g}^{avg}_{(k)}\| &\leq c \|\mathbf{g}^{avg}_{(k)}\| \\ \|\mathbf{g}^{avg}_{(k)}\| &\leq c \|\mathbf{g}^{avg}_{(k)}\| \\ (1-c) \|\mathbf{g}^{avg}_{(k)}\| &\leq 0. \end{align}\] Since \(c < 1\), the term \((1-c)\) is strictly positive. Therefore, the only solution to this inequality is \(\|\mathbf{g}^{avg}_{(k)}\| = 0\), which implies: \[\begin{align} \mathbf{g}^{avg}_{(k)} = \sum_{t=1}^T \frac{1}{T} \nabla_{\boldsymbol{\theta}_{(k)}}\mathcal{L}_t = \mathbf{0}. \end{align}\] This holds for all \(k\). In the context of Refined Pareto Stationarity, this corresponds to the specific case where the weighting vector \(\boldsymbol{\alpha}^k\) is uniform (\(\alpha_t = 1/T\)) for all buckets. Thus, the parameters satisfy the condition: \[\begin{align} \mathbf{0} \in \prod_{k=1}^{K} \text{conv}\left\{ \nabla_{\boldsymbol{\theta}_{(k)}}\mathbf{L}(\boldsymbol{\theta})\right\}, \end{align}\] confirming Refined Pareto Stationarity. ◻
10 Algorithm: Bucket-Level Multi-Objective Optimization for Distributed LLM Fine-Tuning↩︎
The core innovation of Algorithm 6 lies in intercepting the standard distributed backward pass to resolve cross-lingual gradient conflicts locally within memory buckets. By doing so, it completely bypasses the need to materialize the full global gradient vector, preserving the memory and communication efficiency of distributed training. The execution flow proceeds as follows:
Language-Specific Forward Pass: Training begins with \(T\) distributed GPUs, each exclusively assigned to process a micro-batch from a distinct language dataset \(\mathcal{D}_t\). During the forward pass, each GPU computes a language-specific loss \(\mathcal{L}_t(\boldsymbol{\theta})\).
Bucket-Level Backward Interception: During the backward pass, gradients are not immediately accumulated globally. Instead, the algorithm intercepts the gradients sequentially as fixed-size parameter memory buckets (\(k = 1, 2, \dots, K\)) are populated.
Localized Synchronization (
All-Gather): The moment bucket \(k\) is filled with local gradients, anAll-Gatheroperation is triggered. This synchronizes the bucket across the distributed group, providing every GPU with the complete set of language-specific gradients for that specific bucket, denoted as \(G_{(k)} = \{g_{(k), 1}, \dots, g_{(k), T}\}\).In-Place Conflict Resolution: With the local gradients gathered, each GPU independently applies the selected MOO strategy (MGDA, CAGrad, or PCGrad) strictly to the parameters within bucket \(k\). This mathematical resolution yields a single, conflict-free update direction \(\mathbf{d}_{(k)}\) that serves the competing language objectives simultaneously.
ZeRO-2 Partitioning and Memory Recovery: To strictly adhere to ZeRO-2 memory partitioning, the algorithm immediately performs a
Reduce-Scatteron the newly resolved update \(\mathbf{d}_{(k)}\). Consequently, each GPU retains only its explicitly assigned partition of the bucket. Crucially, the full multi-language gradient set \(G_{(k)}\) is then flushed from GPU memory to prevent Out-Of-Memory (OOM) failures.Global Optimizer Step: Once all \(K\) buckets have been sequentially intercepted, resolved, scattered, and freed, each GPU holds a safely partitioned, conflict-resolved gradient vector for its assigned parameters. The optimizer then executes a standard step to update the local parameter partitions (\(\boldsymbol{\theta} \gets \boldsymbol{\theta} - \eta \mathbf{d}\)).
11 Experiment Details↩︎
All fine-tuning experiments are implemented using the TRL4 library with a maximum sequence length of 2,048 tokens, and batch size 128. Training is conducted
in bfloat16 precision using the AdamW optimizer. We apply a weight decay of 0.1, a cosine learning rate schedule with a warmup ratio of \(0.03\). We applied a weight decay of 0.1 and a peak learning rate of
2e-5 for the Qwen 3 models, mirroring the 0.04 and 4e-6 used for Llama 3. Models are trained for 3 epochs during the post-training phase. All experiments were executed on a single compute node equipped with 8 NVIDIA H200 GPUs. Furthermore, for Bucket-Level
CAGrad, the conflict aversion hyperparameter \(c \in [0, 1)\) is set to 0.5 across all applicable runs.
12 Training Time↩︎
| Method | V. SFT | MGDA | CAGrad | PCGrad |
|---|---|---|---|---|
| Run-time | \(0.6\) | \(0.8\) | \(0.7\) | \(0.8\) |
Table 3 compares the computational cost of Vanilla SFT against the gradient-based MOO methods on 8 H200 GPUs. In the Post-Training phase, the absolute time increase is minimal: MGDA and PCGrad require 0.8 hours compared to the baseline’s 0.6 hours, while CAGrad introduces minimal time overhead (0.7 hours). These results demonstrate that the Bucket-Level MOO framework maintains high computational efficiency, incurring only a modest time tradeoff in exchange for the significant gains achieved in multilingual and cross-lingual performance.
13 Multilingual Representation Analysis↩︎
Figure 7: Layer-wise Silhouette scores measuring the representational clustering of languages..
In this section, we provide the extended layer-wise Silhouette score analysis for the remaining base models evaluated in our study: Meta-Llama-3-8B, Llama-3.1-8B, and Qwen3-4B-Base. These results expand upon the representational clustering findings discussed in Section 4.4.
Consistent with the behavior observed in Qwen3-8B-Base, all three architectures exhibit a distinct, wave-like clustering trajectory across their depth. The internal representations are relatively segregated by language in the earliest layers, become heavily entangled (yielding near-zero Silhouette scores) in the middle computational layers where shared semantic reasoning likely occurs, and finally separate into highly distinct, language-specific clusters in the terminal layers.
Crucially, these plots confirm that Bucket-Level MOO methods (MGDA, CAGrad, and PCGrad) consistently enhance language separability over Vanilla SFT across all architectures. This divergence is particularly pronounced in the final third of the network. For instance, in Llama-3.1-8B, the MOO methods open a substantial gap over the baseline in the final 10 layers.
Furthermore, while the Qwen3 models naturally exhibit stronger baseline multilingual capabilities—evidenced by their higher peak Silhouette scores (approaching 0.60 compared to Llama’s 0.45)—our localized conflict resolution still consistently elevates their clustering quality. This demonstrates that mitigating gradient conflicts universally aids in carving out orthogonal, language-specific representation dimensions, regardless of the base model’s inherent multilingual capacity.
14 Language-Critical Neurons↩︎
Drawing on recent interpretability and network pruning methodologies [26]–[28], we quantify the degree of language specialization within the model by identifying critical neurons. Let \(\theta_i\) denote an individual neuron (e.g., a specific row or column within a weight matrix). To evaluate its functional importance for a given language \(t \in \mathcal{T}\), we approximate the expected change in the loss function \(\mathcal{L}\) over a language-specific dataset \(\mathcal{D}_t\) if \(\theta_i\) were to be deactivated (i.e., pruned to 0). Using a first-order Taylor expansion, this impact score \(\mathcal{I}_t(\theta_i)\) can be efficiently estimated as: \[\begin{align} \mathcal{I}_t(\theta_i) = \left|\mathcal{L}(\theta_i, \mathcal{D}_t) - \mathcal{L}(0, \mathcal{D}_t)\right| \approx \left|\theta_i \nabla_{\theta_i}\mathcal{L}(\theta_i, \mathcal{D}_t)\right| \end{align}\]
Let \(\mathcal{N}_t\) denote the set of the top-\(p\) most critical neurons for language \(t\) based on their impact score \(\mathcal{I}_t(\theta_i)\). In our analysis, we set the threshold to \(p=1\%\) and evaluate across the language set \(\mathcal{T} = \{\text{en, zh, it, ar, ko, id, bn, sw}\}\). From these language-specific sets, we define the following structural partitions:
Activated Neurons (\(\mathcal{N}_{\text{all}}\)): The union of all critical neurons across the evaluated languages, \(\mathcal{N}_{\text{all}} = \bigcup_{t \in \mathcal{T}}\mathcal{N}_t\).
Language-Shared Neurons (\(\mathcal{N}_{\text{shared}}\)): The intersection of critical neurons deemed essential to every language, \(\mathcal{N}_{\text{shared}} = \bigcap_{t \in \mathcal{T}}\mathcal{N}_t\).
Total Neurons (\(\mathcal{N}_{\text{total}}\)): The complete set of neurons (weight columns) within the evaluated model layers.
To evaluate the structural disentanglement of the model, we calculate the ratio of strictly language-specific neurons to the total network capacity. This ratio is defined as: \[\begin{align} R_{\text{specific}} = \frac{|\mathcal{N}_{\text{all}} \setminus \mathcal{N}_{\text{shared}}|}{|\mathcal{N}_{\text{total}}|} \end{align}\]
A higher ratio indicates that the model constructs highly orthogonal, language-specific dimensions, effectively isolating linguistic representations to mitigate negative interference. Conversely, a lower ratio indicates a highly shared representation space. While extensive parameter sharing is widely recognized as beneficial for transferring universal semantics and enabling cross-lingual generalization, it simultaneously acts as a structural bottleneck. When fine-tuning across diverse languages, forcing competing linguistic constraints into this shared capacity frequently precipitates the gradient conflicts and negative interference observed in standard training paradigms.
References↩︎
Our code is publicly available at https://github.com/iNLP-Lab/BK-MOO.↩︎
We filter s1k-1.1 to include only samples marked as correctly distilled from DeepSeek-R1.↩︎
For instance, the
‘<|im_start|>assistant\n’token inQwenmodels.↩︎