1 Introduction↩︎

Large language models (LLMs) increasingly serve multilingual users, yet often produce inconsistent reasoning and answers for semantically equivalent prompts in different languages [1], [2]. Recent work studying the internal representations of multilingual models [3]–[5] reveals a consistent mechanism: intermediate Transformer layers encode largely language-agnostic semantics, while final layers become language-specialized as the model selects discrete tokens in the target language’s script (e.g., Chinese characters or Thai glyphs) for generation. This late-stage divergence from shared semantics to language-specific surface forms is hypothesized to contribute to inconsistent reasoning across languages, even when intermediate representations may have converged on the same solution.

The transition from final-layer representations to discrete tokens can act as a bottleneck, collapsing probability mass over semantically related tokens into a single language-specific choice. Continuous-space reasoning offers a way to soften this collapse: soft thinking [6] replaces discrete next-token choices with probability-weighted mixtures over vocabulary embeddings. The resulting soft tokens aggregate information across the vocabulary and are fed back as input embeddings for the next autoregressive step. Because soft tokens are not tied to a single token or language, they may retain cross-lingual semantic content that discrete token selection discards. However, prior soft-thinking methods use soft tokens mainly as an inference-time generation mechanism. They soften discrete decoding decisions but do not directly encourage semantically equivalent reasoning paths in different languages to stay aligned. This gap motivates our hypothesis: soft-token representations can provide a better training-time signal for cross-lingual alignment than discrete tokens.

We propose SOLAR (Soft Token Alignment for Cross-Lingual Reasoning), a lightweight auxiliary fine-tuning objective that aligns soft-token representations between English and non-English parallel reasoning traces during supervised fine-tuning (SFT) (Figure 1). For each reasoning trace and final response, SOLAR summarizes the model’s next-token distribution as an expectation over the embedding matrix. It then mean-pools across positions to obtain a single continuous vector that is independent of any individual token identity. Using English as a pivot, SOLAR minimizes cosine distance between the English representation and each non-English counterpart, drawing all languages toward an English-anchored semantic space. Because soft tokens operate in the shared embedding space, they enable direct comparison across languages regardless of surface-form differences.

Our experiments show that SOLAR improves both task accuracy and cross-lingual consistency across four multilingual reasoning benchmarks. For Qwen3-4B [7], SOLAR improves accuracy by up to +3.8 over SFT and cross-lingual consistency by up to +4.5 on AIME 2024, with the largest gains on low-resource languages such as Swahili. Representation analysis shows that soft tokens carry a stronger final-layer cross-lingual signal than discrete tokens on base models, and that SOLAR produces substantially stronger final-layer cross-lingual alignment than SFT and inference-time soft thinking alone. At the same time, behavioral analysis shows that these gains preserve native-language reasoning: on MGSM, SOLAR-tuned Qwen3-4B reasons in the target language’s script at 98.13% (vs.% for the base), and SOLAR-tuned Qwen3-8B at 98.16%. This reverses the base 8B model’s collapse to English on non-Chinese languages (Ja/Th/Te below 5%; 27.43% four-language average). To our knowledge, SOLAR is the first method to leverage soft tokens, originally introduced for inference-time continuous reasoning, in a cross-lingual alignment objective during post-training.

2 SOLAR: soft token alignment for cross-lingual reasoning↩︎

In this section, we first review discrete and soft thinking as two paradigms for constructing input representations during autoregressive reasoning (§2.1), then describe how SOLAR repurposes soft-token embeddings as a cross-lingual alignment signal during SFT (§2.2).

2.1 Background: discrete vs. soft thinking↩︎

In Chain-of-Thought (CoT) [8], which we call Discrete Thinking, reasoning proceeds by autoregressively generating discrete tokens. Let \(x_{1:n}\) denote the input prompt tokens and \(\hat{x}_{n+1:t-1}\) the previously generated reasoning tokens. Let \(\mathbf{E}[\cdot]\) denote the token embedding lookup function, with embeddings \(\mathbf{e}_i=\mathbf{E}[x_i]\) for \(i\le n\) and \(\mathbf{e}_i=\mathbf{E}[\hat{x}_i]\) for \(i>n\). The next token at step \(t\) is decoded from \(p_\theta(\cdot\mid \mathbf{e}_{1:t-1})\) (e.g., greedy or sampling): \[\hat{x}_t \sim p_\theta(\cdot \mid \mathbf{e}_{1:t-1}), \qquad \mathbf{e}^{\,\text{discrete}}_t = \mathbf{E}[\hat{x}_t]. \label{eq:discrete}\tag{1}\]

For a language model \(f_\theta\) with vocabulary \(V\) and token embedding matrix \(\mathbf{E} \in \mathbb{R}^{|V| \times d}\), the logit vector \(\mathbf{z}_t = f_\theta(\mathbf{e}_{1:t-1}) \in \mathbb{R}^{|V|}\) induces a distribution \(p_\theta(\cdot \mid \mathbf{e}_{1:t-1})=\operatorname{softmax}(\mathbf{z}_t / \tau)\), where \(\tau\) is the temperature. Eq. 1 thus constitutes a hard projection from the distribution over the vocabulary \(V\) to the embedding of a single discrete token \(\hat{x}_t \in V\). That is, regardless of how probability mass is distributed, only \(\mathbf{E}[\hat{x}_t]\) is fed as the input representation for the next step.

Soft Thinking [6] is an inference-time, autoregressive method that replaces this hard discrete projection during reasoning with a continuous representation formed from the next-token distribution. At step \(t\), it defines a soft-token embedding as the expectation over token embeddings \(\mathbf{E}[v]\) for \(v \in V\) under a truncated-and-renormalized distribution: \[\mathbf{e}^{\,\text{soft}}_t = \sum_{v \in V} \tilde{p}_\theta(v \mid \mathbf{e}_{1:t-1}) \cdot \mathbf{E}[v], \label{eq:soft-embed}\tag{2}\] where \(\tilde{p}_\theta\) is constructed by applying top-\(k\) filtering to the original distribution \(p_\theta(v \mid \mathbf{e}_{1:t-1})=\left[\operatorname{softmax}\!\left(\mathbf{z}_t/\tau\right)\right]_v\). Specifically, let \(F_{t} \subseteq V\) be the token set obtained by retaining the \(k\) highest-probability tokens. The filtered distribution is \[\tilde{p}_\theta(v \mid \mathbf{e}_{1:t-1})=\frac{p_\theta(v \mid \mathbf{e}_{1:t-1})\mathbf{1}[v \in F_{t}]}{\sum_{v' \in F_{t}}p_\theta(v' \mid \mathbf{e}_{1:t-1})}. \label{eq:filtered-prob}\tag{3}\] The resulting \(\mathbf{e}^{\,\text{soft}}_t\) corresponds to no single discrete token and is fed back as the input embedding at step \(t{+}1\), closing an autoregressive loop over continuous representations instead of discrete tokens.

2.2 Aligning soft tokens for cross-lingual consistency↩︎

2.2.0.1 From inference-time decoding to training-time alignment.

Soft tokens were originally introduced as a single-language, inference-time enhancement to CoT [6], where they are fed back autoregressively during decoding. For cross-lingual alignment, their continuous form is useful because each soft token mixes shared vocabulary embeddings rather than committing to a single language-specific token or script. Inference-only soft thinking can soften decoding decisions, but does not explicitly train semantically equivalent reasoning paths across languages to stay aligned. SOLAR instead uses soft tokens as a training-time alignment signal during SFT: the forward pass stays teacher-forced on discrete reference tokens, soft tokens are used only to compute an auxiliary alignment loss, and non-English soft-token summaries are pulled toward an English pivot in the shared embedding space.

2.2.0.2 Soft-token summary and alignment.

Given a mini-batch of parallel input-response pairs \({(x^{(\ell)}, y^{(\ell)})}_{\ell \in \mathcal{L}}\) across a set of languages \(\mathcal{L}\) (with English \(\text{en}\in\mathcal{L}\)), a teacher-forced forward pass produces logits \(\mathbf{z}_t\) at each response position \(t \in \{1,\dots,T\}\). After top-\(k\) filtering and temperature scaling, \(\tilde{p}_\theta(v \mid \mathbf{e}_{1:t-1})\) defines the soft-token embedding \(\mathbf{e}^{\,\text{soft}}_t\) via Eq. 2 . For each reasoning trace and final response, we mean-pool these soft tokens into a summary vector \(\bar{\mathbf{e}}^{\,\text{soft}}_\ell = \frac{1}{T}\sum_{t=1}^{T}\mathbf{e}^{\,\text{soft}}_t \in \mathbb{R}^d\). Unlike a discrete token embedding, this vector summarizes likely continuations in the shared embedding space instead of a single vocabulary item or a distribution over \(\mathbb{R}^{|V|}\). Responses can therefore be compared even across languages with disjoint vocabularies: SOLAR aligns semantic mixtures of embeddings, not raw token-distribution support.

2.2.0.3 Cross-lingual alignment objective.

Given a parallel training pair consisting of responses in two different languages \(\ell^*\) and \(\ell\), we let \(\bar{\mathbf{e}}^{\,\text{soft}}_{\ell^*}\) and \(\bar{\mathbf{e}}^{\,\text{soft}}_{\ell}\) denote the mean-pooled soft-token embeddings of the pivot and target responses. To encourage their response-level soft-token summaries to be close in the shared embedding space, we minimize their discrepancy using cosine distance: \[\mathcal{L}_{\text{align}}(\ell^*, \ell) = 1 - \cos\!\left(\bar{\mathbf{e}}^{\,\text{soft}}_{\ell^*},\; \bar{\mathbf{e}}^{\,\text{soft}}_{\ell}\right), \label{eq:pair-loss}\tag{4}\] where \(\ell^*\) denotes the pivot language. We form \((\ell^*,\ell)\) pairs for every non-pivot language \(\ell\), with the pivot summary \(\bar{\mathbf{e}}^{\,\text{soft}}_{\ell^*}\) treated as a fixed target so that alignment gradients flow exclusively through \(\bar{\mathbf{e}}^{\,\text{soft}}_{\ell}\). This pulls non-pivot representations toward the pivot. We set \(\ell^* = \text{en}\) (English) in all experiments, since English consistently yields the highest accuracy across our benchmarks (Table 2).

2.2.0.4 Integration with supervised fine-tuning.

In the SFT setting, the alignment objective \(\mathcal{L}_{\text{align}}\) is added as an auxiliary loss for each non-English training sample in language \(\ell\): \[\mathcal{L}^{(\ell)}_{\text{SFT}} = \mathcal{L}^{(\ell)}_{\text{CE}} + \lambda \cdot \mathcal{L}_{\text{align}}(\ell^*, \ell), \label{eq:sft-loss}\tag{5}\] where \(\mathcal{L}^{(\ell)}_{\text{CE}}\) is the cross-entropy loss for the response in language \(\ell\), \(\lambda > 0\) controls alignment strength, and soft tokens are computed from the same teacher-forced forward pass used to evaluate \(\mathcal{L}_{\text{CE}}\). For English samples, the alignment term is omitted, leaving \(\mathcal{L}^{(\text{en})}_{\text{SFT}} = \mathcal{L}^{(\text{en})}_{\text{CE}}\).

3 Experiments↩︎

We evaluate SOLAR on multilingual reasoning benchmarks to test whether soft-token alignment improves both accuracy and cross-lingual consistency. Further details are in Appendix 10.

3.1 Experimental setup↩︎

3.1.0.1 Training.

We apply supervised fine-tuning to Qwen3-4B/8B and Gemma3-4B/12B [9] using M-s1k [10]. M-s1k is a multilingual long CoT reasoning dataset translated from s1k [11] by Gemini-2.0-Flash [12]. s1k consists of 1,000 diverse, high-quality, and challenging questions paired with complex reasoning traces and solutions. We compute soft tokens with \(\tau{=}1.0\) and top-\(k{=}30\).

3.1.0.2 Evaluation.

We evaluate on MGSM [13] and three XReasoning benchmarks [14]: AIME 2024 [15], AIME 2025 [16], and GPQA Diamond [17] (denoted GPQA hereafter). Each benchmark covers 11 languages; we report the 7 overlapping with M-s1k: En, Fr, Ja, Sw, Te, Th, and Zh. Following [6], we decode with temperature \(0.6\), \(k{=}30\), \(p{=}0.95\), and a maximum length of 32,768 tokens. To reduce variance, we average over 5 runs for MGSM and GPQA and 10 for AIME.

3.1.0.3 Metrics.

We report (1) Task Accuracy (\(\uparrow\)): the proportion of questions answered correctly; and (2) Cross-Lingual Consistency (CLC \(\uparrow\)): the proportion of questions answered correctly in both English and non-English languages simultaneously. Inspired by [4], we define CLC for language pair \((\ell_1, \ell_2)\) as \(\mathrm{CLC}(\ell_1, \ell_2) = \frac{1}{N_q} \sum_{i=1}^{N_q} \mathbf{1}\!\left[a_i^{\ell_1} = a_i^{\ell_2} = a_i^*\right]\), where \(a_i^{\ell}\) is the model’s answer for the \(i\)-th question in language \(\ell\), \(a_i^*\) is the ground-truth answer, and \(N_q\) is the total number of questions.

3.1.0.4 Baselines.

We compare SOLAR with several approaches: (1) base models with standard autoregressive decoding; (2) Soft Thinking [6]: inference-time replacement of discrete tokens with soft-token embeddings; (3) SFT: standard supervised fine-tuning without SOLAR’s alignment objective; (4) MidAlign [18]: a middle-layer representation alignment method based on contrastive learning; (5) MAPO [19]: a preference optimization method aligning non-English reasoning with English via translation model scores; and (6) AlignX [20]: a two-stage framework combining intermediate-layer contrastive alignment with a language matching classifier.

3.2 Main results↩︎

Table 1 reports aggregate results across four benchmarks, and Table 2 breaks down MGSM accuracy by language. Per-language results for AIME 2024, AIME 2025, and GPQA are provided in Appendix 11.

3.2.0.1 SOLAR outperforms all evaluated baselines across benchmarks and model sizes.

For Qwen3-4B, SOLAR improves accuracy over SFT by +1.5 to +3.8 and CLC by +3.1 to +4.5 across all benchmarks. On Qwen3-8B, it achieves +1.3 to +1.9 accuracy and +1.5 to +2.9 consistency gains over SFT. Among cross-lingual alignment baselines, MidAlign is competitive on the harder benchmarks, reaching second-best accuracy on AIME 2024 (36.67%) and AIME 2025 (29.05%). MAPO is competitive on MGSM (77.52%) but underperforms on AIME 2025 and GPQA, likely due to sparse preference pairs: Qwen3-4B answers only 1,148 of 9,000 non-English M-s1k questions correctly under 20-sample decoding. AlignX is comparable to MidAlign and MAPO (e.g., 77.69% on MGSM).

3.2.0.2 Low-resource languages benefit most, with the largest gains on Swahili.

SSwahili accuracy on MGSM nearly doubles from 29.04% to 53.48% for Qwen3-4B and increases from 59.36% to 81.36% for Qwen3-8B, while high-resource languages maintain stable performance (Table 2). This reflects the intuition that cross-lingual alignment is most helpful where cross-lingual divergence is greatest.

3.2.0.3 Inference-time soft thinking does not consistently improve cross-lingual reasoning.

For Qwen3-4B, accuracy changes stay within \(\pm\)​0.8 points across all benchmarks, and CLC shifts are inconsistent (+1.06 on MGSM but \(-\)​0.25 on GPQA). This indicates that simply replacing discrete tokens with soft tokens at inference time cannot substitute for explicit training-time alignment as in SOLAR.

4 Analysis↩︎

This section analyzes the mechanisms behind SOLAR. We examine how SOLAR reshapes cross-lingual representations (§4.1), verify that SOLAR preserves target-language reasoning without shifting generation toward English (§4.2), analyze the sensitivity of the alignment weight and soft-token temperature (§4.3), and isolate which component of the objective drives the gain (§4.4).

4.1 Representation analysis↩︎

We examine how hidden representations are organized across languages and how soft tokens affect cross-lingual alignment. To do so, we first quantify the effect of inference-time soft thinking (Table 3), then ask whether SOLAR further reshapes these representations on Qwen3-4B (Table ¿tbl:tab:repr-after-training?). Let \(\mathbf{h}_i^\ell \in \mathbb{R}^d\) be the hidden state for sample \(i\) in language \(\ell\), with language set \(\mathcal{L}\), \(L=|\mathcal{L}|\), and \(N\) parallel samples per language. We report two metrics in \([-1,+1]\): (1) Cross-Lingual Similarity (CLS \(\uparrow\)), the average pairwise cosine similarity between parallel samples across all language pairs, where higher values indicate more language-agnostic representations; and (2) Silhouette Score (SS \(\downarrow\)) [21], the standard cosine-distance silhouette score measuring how strongly hidden states cluster by language, where lower values indicate weaker language separability. We probe early, middle, and final layers, using layer 1, the 50%-depth layer, and the last layer for each model. Metrics are computed on \(N{=}50\) parallel MGSM instances across the seven training languages. For soft-thinking representations, we first run a discrete-token forward pass to construct soft tokens, then run a second pass with these soft tokens as inputs to extract the corresponding hidden states. Full results for models across the Qwen3, Gemma3, and DeepSeek-R1-Distill [22] families are in Appendix 12.2.

4.1.0.1 Cross-lingual similarity follows an inverted-U (low-high-low) pattern across early, middle, and final layers.

Table 3 compares representations under discrete and inference-time soft thinking on MGSM for three base models, isolating the effect of soft tokens at inference time before introducing SOLAR’s training-time alignment. With discrete tokens, CLS is high in early and middle layers but drops sharply at the final layer. Figure 3 visualizes this trend across all layers for Gemma3-12B. SS shows a U-shaped pattern, reaching its lowest values at the middle layer before rising sharply at the final layer. This pattern matches prior observations that LLMs share a more language-agnostic semantic space mid-network while final layers reintroduce language-specific structure [3]–[5], [23], [24].

4.1.0.2 Soft tokens improve final-layer alignment but do not dissolve language clusters.

Compared to discrete thinking, soft tokens raise final-layer CLS across all models, with the largest gain in Gemma3-12B (\(\Delta\)CLS=+0.68, Table 3). Early and middle layers see little benefit and can even degrade (e.g., Qwen3-8B has \(\Delta\)CLS=\(-\)​0.03 at the middle layer). These layers already reach high CLS under discrete thinking, leaving little room to improve, and the distributional shift from soft tokens may slightly disrupt their alignment. SS improvements are limited and inconsistent: only Gemma3-12B shows a large decrease (\(\Delta\)SS=\(-\)​0.73), while other models show negligible change or slight increases. Language clusters thus largely remain intact for most models (Figure 2).

4.1.0.3 SOLAR produces the strongest cross-lingual representations.

Having shown that soft tokens carry useful cross-lingual signal but that inference-time use alone does not improve performance (§3.2), we examine whether fine-tuning reshapes representations more effectively (Table ¿tbl:tab:repr-after-training?; same 50 MGSM samples). The base Qwen3-4B shows the inverted-U pattern, with CLS dropping to 0.14 at the final layer. SFT raises final-layer CLS to 0.30 (+0.16), while SOLAR reaches 0.50 (+0.36, more than 2\(\times\) the SFT gain) and substantially reduces final-layer SS from 0.89 to 0.77 (\(-\)​0.12), indicating weaker language clustering. Applying soft-thinking inference to fine-tuned models improves SFT’s CLS (0.30\(\rightarrow\)​0.40) but slightly reduces SOLAR’s (0.50\(\rightarrow\)​0.42). This suggests that SOLAR already internalizes the alignment benefit that soft tokens provide, and that training-time alignment subsumes what inference-time soft thinking can recover post-hoc. On Qwen3-8B, SOLAR’s discrete-thinking gain is larger (\(\Delta\)CLS = +0.53) and, unlike on 4B, remains nearly intact under soft thinking (+0.51).

4.2 Behavioral analysis↩︎

We examine whether fine-tuning changes the model’s reasoning behavior on MGSM (Zh, Ja, Th, Te) at the token level (the generated tokens of each reasoning trace) and the distribution level (the top-\(k\)/top-\(p\) candidates in the output distribution during teacher-forced decoding). At both levels, every token is classified into one of three categories using Unicode code-point ranges: target-script (the prompt language’s native writing system, e.g., CJK ideographs for Chinese), Latin (Unicode names containing latin, used as a proxy for English), and neutral (digits, punctuation, math symbols, and whitespace). We define the target-script ratio as \(\#\,\text{target} \,/\, (\#\,\text{target} + \#\,\text{Latin})\) to exclude neutral tokens so that script preference is not diluted by language-agnostic symbols.

4.2.0.1 Fine-tuning preserves native language choice at the token level.

Qwen3-4B reasons almost entirely in the target script (98.81%). SFT and SOLAR maintain comparable ratios (Figure 4). The ratio is nearly identical for correct and incorrect answers (average \(\Delta < 0.5\%\); Table 30), indicating that fine-tuning improves task performance without shifting language choice toward English.

4.2.0.2 SFT and SOLAR achieve high target-script ratios through distinct mechanisms.

The 8B base model shows a stronger English preference than 4B (e.g., Telugu 11.97% vs.above 65%; Figure 5, Table [tbl:tab:target-script-ratio-main]). SFT and SOLAR close this gap (8B average 38.05%\(\rightarrow\)above 87%) through different mechanisms: SFT raises target-script token density in the top-30 predictions, while SOLAR suppresses Latin competition. The contrast is clearest for Telugu, where SOLAR nearly eliminates Latin competition (98.37% for 4B) but SFT leaves it largely unchanged (65.21% vs.%). Under top-\(p\) sampling, all fine-tuned models reach 95–99% target-script ratios, so remaining cross-lingual competition is confined to low-probability alternatives.

4.3 Hyperparameter sensitivity↩︎

4.3.0.1 Alignment weight \(\lambda\).

r0.4

We vary the alignment weight \(\lambda \in \{0.25, 0.50, 0.75, 1.00\}\), with the soft-token temperature \(\tau{=}1.0\) fixed. Both \(\lambda{=}0.25\) and \(\lambda{=}0.50\) perform strongly on MGSM (Table 4), and \(\lambda{=}0.50\) also leads on the harder benchmarks (Table 14). Larger weights degrade performance, with \(\lambda{=}1.00\) falling below the base model, suggesting that excessive alignment overwhelms the cross-entropy objective. We adopt \(\lambda{=}0.50\) for all main experiments in §3.2.

4.3.0.2 Soft-token temperature \(\tau\).

We vary the soft-token temperature \(\tau \in \{0.6, 1.0, 2.0\}\), with \(\lambda{=}0.50\) fixed. \(\tau\) sets the sharpness of the soft-token distribution: lower \(\tau\) yields more peaked distributions near discrete tokens, while higher \(\tau\) yields smoother mixtures over the vocabulary. Setting \(\tau{=}1.0\) performs best (78.3% accuracy, 75.6% CLC; Table 4). Lowering it to \(\tau{=}0.6\) degrades sharply (60.5% accuracy, 57.0% CLC), indicating that overly peaked distributions lose the cross-lingual signal, whereas \(\tau{=}2.0\) yields only a moderate drop (76.4% accuracy, 73.1% CLC).

4.4 What drives the alignment gain?↩︎

To isolate which component of SOLAR’s objective drives its gain, we vary one design choice at a time while holding the rest of the pipeline fixed. The five choices we study are the alignment target, the pooling granularity, the top-\(k\) filter, the alignment representation space, and the pivot language.

4.4.0.1 Alignment target.

We compare four alignment targets for the auxiliary loss: (1) Discrete, the straight-through embedding of the top-1 token (\(\mathbf{e}_t = \mathbf{E}[y_t]\)), replacing the soft-token embedding with one hard token; (2) Final-hidden, the mean-pooled final-layer hidden state, testing whether any final-layer matching signal suffices; (3) Untruncated, the soft-token embedding over the full vocabulary with no top-\(k\) filter; and (4) SOLAR, the default top-\(k{=}30\) soft-token embedding. On Qwen3-4B, SOLAR is the best alignment target (48.25% average accuracy, 41.51% CLC; Table 5). The other targets cluster within run-to-run noise: Discrete and Final-hidden stay at the SFT level (46.00% and 46.09% vs.%), and Untruncated, the strongest alternative (46.55%), still trails SOLAR. A single hard token (Discrete) thus recovers almost none of SOLAR’s gain, so top-\(k\) truncation carries semantic value beyond compute savings. We attribute this to two factors. First, the long tail kept by Untruncated injects cross-lingual noise that top-\(k\) filtering removes. Second, aligning only the final hidden state (Final-hidden) underperforms because the embedding-space soft-token mixture is more language-invariant than the final hidden state itself.

4.4.0.2 Pooling granularity.

We fix the soft-token target and vary the pooling granularity of the alignment loss: (1) Answer-only, pooling over the final \boxed{} span; (2) Step-level, pooling within each step (split on \n\n) and matched by relative position; and (3) Full, the default single pooled pair over the entire response. Answer-only pooling matches SFT (MGSM 76.00% vs.%, Table 6), indicating that the alignment signal lives in the reasoning trace. Step-level, the finest variant with one pair per step, improves over Answer-only but still trails Full (MGSM 77.04% vs.%). We hypothesize that finer granularity helps only when steps correspond across languages. Matching by relative position presumes a parallel step structure that the \n\n split does not guarantee, since equivalent reasoning often spans a different number of steps across languages. Full-response pooling avoids this misalignment. Recovering the benefit of finer granularity likely requires content-aware segmentation that aligns steps across languages, which we leave to future work.

4.4.0.3 Top-\(k\) sensitivity.

We sweep \(k \in \{1, 5, 10, 30, 50\}\) for the top-\(k\) filter defining the soft expectation (Table 7). \(k{=}1\) recovers the Discrete target, since a top-1 filter with renormalization is a hard selection. Accuracy rises steeply from \(k{=}1\) to \(k{=}10\), then plateaus: \(k \in \{10, 30, 50\}\) all reach roughly 48% average accuracy, with no value dominating across benchmarks. Because \(k\) affects only the alignment-loss sum over the top-\(k\) distribution, larger \(k\) raises per-step compute with no accuracy gain. We adopt \(k{=}30\), which attains the best average accuracy among the plateau values.

4.4.0.4 Alignment representation space.

r0.4

We vary the alignment representation space between \(\mathbb{R}^d\) (soft-token embeddings) and \(\mathbb{R}^{|V|}\) (probability distributions), with \(\lambda{=}0.50\) and \(\tau{=}1.0\) fixed. Aligning in probability space degrades accuracy to 60.3% and CLC to 44.8% (Table 8). This degradation suggests that token distributions have largely disjoint support across languages, making cosine distance less informative. Projecting them into the shared embedding space \(\mathbb{R}^d\) instead preserves cross-lingual correspondences and provides a better basis for alignment.

4.4.0.5 Pivot language \(\ell^*\).

We vary the pivot language \(\ell^*\) across English, Chinese, and French, with \(\lambda{=}0.50\) and \(\tau{=}1.0\) fixed. English gives the best improvement, followed by Chinese (Table 8). The gap concentrates in low-resource languages: MGSM Swahili accuracy drops from 53.48% (En) to 40.56% (Zh) and 36.80% (Fr) (Table 15), showing that transfer effectiveness depends on the pivot’s pre-trained representation strength, with English as the strongest anchor.

5 Related work↩︎

5.0.0.1 Multilingual reasoning.

LLMs trained on multilingual corpora exhibit inconsistent cross-lingual reasoning performance [1], [13], and recent work attributes these disparities to internal mechanisms. [3] show that errors arise in the final-layer transition from language-agnostic intermediate representations to language-specific outputs. [25] identify language-specific attention heads that mediate cross-lingual attention transfer. [4] find that larger models rely more on language-specific representations. Methods for improving multilingual reasoning, particularly for low-resource languages, fall broadly into training-based and inference-based categories [26]. Training-based methods leverage parallel multilingual data during pre-training [27] or post-training [28], language-mixed CoT data [29], adapter merging [30], and representation alignment [20], [31]. A recent wave of multilingual reinforcement learning (RL) methods uses English as a pivot with translation-quality, language-consistency, or self-feedback rewards [32]–[34]. Inference-based methods include in-context learning [35], activation steering [5], test-time scaling [36], and inference-time linear transformations between source- and target-language hidden states [37].

5.0.0.2 Continuous-space reasoning.

While CoT prompting [8] enhances LLM reasoning through discrete tokens, a growing line of work argues that reasoning may instead unfold in continuous representations that encode richer information than discrete tokens [38], [39]. Several methods train LLMs to reason directly in continuous space [40], [41]; for instance, COCONUT [42] feeds the last hidden state back as the next input embedding, allowing multiple reasoning steps to unfold in parallel. [43] train continuous CoTs with reinforcement learning by injecting noise into the input embedding mixture for exploration, and report that decoding with discrete tokens at inference on top of a continuous-CoT trained model performs best. Among training-free approaches, Soft Thinking [6] replaces discrete token embeddings with probability-weighted soft tokens for richer inference-time representations, and theoretical work further shows that continuous CoT can efficiently solve graph-reachability problems [44]. Taken together, these results suggest that continuous-space reasoning carries expressive advantages over its discrete counterpart [39].

6 Conclusion↩︎

We introduce SOLAR, an auxiliary training objective that aligns soft-token representations across parallel multilingual reasoning traces. SOLAR improves task accuracy and cross-lingual consistency over standard SFT across four multilingual benchmarks, with the largest gains on low-resource languages. At the representation level, SOLAR more than doubles SFT’s final-layer cross-lingual similarity gain and substantially reduces language-cluster separability, showing that aligning soft-token representations helps preserve shared semantics and improve cross-lingual consistency.

Acknowledgments↩︎

This research is supported in part by the NSF under grant numbers IIS-2144493, IIS-2052498, SMA-2418946, and NAIRR250217, in addition to a gift from Google. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

7 Further discussion, limitations, and future work↩︎

While our study shows that aligning soft-token representations during SFT improves multilingual reasoning, it opens several directions for future work.

7.0.0.1 From SFT-only alignment to reinforcement learning.

SOLAR is studied as an auxiliary loss alongside supervised cross-entropy. Recent work suggests that reinforcement learning consistently outperforms SFT for cross-lingual reasoning and generalizes better to unseen languages [45]. Future work could integrate soft-token alignment into preference- or reward-based post-training, for example by combining the alignment loss with Direct Preference Optimization (DPO) [46] or Group Relative Policy Optimization (GRPO) [47]. This could compound the consistency gains of soft-token alignment with the generalization that policy optimization brings to unseen languages. Recent pivot-based multilingual RL methods such as PASMR [32], TAPO [33], and M-Thinker [34] reward consistency between English and target-language reasoning at the output level. SOLAR’s soft-token alignment offers a complementary representation-level signal that could be added to these reward-based objectives as an auxiliary alignment term during policy optimization.

7.0.0.2 Reasoning domains beyond math and science.

Our evaluation focuses on multilingual mathematical reasoning (MGSM, AIME 2024, AIME 2025) and graduate-level scientific question answering (GPQA Diamond), where final answers are short and verifiable. Whether response-level mean-pooled soft-token summaries carry the same cross-lingual signal for tasks with open-ended outputs (e.g., multilingual summarization [48]) remains an open question. Future work could extend SOLAR to broader generation tasks and investigate step-level or chunk-level alignment for reasoning traces whose semantic structure is non-monotonic or interleaved with retrieval.

7.0.0.3 Scaling parallel reasoning data across languages.

SOLAR is trained on M-s1k [10], which provides parallel long CoT traces in 10 languages. We report results on the 7 that overlap with our evaluation suites (Appendix 10.4). Expanding coverage requires translating long CoT traces into each new language with mathematical-fidelity verification, which incurs non-trivial cost. SOLAR’s effectiveness on a broader set of languages, particularly low-resource ones, therefore remains to be validated. One direction is to scale parallel reasoning data through verified translation pipelines. Another is to investigate whether soft-token alignment can be combined with weaker supervision, such as machine-translated traces with quality filtering, or unpaired monolingual reasoning anchored through a shared retrieval signal, to relax the strict parallel-data requirement.

7.0.0.4 Toward broader model families and scales.

Our main fine-tuning experiments target Qwen3-4B/8B and Gemma3-4B/12B. Although the representation analysis (§4.1) covers 10 models from the Qwen3, Gemma3, and DeepSeek-R1-Distill families, the SFT pipeline is validated on two families at moderate scales. Larger models (e.g., 30B+), mixture-of-experts architectures, and multimodal LLMs may exhibit different vocabulary geometries and pre-training compositions, potentially altering the strength and shape of the soft-token alignment signal. Extending SOLAR to a broader set of families and scales offers a route to assessing how generally the soft-token alignment principle holds.

8 Broader impacts↩︎

As multilingual LLMs serve users worldwide, ensuring consistent reasoning across languages is both a technical and an equity concern. Most reasoning gains in modern LLMs remain concentrated in English and a handful of high-resource languages [49], leaving speakers of underrepresented languages with weaker access to advanced reasoning in domains such as decision support [50]. Our work targets this gap by aligning soft-token representations across languages during SFT, with the largest gains on low-resource languages (e.g., Swahili MGSM accuracy rises from 29.04% to 53.48% for Qwen3-4B, and Telugu AIME 2024 from 5.67% to 29.67%), narrowing performance disparities while preserving native-language generation (Appendix 12.3).

Beyond accuracy, our analysis helps make multilingual model behavior more interpretable and controllable. Soft-token embeddings provide a continuous, vocabulary-shared signal in which cross-lingual reasoning can be inspected and shaped, complementing hidden-state probing and offering a lightweight, architecture-agnostic mechanism that can be combined with other multilingual training pipelines. Because SOLAR adds no architectural modification and reuses the model’s own forward-pass logits, it is also accessible when compute or model-modification budgets are limited.

At the same time, anchoring all non-English summaries to an English pivot raises questions about cultural and linguistic representation. An English-anchored shared space may carry English-centric reasoning conventions, framing assumptions, or normative answers that surface only on culturally grounded tasks not covered by our benchmarks [51]. Stronger multilingual reasoning also extends the reach of downstream misuse currently bottlenecked by English-only proficiency, and our work does not address multilingual safety alignment. We see multi-pivot or pivot-free alignment, evaluation on culturally grounded reasoning, and multilingual safety post-training as important complements to our work. Overall, we hope our work contributes to a more inclusive and principled approach to multilingual reasoning and helps narrow the cross-lingual capability gap in a way that is mindful of the linguistic communities these systems aim to serve.

9 Related work↩︎

9.1 Multilingual reasoning↩︎

LLMs trained on multilingual corpora exhibit cross-lingual reasoning abilities [2], [7], [9], yet show inconsistent performance across different languages [1], [3]. Specifically, given the same task, LLMs generally perform better when prompted in English, whereas their reasoning performance deteriorates when the task is expressed in low-resource languages. Recent work examines these disparities through internal model mechanisms. [3] reveal that models encode factual knowledge in a language-agnostic concept space through intermediate layers, but errors arise during the final-layer transition to language-specific representations. [25] identify language-specific attention heads that enable cross-lingual attention transfer. Recently, [4] show that larger models increasingly rely on language-specific representations, hindering cross-lingual transfer. These mechanistic insights provide a foundation for understanding and improving cross-lingual reasoning.

9.2 Cross-lingual alignment↩︎

Efforts to improve multilingual reasoning in LLMs can be broadly categorized into training-based and inference-based approaches [26]. Training-based methods use translated parallel data during pre-training [27], [52], [53] or post-training [10], [28], [54]–[58], language-mixed CoT data [29], [59], adapter merging  [30], representation alignment [18], [20], [31], [60], and preference optimization methods without parallel data [19], [61]–[66]. [45] demonstrate that RL consistently outperforms SFT in cross-lingual reasoning and achieves better generalization even when trained on non-English data. PASMR [32] maps multilingual inputs to an English pivot via SFT and then runs self-feedback RL that rewards consistency between pivot and target-language reasoning. TAPO [33] couples a translation-quality reward with the standard reasoning reward under a step-level relative-advantage mechanism that decouples credit assignment between translation and reasoning tokens. Recently, M-Thinker [34] combines a language-consistency reward with a cross-lingual thinking-alignment reward computed via LLM-as-judge against an English reasoning trace. These methods are complementary to SOLAR’s soft-token alignment signal, and we discuss their potential integration in Appendix 7. Inference-based methods include in-context learning [35], [67], cross-lingual activation steering [5], [68], test-time scaling [36], [69], and learned linear transformations between source- and target-language hidden states applied at inference [37]. Studies show that prompting models to reason in English generally preserves or improves non-English task performance [10], [36]. SOLAR differs from these inference-time interventions in that it modifies the model parameters during SFT through an auxiliary soft-token alignment loss, with no per-language transformation at inference and no test-time overhead beyond standard discrete decoding.

9.3 Continuous-space reasoning↩︎

While CoT prompting [8] enhances LLM reasoning through step-by-step discrete linguistic tokens, recent work argues that LLMs should mirror human cognition, which operates through abstract, continuous concepts rather than explicit verbalized steps [38]. Several methods train LLMs to reason in continuous space [40], [41], [70]. COCONUT [42] feeds the last hidden state as the next input embedding, enabling the model to encode multiple reasoning steps simultaneously. For training-free approaches, Soft Thinking [6] replaces discrete token embeddings with probability-weighted soft tokens, forming a continuous concept space for richer representations at inference time. [71] enhance this with noise via Dirichlet resampling and the Gumbel-Softmax trick. [43] train continuous CoTs with reinforcement learning by injecting Gaussian noise into the input embedding mixture to enable exploration, removing the need to distill from ground-truth discrete reasoning traces. They report that continuous-CoT training matches discrete training at pass@1 and surpasses it at pass@32 on math benchmarks, and that decoding with discrete tokens at inference on top of a continuous-CoT trained model performs best. Theoretical frameworks further demonstrate that continuous CoT can solve graph reachability problems more efficiently by maintaining multiple search frontiers simultaneously [44], [72]. Together, these works show that continuous-space reasoning enables richer information encoding compared to discrete token-based methods [39], [73]. While prior work focuses on monolingual reasoning, SOLAR repurposes the soft-token formulation [6], [43] as a cross-lingual alignment signal during post-training. We investigate whether probability-weighted mixtures over the shared vocabulary embedding matrix can serve as a comparable signal across languages, even when their surface tokens are disjoint.

10 Implementation details↩︎

10.1 Models↩︎

For main experiments (§3), we fine-tune Qwen3-4B/8B [7] and Gemma3-4B/12B [9] via full-parameter SFT on M-s1k. We center the headline analysis on the Qwen3 family for its strong multilingual reasoning and wide adoption; full Gemma3 results (MGSM and GPQA) appear in §11. We use the 4B and 8B scales for Qwen3 because they balance reasoning capability against the compute cost of full-parameter fine-tuning on A40 GPUs (48 GB). For representation experiments (§4.1), we select 10 models from three families: Qwen3, Gemma3, and DeepSeek-R1-Distill [22]. These span multiple scales (1B–27B) and both base and instruction-tuned variants (e.g.,Qwen3-4B-Base vs.Qwen3-4B). Specifically, Qwen3 and Gemma3 are advanced multilingual models supporting 119 and 140+ languages, respectively.

10.2 Training setup↩︎

All main-experiment models undergo full-parameter fine-tuning using the LLaMA-Factory framework [74]. Qwen3-4B is trained in non-thinking mode, while Qwen3-8B is trained in thinking mode. We train for 5 epochs with a batch size of 32. All runs use bfloat16 precision with a learning rate of \(1 \times 10^{-5}\) that warms up linearly over the first 10% of training and then decays to 0 following a cosine schedule. We use the AdamW optimizer [75] with \(\beta_1 = 0.9\) and \(\beta_2 = 0.999\). The maximum sequence length is 16,384 tokens. We use DeepSpeed ZeRO Stage 2 [76] to reduce memory usage. All experiments are conducted on NVIDIA A40 GPUs (48 GB). For the SOLAR alignment objective, we compute soft tokens with \(\tau{=}1.0\) and top-\(k{=}30\). We ablate the alignment weight \(\lambda \in \{0.25, 0.50, 0.75, 1.00\}\) and soft-token temperature \(\tau \in \{0.6, 1.0, 2.0\}\) in §4.3. We employ a group-aware batch sampler that places all translations of each problem contiguously, enabling co-occurrence within the same training step even under small per-device batch sizes.

10.3 Evaluation setup↩︎

We use SGLang (v0.4.6) [77] as the inference backend. For all evaluations, we use a sampling temperature of \(0.6\), top-\(k{=}30\), top-\(p{=}0.95\), and a maximum generation length of 32,768 tokens. Note that this sampling temperature is used solely for next-token decoding at inference time and is distinct from the soft-token temperature \(\tau\) used during SOLAR training (Eq. 2 –3 ). We run each evaluation multiple times and report the average to reduce estimation variance: five runs for MGSM and GPQA, and ten runs for AIME 2024 and AIME 2025. We count a model’s answer as correct only if the extracted final answer (i.e., the content within \boxed{}) matches the ground-truth label.

10.4 Datasets↩︎

10.4.0.1 Training data.

M-s1k [10] is a multilingual long CoT reasoning dataset translated from s1k [11] by Gemini-2.0-Flash [12]. It contains 1,000 parallel problems across 10 languages (English (En), Chinese (Zh), French (Fr), Japanese (Ja), Latvian (Lv), Swahili (Sw), Telugu (Te), Thai (Th), Afrikaans (Af), and Marathi (Mr)), totaling 10,000 training examples. Each pairs a question with a detailed reasoning trace and final solution. We additionally annotate each with a parallel_id linking translations of the same problem across languages and a language tag for fine-tuning.

10.4.0.2 Evaluation benchmarks.

We evaluate on four benchmarks covering multilingual mathematical reasoning and science question answering. MGSM (Multilingual Grade School Math) [13] extends GSM8K [78] with 250 parallel grade-school math word problems requiring multi-step arithmetic. AIME 2024 [15] and AIME 2025 [16] each contain 30 competition-level problems from the American Invitational Mathematics Examination. GPQA Diamond [17] is a graduate-level, Google-proof benchmark of 198 multiple-choice questions across physics, chemistry, and biology, designed to challenge domain experts. The multilingual versions of AIME 2024, AIME 2025, and GPQA Diamond are from XReasoning [14]. All four cover 11 languages: English (En), Spanish (Es), French (Fr), German (De), Russian (Ru), Chinese (Zh), Japanese (Ja), Thai (Th), Swahili (Sw), Bengali (Bn), and Telugu (Te). Since only 7 of the 10 M-s1k languages overlap with the evaluation set (En, Fr, Ja, Sw, Te, Th, Zh), we report main results on these seven. For the representation analysis (§4.1), we sample 50 instances per language from MGSM and GPQA Diamond and use all 30 per subset for AIME 2024 and AIME 2025; the main experiments (§3) use all instances. Expanding to all 11 languages would require translating s1k reasoning traces, which incurs non-trivial API cost for long CoT data and needs quality verification to ensure mathematical fidelity across languages. We leave broader language expansion to future work.

10.5 Baselines↩︎

10.5.0.1 Soft Thinking.

We follow the formulation in §2.1 and apply inference-only soft thinking [6] to base models. We use the same decoding hyperparameters as our evaluation setup (§10.3).

10.5.0.2 MidAlign.

A middle-layer contrastive alignment objective that encourages cross-lingual similarity by maximizing cosine similarity between mean-pooled hidden states of parallel translation pairs while minimizing similarity to non-translations within each batch [18]. We adapt MidAlign from its original LoRA-based setup to our full-parameter SFT setting and apply the contrastive alignment loss at the middle layer (18 for Qwen3-4B) jointly with the cross-entropy loss, using the same M-s1k parallel training data and hyperparameters as SOLAR (§10.2). MidAlign aligns discrete hidden states at intermediate layers, whereas SOLAR aligns soft-token embeddings.

10.5.0.3 MAPO.

An optimization framework that uses a pre-trained machine translation model to score the consistency between non-English and English reasoning chains, treating these scores as signals for preference training [19]. MAPO optimizes cross-lingual consistency at the output level via preference learning over discrete reasoning, whereas our method aligns soft-token embeddings at the representation level during SFT. For each of the 9,000 non-English questions in M-s1k, we sample \(n{=}20\) candidate outputs from Qwen3-4B with sampling temperature \(1.0\). We then score each output using NLLB-200-distilled-600M [79]. The alignment score for each non-English output is computed as \(1 / \mathcal{L}_{\text{CE}}\), where \(\mathcal{L}_{\text{CE}}\) is the cross-entropy loss of the NLLB model translating the non-English output to the corresponding English reference answer. Preference pairs are constructed by selecting, for each question, a chosen response that is both correct and has a higher alignment score, paired with a rejected response that has a lower alignment score. This yields 6,305 preference pairs from 1,148 questions; the remaining questions are excluded because none of the 20 sampled outputs produce a correct answer. We perform full-parameter DPO [46] with \(\beta{=}0.1\). All other training hyperparameters follow the same setup as SOLAR (§10.2).

10.5.0.4 AlignX.

A two-stage framework for multilingual representation alignment [20]. Stage 1 combines three objectives: (1) an InfoNCE-style contrastive loss that aligns mean-pooled hidden states at an intermediate layer across parallel translation pairs using cosine similarity (with temperature \(0.1\)), (2) a language-matching classifier at the final layer to reduce off-target generation, and (3) the standard next-token prediction loss. Stage 2 performs multilingual instruction fine-tuning with only the NTP loss. We apply contrastive alignment at the middle layer of the model. The language-matching classifier is a 2-layer MLP with intermediate dimension 128. We use \(\alpha_1{=}0.3\) and \(\alpha_2{=}0.4\) as the loss weights for the contrastive and language matching losses, respectively, following the original paper. All other training hyperparameters follow SOLAR (§10.2).

11 Experimental results↩︎

Tables 9–12 report per-language accuracy and cross-lingual consistency for Qwen3-4B, Qwen3-8B, Gemma3-4B, and Gemma3-12B, along with their fine-tuned variants, across four benchmarks. On MGSM and AIME, SOLAR’s accuracy gains over the base model are largest for low-resource languages, while high-resource languages change far less (within roughly \(\pm\)​1 point) and occasionally regress. For example, Qwen3-4B Swahili improves by +24.4 points on MGSM (+22.0 for Qwen3-8B), and Qwen3-4B Telugu by +24.0 on AIME 2024. On GPQA, all seven Qwen3-4B languages improve under SOLAR, including English, showing broad SFT-level benefits on this science benchmark. For Gemma3, MGSM and GPQA Diamond follow the Qwen3 patterns: SOLAR yields the highest average accuracy and CLC at both scales, with low-resource languages benefiting most. Baseline strength differs across families: Gemma3-12B exceeds Qwen3-8B on MGSM (89.50% vs.%) while Gemma3-4B trails Qwen3-4B (53.31% vs.%). On GPQA, base Gemma3 performs near or below the four-way random baseline of 25% (16.34% at 4B, 26.61% at 12B), and SOLAR raises both scales over SFT. Because Gemma3-4B’s GPQA accuracy already sits near the random floor, we omit Gemma3 from AIME 2024 and 2025. Among Qwen3-4B baselines, MidAlign is strongest on AIME 2024, while MAPO posts high single-language MGSM scores without consistent CLC gains, so preference optimization can raise individual-language accuracy without aligning cross-lingual reasoning. MAPO’s English accuracy falls below the base model on MGSM and both AIME sets, as expected when training only on non-English preference pairs, and its limited signal (6,305 pairs from 1,148 questions) likely constrains low-resource gains.

Table 13 reports the Pearson correlation between average accuracy and CLC across model variants. The two are nearly perfectly correlated (\(r \ge 0.997\) on every individual benchmark and \(r = 0.995\) pooled across all), so accuracy gains reflect stronger cross-lingual consistency rather than improvement in a single dominant language.

12 Analysis↩︎

12.1 Ablation study↩︎

12.1.0.1 Alignment weight \(\lambda\).

Table 14 reports \(\lambda \in \{0.25, 0.50, 0.75, 1.00\}\) with \(\tau{=}1.0\) across all four benchmarks. The weight trades off cross-entropy against the alignment loss (Eq. 5 ). \(\lambda{=}0.25\) attains the highest MGSM accuracy and CLC, with \(\lambda{=}0.50\) close behind, but we adopt \(\lambda{=}0.50\) because it leads on the harder benchmarks (e.g., 38.52% vs.% on AIME 2024). Performance degrades at \(\lambda{=}0.75\) and falls below the base model at \(\lambda{=}1.00\) (69.25% vs.% on MGSM), where low-resource languages lose the most (Swahili AIME 2024: 32.67% \(\rightarrow\) 10.00%), indicating that excessive alignment overwhelms cross-entropy and disrupts languages with sparser pre-training coverage.

12.1.0.2 Soft-token temperature \(\tau\).

Table 15 (a) reports \(\tau \in \{0.6, 1.0, 2.0\}\) with \(\lambda{=}0.50\). Lower \(\tau\) produces more peaked distributions that approximate discrete token embeddings, while higher \(\tau\) yields smoother mixtures over more vocabulary tokens. At \(\tau{=}0.6\), average accuracy and CLC drop sharply (by 17.8 and 18.6 points relative to \(\tau{=}1.0\)), and the loss is highly uneven: Telugu falls to 14.48% while English remains high at 89.90%. With the continuous signal gone, the objective reduces to aligning discrete representations, so low-resource languages, which have the weakest baseline alignment, suffer the most. At \(\tau{=}2.0\) the drop is moderate (\(-\)​1.9 accuracy, \(-\)​2.5 CLC) and the per-language pattern partially reverses: Swahili and Telugu reach their best accuracies across the three temperatures while high-resource languages decline (e.g., French 86.32% \(\rightarrow\) 79.28%). Smoother distributions thus spread the signal more broadly, helping sparse-coverage languages at the cost of the semantic specificity high-resource languages rely on. The optimum at \(\tau{=}1.0\) balances the two: soft tokens must differ meaningfully from discrete tokens yet stay specific enough for effective alignment.

12.1.0.3 Alignment representation space.

Table 15 (b) compares aligning mean-pooled soft-token embeddings in \(\mathbb{R}^d\) (SOLAR) against mean-pooled probability distributions in \(\mathbb{R}^{|V|}\), which bypasses the embedding layer and minimizes cosine distance between \(\bar{p}_{\text{en}}\) and \(\bar{p}_{\ell}\) (Eq. 3 ). Embedding-space alignment wins by a wide margin: in probability space accuracy drops 18.0 points and CLC 30.9 points, falling below SFT and the base model. The collapse is especially severe for CLC because equivalent reasoning across languages activates largely disjoint token subsets in \(\mathbb{R}^{|V|}\), so cosine distance gives little useful gradient. The embedding projection, by contrast, maps these probabilities into a shared space where cross-script equivalences are captured. Alternative distance metrics such as KL or Jensen–Shannon divergence are left to future work.

12.1.0.4 Pivot language \(\ell^*\).

Table 15 (c) reports per-language results for Qwen3-4B trained with different pivot languages. All other hyperparameters are fixed at \(\lambda{=}0.50\) and \(\tau{=}1.0\). English yields the best aggregate performance (78.27% accuracy, 75.61% CLC), followed by Chinese (74.50%, 70.97%) and French (72.87%, 66.67%). The performance gap is concentrated in low-resource languages, where Swahili drops from 53.48% (En) to 40.56% (Zh) and 36.80% (Fr), consistent with the hypothesis that transfer effectiveness depends on the quality of the pivot’s pre-trained representations. English, as the dominant language in pre-training corpora, provides the strongest anchor for alignment.

12.1.0.5 Full alignment-target results.

Table 16 extends the alignment-target ablation of §4.4 to Qwen3-8B alongside Qwen3-4B. At both scales SOLAR is the best target and a single hard token (Discrete) stays at the SFT level, while the intermediate targets (Final-hidden, Untruncated) fall within about a point of one another and are not strictly ordered across benchmarks. The consistent SOLAR-best pattern across scales indicates that the soft mixture, rather than model size, drives the gain.

12.2 Representation analysis↩︎

12.2.0.1 Setup.

We sample 50 instances per language for MGSM and GPQA and all 30 from each AIME subset. We extract hidden states in two passes. Pass 1 runs a standard forward pass, mean-pools hidden states at each target layer, and builds soft tokens via Eq. 2 with \(\tau = 0.6\) and \(k = 30\). Pass 2 feeds these soft tokens back into the same model and extracts hidden states at the same layers. We probe three depths: early (layer 1), middle (\(\sim\)​50%), and final (layer \(-1\)), reporting \(\Delta\)CLS and \(\Delta\)SS relative to discrete tokens. Positive \(\Delta\)CLS or negative \(\Delta\)SS indicates improved cross-lingual mixing.

12.2.0.2 Results.

Tables 17–28 present the layer-wise cross-lingual alignment analysis for all 10 models across four benchmarks and 11 languages. The inverted-U pattern from §4.1 holds across all models and benchmarks: CLS peaks at middle layers and drops sharply at the final layer. Soft thinking improves final-layer CLS for most models and benchmarks, with magnitude varying by family: Gemma3 shows the largest gains (\(\Delta\)CLS up to +0.68), Qwen3 improves more modestly, and DeepSeek-R1-Distill shows the smallest effect, with small negative \(\Delta\)CLS for DeepSeek-R1-Distill-Llama-8B on AIME 2024/2025 and GPQA. Table 29 reports CLS and SS at three depths for Qwen3-4B, Qwen3-8B, and their SFT and SOLAR variants, extending the final-layer summary in §4.1 to early and middle layers. The 8B results corroborate the 4B patterns, with gains concentrated at the final layer: SOLAR raises final-layer discrete CLS from 0.23 to 0.76 (\(\Delta\)CLS = +0.53), and soft thinking lifts it further to 0.87 (+0.64). For 4B, SOLAR’s discrete gain (+0.36) exceeds its cumulative soft-thinking gain (+0.28), so SOLAR has largely internalized what soft thinking provides. On 8B, soft thinking still adds on top of SOLAR (+0.64 vs.+0.53), suggesting the larger model retains capacity for further inference-time alignment. Early and middle layers change minimally at both scales, confirming the effect is specific to the final layer.

12.3 Behavioral analysis↩︎

We examine whether fine-tuning changes the model’s reasoning language when the prompt is non-English. We evaluate Qwen3-4B on MGSM across four non-Latin-script languages spanning high-, mid-, and low-resource settings (Zh, Ja, Th, Te) and analyze both the token-level (generated tokens) and the distribution-level (top-30 tokens in the output distribution during decoding). To quantify language use in reasoning traces and top-\(k\) distributions, we classify every token into one of three categories by Unicode code-point range: (1) target-script: characters belonging to the prompt language’s native writing system (CJK Unified Ideographs U+4E00--9FFF and extensions for Chinese; CJK plus Hiragana U+3040--309F and Katakana U+30A0--30FF for Japanese; Thai U+0E00--0E7F; Telugu U+0C00--0C7F); (2) Latin: characters whose Unicode name contains latin, a proxy for English; and (3) neutral: digits, punctuation, mathematical symbols, and whitespace (Unicode categories N, P, S, and Z), which are script-independent and shared across languages. We average over five MGSM runs. The target-script ratio is defined as \(\text{target} / (\text{target} + \text{Latin})\), which excludes neutral tokens from the comparison.

12.3.0.1 Token-level composition.

Tables 30–33 report the target-script ratio of generated reasoning traces, overall and by answer correctness. On MGSM, all 4B variants reason almost entirely in the target script (96–99%, average correct/incorrect \(\Delta < 0.5\%\)). The 8B base splits sharply: Chinese stays high (99.15%), but Japanese, Thai, and Telugu fall below 5%, with the model reasoning in English. SFT and SOLAR both recover all four above 97%. On harder benchmarks (AIME 2024, AIME 2025, GPQA), target-script ratios drop for all models. The 4B base declines the most (75–80% average). Fine-tuned models stay more robust (roughly 77–99%, Telugu highest). The 8B base retains its English-thinking pattern (Ja/Th/Te below 1.1%), while SFT and SOLAR again recover to 86–89%.

12.3.0.2 Top-\(k\) composition.

We perform teacher-forced decoding and classify the top-30 tokens at each position as target-script, Latin, or neutral. Figure 5 (top row) shows the composition breakdown and Table 34 the target-script ratio, revealing two findings. First, the 8B base has a stronger English preference than the 4B base. SFT and SOLAR raise its MGSM average from 38.05% to above 87%. Second, the two methods shift this preference differently: SFT increases target-language token density, while SOLAR suppresses Latin competition without proportionally adding target tokens. The contrast is clearest for Telugu, where SOLAR reaches 98.37% (4B) while SFT barely moves the base ratio (65.21% vs.%), consistent with SOLAR’s alignment objective reducing cross-lingual hedging. These patterns extend to the harder benchmarks. The 4B base now shows the distribution-level collapse previously seen only in the 8B base on MGSM. Fine-tuning again recovers most of the loss, and SOLAR keeps the highest Telugu ratio on every benchmark.

12.3.0.3 Top-\(p\) composition.

Because the fixed budget (\(k{=}30\)) may include low-probability tokens the model would never sample, we repeat the composition analysis under nucleus (top-\(p\)) sampling with \(p{=}0.9\), retaining the minimal token set whose cumulative probability exceeds the threshold. Table 34 reports the top-\(p\) target-script ratio, and Figure 5 (bottom row) the full breakdown. The 8B base’s English dominance persists under top-\(p\) (below 33% for Japanese, Thai, and Telugu), confirming that the top-\(k\) preference is not an artifact of low-probability tail tokens. SFT and SOLAR again raise its average from 41.37% to above 97%. After fine-tuning, the primary generation intent is strongly target-language, and the residual cross-lingual competition sits in the low-probability tail. The same trends hold on AIME 2024, AIME 2025, and GPQA: base models retain their collapse while trained models reach high top-\(p\) ratios. The top-\(p\)/top-\(k\) gap narrows on the harder benchmarks, indicating that on hard problems English alternatives sit higher in the probability mass. SOLAR attains the highest Telugu ratios throughout (99.34% / 99.47% top-\(p\) for 4B / 8B).

12.4 Case Study↩︎

We present two case studies where the SOLAR-trained model answers correctly while the base Qwen3-4B and the SFT-only model both fail. The examples show two qualitative advantages of SOLAR. First, it produces shorter, more structured reasoning chains, avoiding the repeated dead ends of the base model. Second, it reduces interpretation errors on multilingual mathematical expressions, translating symbolic constraints in non-English problem statements into geometric reasoning.

12.4.0.1 AIME 2025 Q28 (Chinese).

Figure [fig:case-study-q28] shows the full reasoning traces. This geometry problem asks for the area of a quadrilateral inscribed in a right triangle. The base model correctly sets up the coordinate system and derives the main constraints but fails to solve the resulting nonlinear system, abandoning the derivation and guessing (\(n=147\)). The SFT model introduces an unsupported symmetry assumption, incorrectly setting \(b=c\), which yields the wrong answer \(98\sqrt{3}\). SOLAR correctly enforces the constraints, obtains \(b=16\sqrt{3}\) and \(c=26\), computes the vertex coordinates, and applies the shoelace formula to reach the correct answer \(n=104\).

12.4.0.2 AIME 2025 Q7 (Japanese).

Figure [fig:case-study-q7] shows the full reasoning traces. This problem requires finding real values of \(k\) for which a circle and a perpendicular bisector intersect exactly once. A key challenge is correctly interpreting \(|z-4-k|\) and \(|z-3i-k|\), which correspond to distances to \((4+k,0)\) and \((k,3)\). The base model misreads them as \((4,k)\) and \((0,k)\), placing both points on \(y=k\) and producing a fixed bisector \(x=2\), which leads it to conclude that the line cannot intersect the circle after repeatedly revisiting the same steps. The SFT model makes a different parsing error, implicitly treating \(k\) as \(ki\), which yields incorrect reference points and the wrong result \(m+n=8\). SOLAR derives the correct bisector \(-8x+6y+8k+7=0\) and solves \(|8k-73|=50\), producing the correct result \(m+n=77\).

References↩︎