1 Introduction↩︎

Large Language Models (LLMs) have achieved unprecedented performance across a wide range of tasks. Yet, as ML models, they remain fundamentally constrained when used in data-scarce, out-of-distribution scenarios. A particularly revealing example is the processing of extremely low-resource or entirely unseen languages, e.g. unseen language translation [1], a setting that we argue constitutes an ideal testbed for LLM research for several reasons. First, reasoning over linguistic patterns with limited pre-training data remains challenging even for frontier LLMs [2], [3]. Second, progress here has direct societal implications: it might enable more effective documentation and preservation of endangered languages, for which large-scale (parallel) corpora hardly exist [4].

Two paradigms dominate LLM approaches to new languages: post-training the model on additional language/task-specific data [5], [6]; and where resources are even more limited, in-context learning using inductive or deductive examples in the prompt [1], [7], [8]. We argue that a more principled and scalable approach should instead be language-independent learning, which is inspired by meta-learning [9], [10]. Rather than memorize the content or knowledge of any specific language, an LLM should acquire the ability to generalize across new languages by reasoning over an accompanying context with rich linguistic features, grammatical descriptions, morphological paradigms, and lexicons. We refer to this transferable ability as a meta-skill of contextual leveraging, and view it as the key property that any general-purpose method for very low-resource languages should target.

Motivated by this perspective, we introduce a reinforcement learning (RL) method which can elicit LLMs to leverage linguistic context when translating new unseen languages. The recent success of RL with verifiable rewards (RLVR) on reasoning tasks [11], [12] suggests that outcome-based RL is particularly effective at eliciting transferable skills (e.g. reasoning) rather than memorized fixed solutions. We bring this philosophy to low-resource translation: we treat “translating a new language given its grammar” as a verifiable, context-dependent reasoning problem. Concretely, the model is trained to produce translations conditioned on a context of linguistic knowledge and dictionary entries, with a translation quality metric (e.g., chrF) as the outcome reward. By simply rewarding the translation using RL, we can elicit LLMs to learn the meta-skill of leveraging linguistic context and can gain better generalization performance for unseen languages. In summary, our contributions are as follows:

• A reframing of extreme low-resource translation as the acquisition of a language-independent meta-skill, reasoning over in-context linguistic context, rather than memorization of any individual target language, unifying in-context learning and post-training under one objective.

• An outcome-reward RL recipe for context leveraging. We propose a simple training method that conditions the model on linguistic knowledge and dictionary context and uses translation quality as the outcome reward.

• Empirical evidence for generalization. Through controlled context learning comparisons, we show that RL can bring better performance on unseen languages, while SFT overfits to in-domain training languages, showing that the method successfully generalizes.

2 Related Work↩︎

[1] introduced MTOB, a benchmark for translating between English and Kalamang (an extreme low-resource language). By providing a grammar book [13] as context, this work demonstrates how LLMs can leverage explicit linguistic documentation to translate an entirely new language. [14] include bilingual dictionaries, grammar books, and morphologically analyzed text in context to guide LLM translation without parameter updates. Following the DiPMT [15] framework, where an LLM is fed bilingual lexicons of rare words in the prompt, [16] used dictionary retrieval and in-context learning to teach LLMs the Zhuang language on the fly. [17] further explored the effectiveness of grammar by decomposing it into rule retrieval and application, and specifically proposed to convert grammar into pseudocode. On the contrary, more recently, [18] showed that LLMs make little use of grammar descriptions but rely on the parallel examples in the book when translating an unseen language. This finding is backed by the study by [19] on Manchu that bilingual dictionaries are more helpful, while grammar and CoT steps do not bring noticeable benefit. Nonetheless, [20] found that LLMs have measurable capacity to learn constructed languages from grammar descriptions and fine-tuning on CoT greatly improves it, but generalizability is limited for complex linguistic phenomena. Other approaches using LLMs to process low/no-resource languages include combining an LLM with rules [21], [22] or finite-state transducers [23].

Orthogonal to above approaches, a recent line of work explores fine-tuning via reinforcement learning (RL). Building on RLVR [12], [24] and GRPO [25], several works have transferred this paradigm to MT despite the absence of a single “correct” output. [26] introduce MT-R1-Zero, the first R1-Zero-style adaptation for MT, using a hybrid BLEU + COMET-Kiwi reward under GRPO; [27] pair COMET-based rewards with format signals to incentivize translation reasoning, while [28] and [29] move beyond reference-based metrics via trajectory-level generative or self-rewarding signals. [30] earlier showed that quality-estimation models provide a data-efficient reward for MT RLHF. Closer to the low-resource regime, [31] use RL to teach an LLM to consult a bilingual dictionary as an external tool during Spanish–Wayuunaiki translation. and [32] propose self-supervised round-trip RL with intrinsic chrF++ rewards for Aymara, Friulian, and Wolof. However, existing RL-for-MT methods optimize translation quality directly from source-target pairs, leaving open how RL can instead teach models to better exploit in-context linguistic resources.

Our setup can be naturally framed as meta-learning: rather than optimizing for any single translation instance, we train the model to make better use of a contextual “support set” of linguistic resources at inference time. This perspective has a long history in low-resource MT, where [10] adapted MAML [9] to learn initializations that transfer quickly to new language pairs from limited parallel data. With the rise of in-context learning, [33] reframed few-shot learning as implicit meta-learning over the pre-training distribution, and a subsequent line of work has sought to make this capability explicit: MetaICL [34] meta-trains on a distribution of tasks formatted as in-context demonstrations to amplify few-shot performance, in-context tuning [35] similarly optimizes models to condition on examples rather than to fit them, and [7] give a theoretical account of what transformers can learn in-context. Unlike these supervised meta-learning approaches, we use RLVR to teach the model to exploit qualitatively heterogeneous in-context resources better for low-resource translation.

3 Methodology↩︎

3.1 Data Curation↩︎

This section describes the construction of the corpus used for our reinforcement learning experiments, covering language selection, parallel-sentence extraction, the dedicated treatment of Romansh, the synthesis of dictionary entries, and the structure of the assembled prompts.

3.1.1 Language Selection and Sources↩︎

We investigate fourteen (very) low-resource languages, grouped by data-sourcing pipeline rather than by language family. The first group comprises eight languages whose parallel data is extracted from Language Science Press grammar books: Choguita Rarámuri, Gyeli, Japhug, Kagayanen, Kalamang, Tuatschin, Ulwa, and Vamale. We note that Tuatschin is genealogically a dialect of Sursilvan and thus belongs to the Romansh continuum; it is grouped here because its data comes from a grammar book rather than from the parallel corpora and dictionaries used for the standardized Romansh idioms below. The second group consists of the six standardized varieties of Romansh—Puter, Vallader, Surmiran, Sursilvan, Sutsilvan, and Rumantsch Grischun—which, despite a shared heritage, diverge enough lexically and orthographically to warrant separate treatment.

The Romansh varieties were selected because they allow studying meta-learning on a spectrum of language relatedness, and due to the availability of dictionary data and grammar books; other languages were selected based on the availability of open-source grammar books. For the non-Romansh languages, candidates were restricted to those with grammars published by Language Science Press under the CC BY 4.0 licence and distributed in both PDF and LaTeX source. The detail is shown in Appendix 8.

3.1.2 Parallel Sentence Extraction↩︎

Due to the lack of parallel corpora available, we extracted translated examples directly from their grammars. An exception are the five Romansh varieties, where we sample a small amount of parallel sentences from the back-translated training data used by [36]. Although the grammar books occasionally contain OCR or LaTeX artefacts, the underlying translations have been verified by an author-linguist. We worked from the LaTeX source rather than the rendered PDF to preserve idiosyncratic orthographies, locating interlinear-gloss examples by pattern-matching the standard \gll and \glt commands as well as user-defined macros. Residual markup was stripped from both sides of each pair.

We retained only pairs with source sentences between 6 and 50 space-delimited words. A stricter filter targeting extraction artefacts (long uppercase sequences, inline numbering, stray digits, ellipses, underscores, residual newlines) was tested but reduced the corpora to as few as 11 sentences for Japhug and 20 for Kalamang—too few for reinforcement learning—so only the length filter was retained. To enable bidirectional training, each pair was duplicated with the source and target swapped.

3.1.3 Romansh Preprocessing↩︎

The Romansh varieties differ in two respects: they possess large, high-quality parallel corpora and dictionaries (grammar is available only for Puter and Vallader), but their reference language is German rather than English. We discarded sentences longer than 50 words from both the training and test splits and, where more than 2,000 pairs remained, randomly sampled 2,000 to keep per-language volumes comparable. All prompt components were translated into German so that experimental conditions remain identical across language groups. Dictionary entries where retrieved using the Rumlem tool [37] based on the Romansh segments.

3.1.4 Synthetic Dictionary Augmentation↩︎

Every language other than Kalamang and the Romansh varieties lacks sufficient dictionary resources, so we generated synthetic entries using an LLM based on parallel data and grammar books in the format of the MTOB benchmark [1]. Two prompt designs were compared, conditioning on the same parallel sentences and grammar excerpts but differing in the dictionary section: v1 explicitly names each source token and shows two working example entries per token, whereas v2 only describes the general entry format without token-specific demonstrations. For each language and translation direction, we generated dictionaries with both variants, used them in prompts over the training data, and retained the variant with the higher chrF on the resulting translations. The final synthetic dictionary is therefore a per-language, per-direction mixture of v1 and v2.

3.1.5 Prompt Composition↩︎

Each prompt is assembled from five components, as summarised in Table 1, and ends with an instruction to produce step-by-step meta-linguistic reasoning before committing to a final translation. Retrieval uses the Longest Common Subsequence (LCS) metric, which outperformed embedding-based similarity in preliminary experiments. Two configurations of the parallel-sentence section—three and five examples—are evaluated independently to study the effect of in-context example count.

Table 2 summarises the composition of our dataset across the four language groups. In total, the corpus spans 18 languages from 10 distinct language families, yielding 32,335 training pairs and 2,699 test pairs. The Romansh varieties contribute the largest share of the training data (21,890 pairs across six varieties translated to and from German), reflecting both the availability of high-quality parallel corpora and the upper bound of 2,000 sampled sentences per variety. The remaining seven low-resource languages aligned with English contribute 9,695 training pairs but are used exclusively for training, since their grammar book-extracted examples are too scarce to be split further. Kalamang\(\leftrightarrow\)EN is treated separately: we adopt the original MTOB split of 750 training and 100 test pairs to ensure direct comparability with prior work, and additionally hold it out together with Sursilvan\(\leftrightarrow\)De and Surmiran\(\leftrightarrow\)De in our training data configuration to probe cross-lingual generalisation within and across families. Finally, four languages out-of-distribution (one per family) are reserved as a test-only EN \(\to\) X benchmark from [38] to assess transfer to languages entirely unseen during RL training, with 100 sentences randomly sampled per direction for each language.

3.2 RL with Linguistic Knowledge in Context↩︎

With the curated data from Section 3.1, we formulate translation as a reinforcement learning task in which the policy is trained to produce translations conditioned on a meta-linguistic context.

3.2.0.1 Task formulation.

Each training instance is a triple \((x, c, y)\), where \(x\) is a source sentence, \(y\) its reference translation, and \(c\) the language-knowledge context assembled as described in Section 3.1, comprising the introductory description, retrieved dictionary entries, parallel sentences, and grammar passages. The policy \(\pi_\theta\) is a large language model that, given the concatenation of \(c\) and \(x\), generates a response containing step-by-step meta-linguistic reasoning followed by a final hypothesis translation \(\hat{y}\). Only \(\hat{y}\) is scored against \(y\); the reasoning trace is left unconstrained.

3.2.0.2 Reward.

We use chrF [39] between \(\hat{y}\) and \(y\) as the reward signal. Since chrF is reported on a 0–100 scale, we rescale it to \([0, 1]\), which is a typical range for reward functions: \(r(\hat{y}, y) = \frac{1}{100}\mathrm{chrF}(\mathrm{hyp}=\hat{y}, \mathrm{ref}=y)\). If the model fails to produce a parsable final translation, the reward is set to zero.

3.2.0.3 Optimization.

We optimize \(\pi_\theta\) with GRPO [25]. For each prompt, \(G\) responses \(\{\hat{y}_i\}_{i=1}^{G}\) are sampled from the current policy and scored independently. Each reward is then standardized within its group, \(A_i = \frac{1}{\sigma_G}(r_i - \mu_G)\), where \(\mu_G\) and \(\sigma_G\) are the mean and standard deviation of \(\{r_i\}_{i=1}^{G}\), yielding a group-relative advantage that requires no separate value model. The policy is updated with the standard PPO-style clipped objective using \(A_i\), regularized by a KL penalty against a frozen reference policy to limit drift from the initial model. The prompt fed to the policy includes meta-linguistic information about the low-resource language grammar rules, dictionary entries, and parallel sentences, together with the source sentence, allowing the translation quality reward to shape how the policy makes use of these resources when generating \(\hat{y}\).

3.5pt

4 Experiments↩︎

5pt

4.1 Experimental Setup↩︎

4.1.0.1 Models and training.

We fine-tune two backbones, Qwen3-4B-Base and Llama-3.2-3B-Instruct, with both SFT and RL. SFT and RL share an identical prompt format—the full retrieval context from Section 3.1, with two LCS-retrieved dictionary entries per source token, three or five parallel sentences, and two grammar passages where available—and differ only in the supervision signal: SFT minimises cross-entropy against the gold reference, whereas RL optimises chrF reward via GRPO, as described in Section 3.2. All models are trained for one epoch on the 22 training directions, which exclude the two Romansh varieties (SursilvanGerman, SurmiranGerman) held out for the Similar evaluation setting. SFT uses an effective batch size of 128 and RL a batch size of 64. Remaining hyperparameters are listed in Appendix [app:hparams].

4.1.0.2 Evaluation data.

For Romansh–German evaluation, we use the WMT24++ benchmark [40], with each language variety evaluated in the X\(\to\)De direction. For the five unseen languages, the test data come from two sources. Kalamang (100 pairs in each direction) uses the original MTOB benchmark split [1], based on the Kalamang grammar book [13]. The remaining four languages (Dinka, Wolof, Guarani, Kachin, 100 pairs each in the EN\(\to\)X direction) use the test sets from [38].

4.1.0.3 Test-time conditions and metric.

Each checkpoint is evaluated under two prompt context conditions to separate retrieval-dependent skills from those internalized into the policy: full (same prompt as training) and none (only the language description and translation instruction). No model is trained without retrieval. We report chrF results on the 0–1 scale.

4.2 Comparison of SFT and RL↩︎

As shown in Table [tab:main], we evaluate three conditions of increasing difficulty: (1) seen languages Romansh language varieties seen during training, (2) similar held-out Romansh varieties from the same family, and (3) five unseen languages from unrelated families. Table [tab:main] reports results under both full retrieval context and no context (task instruction only).

4.2.0.1 SFT is stronger on seen languages.

On seen Romansh\(\to\)German (averaged over four trained language varieties), Qwen SFT scores 0.60 versus RL’s 0.52; LLaMA SFT scores 0.56 versus RL’s 0.48. A similar gap holds on the held-out Romansh varieties (Qwen SFT 0.55 vs.RL 0.48), suggesting that SFT’s advantage extends within the language family. This is expected: SFT trains on gold translations to fit word-level alignment, whereas RL receives only a sentence-level chrF reward.

4.2.0.2 RL is stronger on unseen languages.

On five unseen languages from different families (Kalamang, Dinka, Wolof, Guarani, Kachin), the ranking reverses. Qwen RL averages 0.27 across these languages, compared to 0.09 for SFT and 0.18 for the untuned base model. Llama-3.2 shows a consistent pattern (RL 0.24 vs.SFT 0.09 vs.base 0.14). The per-language columns in Table [tab:main] show that RL outperforms SFT on each of the five languages individually. Notably, SFT scores below the base model on these languages, suggesting that supervised training on the Romansh directions may reduce the model’s ability to use in-context resources for unfamiliar languages.

4.2.0.3 Context removal clarifies the source of each method’s strength.

Stripping retrieval context at test time (“no context” rows) reveals different behaviours:

• Seen: SFT retains much of its performance without context (Qwen SFT: 0.60\(\to\)​0.46), indicating that it stores language-specific mappings in the weights. RL drops more (0.52\(\to\)​0.30), consistent with greater reliance on context.

• Unseen languages: Without context, all models perform significantly worse.

• The role of context: RL’s advantage on unseen languages (0.27 vs.) appears only when retrieval context is available. This suggests that RL’s generalization is tied to its use of in-context resources rather than to language-specific knowledge acquired during training.

Taken together, the results point to a trade-off between the two training objectives. SFT fits the training languages more tightly but appears to reduce context utilization for new languages. RL fits training languages less precisely but develops a more transferable ability to translate through the provided context: dictionary entries, parallel sentences, and grammar passages, which generalizes to languages from unseen families when such resources are available at test time.

4.3 Ablation study with each role in context↩︎

To understand the contribution of each retrieval component, we train five matched RL runs on Qwen3-4B-Base. In each run, we remove one component from both training and test prompts, so the policy never sees a component at training that will be absent at inference. Results are shown in Table 3.

Among the three components, the bilingual dictionary has the largest impact. Removing it causes a drop of 8.4 chrF on seen Romansh (Puter/Vallader: \(0.5324 \to 0.4483\)) and 8.4 on En\(\to\)Kalamang (\(0.3464 \to 0.2626\)). This is expected because the dictionary provides direct word-level grounding that the other two components cannot fully replace.

Parallel sentences rank second. Removing them has only a moderate effect on seen Romansh (\(-\)​1.0 chrF on Puter/Vallader), but a much larger effect on OOD Kalamang (\(-\)​7.3 on En\(\to\)Kal, from \(0.3464\) to \(0.2733\)). This gap suggests that for languages far from the training distribution, parallel examples provide a complete example for translation.

Grammar passages contribute the least. Removing them causes the smallest drop across all test sets (\(-\)​0.8 chrF on Puter/Vallader, \(-\)​1.5 on En\(\to\)Kal). The model might be unable to extract useful information from grammar passages.

We also examine whether the benefit of context comes from training or from test time (Figure 1). We cross the two conditions: a model trained without context but evaluated with context (no/full), versus a model trained with context but evaluated without (full/no). Across all test sets, no/full consistently outperforms full/no (e.g., En\(\to\)Kal: \(0.28\) vs.\(0.17\)). This shows that test-time context availability is the dominant factor: even a model trained with RL that never saw retrieval during training can exploit it at inference. However, training with context further amplifies this ability. full/full exceeds no/full by \(+\)​7 chrF on En\(\to\)Kal (\(0.35\) vs.\(0.28\)), confirming that exposure to retrieval during RL teaches the policy a more systematic way to use in-context evidence.

4.4 Analysis of training rewards↩︎

4pt

We run GRPO three times on Qwen3-4B-Base over the training split, varying only the retrieval content of the prompt. Full matches Section 4.1 (dictionary, parallel sentences, and grammar passages, all LCS-retrieved). No-dict drops the dictionary block. Task-only drops all three components. The reward function, optimiser, and training budget are unchanged across runs.

Both panels of Figure 2 rank the three runs in the same order: Full \(>\) No-dict \(>\) Task-only. With the chrF reward, Full reaches 0.68, No-dict reaches 0.62, and Task-only flattens near 0.29 by step 50. The 6-point gap between Full and No-dict isolates the dictionary’s contribution, which is not redundant with parallel sentences. The 33-point gap between No-dict and Task-only shows that without any retrieval, GRPO has nothing language-specific to ground on and saturates near the untuned baseline. On WMT24++, the same order holds, but the differences compress to about 0.06, since the RM weighs fluency more heavily than surface form.

The reward trajectories support the ablation findings from the perspective of training dynamics. The Task-only run plateaus early, confirming that without linguistic grounding, GRPO exhausts its improvement budget within about 50 steps and converges to a policy that relies only on pretraining priors. The No-dict run continues to improve past step 100, showing that parallel sentences alone provide a useful learning signal, but the ceiling is lower. Only the Full run sustains reward growth throughout training. This indicates that the dictionary supplies a complementary gradient signal that keeps the policy improving even after the parallel-sentence signal saturates. Together with the ablation results, the reward analysis shows that each retrieval component contributes an additive learning signal, with the dictionary providing the strongest boost and grammar the weakest.

4.5 Case study↩︎

Table [tab:case-study] illustrates this gap with two examples. Without retrieved context, both SFT and RL produce fluent English that recognizes the person name but is semantically unrelated to the source. With a dictionary and parallel-sentence context, RL produces near-perfect translations: in (1) it exactly matches the reference, and in (2) it correctly captures both “bathe” and “cold”. SFT, by contrast, captures only partial meaning—rendering “bathe” as “wash his hands” and dropping “cold” entirely.

5 Conclusion↩︎

In this paper, we propose a reinforcement learning approach for low-resource translation that learns to exploit in-context linguistic information. Using translation quality directly as the reward signal, our empirical results demonstrate that RL-trained LLMs generalize substantially better to unseen languages than their SFT counterparts, and genuinely learn to leverage linguistic context rather than memorize the specific languages seen during training. Our work offers a new perspective on low-resource translation by uniting the complementary strengths of in-context learning and reinforcement learning.

Limitations↩︎

We report chrF++ but do not conduct human evaluation, which would give a sharper picture of fluency and adequacy in low-resource settings; we view the automatic numbers as a reliable signal of relative improvement across methods and leave human evaluation to future work. While our method generalizes to unseen languages far better than SFT, absolute performance on unseen languages still lags behind that for higher-resourced languages, indicating a clear headroom for richer in-context evidence and stronger context-utilization signals.

Acknowledgments↩︎

HH, JV and RS acknowledge funding by the Swiss National Science Foundation (project InvestigaDiff; no. 10000503). We thank RTR and Fundaziun Patrimoni Cultural RTR for their support, and Lia Rumantscha and Uniun dals Grischs for contributing dictionary data.

6 Hyperparameters↩︎

We perform full-parameter fine-tuning with DeepSpeed ZeRO-2 [41], using a learning rate of \(1\mathrm{e}{-5}\) with a cosine scheduler and 10% warmup, an effective batch size of 128 (4 per device \(\times\) 8 gradient accumulation steps). For RL, we use GRPO with chrF/100 as the outcome reward, a learning rate of \(1\mathrm{e}{-6}\), batch size 64, \(n=8\) rollouts per prompt at temperature 1.0, clip ratio 0.25, KL coefficient \(1\mathrm{e}{-4}\), and entropy coefficient \(1\mathrm{e}{-3}\); training uses FSDP with SGLang (TP=4) for rollout and all ablation variants (nodict, nogram, nosent, taskonly) share the same hyperparameters.

7 LLM Usage Declaration↩︎

We used LLM in this research for two purposes. First, during manuscript writing, we used AI to edit language and polish the overall text. Second, we used GPT-5 mini to generate synthetic bilingual dictionary entries for languages lacking sufficient dictionary resources, as described in Section 3.1.4. Since none of the authors is a speaker of the target languages, these synthetic entries were not manually validated for linguistic correctness; instead, we performed an indirect, utility-based selection between two prompt variants (v1 and v2) by comparing downstream chrF on the resulting translations and retaining the better-performing variant per language and direction.

We declare that ideation, methodology, experiment execution, and analysis are the original work of the authors. All AI-assisted text editing has been carefully reviewed by the authors to ensure accuracy.

8 Grammar Books Used For Data Curation↩︎

4pt

References↩︎