1 Introduction↩︎

Adapting a multilingual encoder to a new language and a new task at the same time is the operational reality of low-resource NLP. The standard pipeline continues pretraining on monolingual text and then fine-tunes on labeled task data, which requires a separate end-to-end run for every language–task pair and places the burden of generalisation on one small gold set, often only \(\sim\) \(10^2\) labeled examples. Model merging suggests a cleaner decomposition: train reusable delta updates independently and combine them in weight space [1]–[5]. Existing work has mostly studied composition across tasks within one language. We ask whether the same idea can work across the language and task axes simultaneously: train \(\Delta_L\) on monolingual text and \(\Delta_T\) on labeled English data, then recombine them without a joint fine-tuning run. The question is not whether merging is universally better than fine-tuning; it is which composition rule for \((\Delta_L,\Delta_T)\) preserves the right signal for each kind of task.

1.0.0.1 Approach.

DeltaMerge-LowRes composes a language delta \(\Delta_L\) (trained on unlabeled in-language text via MLM or span corruption) with a task delta \(\Delta_T\) (trained on English-labeled data). We compare four composition rules: additive (\(W \!\gets\! W_0 + \alpha\Delta_L + \beta\Delta_T\)), activation-guided (per-layer scaling fitted from in-language activations), sparsity-aware (top-\(k\) magnitude masking on the union of \(\Delta_L\) and \(\Delta_T\)), and a novel cross-axis TIES. Cross-axis TIES takes the trim, elect, and merge pipeline of [2] and applies it to the \((\Delta_L,\Delta_T)\) pair instead of to two task deltas. As a control, we evaluate task-only adaptation, which uses \(\Delta_T\) alone.

Figure 1 summarizes this decomposition. The key design choice is that \(\Delta_L\) and \(\Delta_T\) are trained once from different supervision sources, then recombined by a dataless rule before evaluation on a target language–task pair. This lets us compare composition rules while holding the underlying language and task updates fixed.

1.0.0.2 Empirical scope.

We evaluate four task families, classification, NER, extractive QA, and summarisation, on Hausa, Swahili, Yoruba (encoder tasks) plus Amharic (summarisation), using XLM-R-base [6] and mT5-base [7]. The task-delta gold budget is \(b{=}256\). In total, we collect \(158\) (language, task, method, seed) evaluation cells, each with a \(10{,}000\)-sample paired bootstrap. The scope is deliberately an internal comparison of five recipes built on identical \((\Delta_L,\Delta_T)\) tensors; this design isolates the composition rule from the delta parameterisation.

1.0.0.3 Findings.

The results are task-structured rather than uniform (Table 2; Figs. 2, 3). Cross-axis TIES is most useful when the target task needs the language and task signals to cooperate throughout generation or extraction. On summarisation, it reaches chrF \(18.59\), compared with \(13.80\) for task-only and \(12.64\) for additive merging, and wins in \(3/4\) languages by \(+4\) to \(+7\) chrF. This margin is \(5{-}10\times\) the per-cell CI width. The same pattern appears, more modestly, on extractive QA: cross-axis TIES improves F1 by \(+2.32\) and exact match by \(+2.91\) over task-only, with sparsity-aware merging close behind (\(+1.67\) F1, \(+2.38\) EM). Classification tells a different story. There, sparsity-aware merging does not raise macro-F1, but it makes the model better calibrated, reducing ECE from \(13.81\) to \(8.86\) (\(-36\%\) relative) at parity macro-F1. NER is similarly mixed: composition shifts the model toward recall (\(+3\) pts) and away from precision (\(-2\) pts), leaving entity-F1 unchanged.

1.0.0.4 Contributions.

Our contributions are: (i) we define and evaluate cross-axis TIES, a TIES-style merge applied to a language–task delta pair, and identify it as the strongest rule for generation in our setup; (ii) we run a tightly controlled five-recipe comparison with \(158\) bootstrapped evaluation cells; and (iii) we show that sparsity-aware composition Pareto-improves classification calibration. Every numerical claim is mapped to a released JSON trace through the claim ledger.

2 Method↩︎

2.1 Setup↩︎

Let \(\theta_0 \in \mathbb{R}^d\) denote the parameters of a multilingual pretrained encoder (or encoder–decoder). A delta is an additive update \(\Delta \in \mathbb{R}^d\) that yields the adapted model \(\theta_0 + \Delta\). We treat language adaptation and task adaptation as two reusable objects. For a target language \(\ell\) and target task family \(t\), the language delta \(\Delta_L^{(\ell)} = \theta_L^{(\ell)} - \theta_0\) comes from continued pretraining on unlabeled text in \(\ell\), while the task delta \(\Delta_T^{(t)} = \theta_T^{(t)} - \theta_0\) comes from fine-tuning on labeled English data for task \(t\). The composed model has parameters \[\theta_{\ell,t} \;=\; \theta_0 \;+\; f\!\left(\Delta_L^{(\ell)}, \Delta_T^{(t)}\right), \label{eq:compose}\tag{1}\] where \(f\) is one of the composition rules below. \(\Delta_L\) and \(\Delta_T\) are trained from disjoint data sources and reused across \(\ell,t\) combinations; the only per-pair operation is the dataless composition. Each delta is realised as a LoRA-style rank-\(16\) update \(\Delta = BA\) and materialised as a dense tensor before composition. For an encoder base we train \(\Delta_L\) with MLM on monolingual text; for an encoder–decoder base, we use span corruption at \(15\%\) noise density.

2.2 Composition rules↩︎

2.2.0.1 Additive.

\[f_{\text{add}}(\Delta_L,\Delta_T) \;=\; \alpha\,\Delta_L \;+\; \beta\,\Delta_T. \label{eq:add}\tag{2}\] This is the merging analogue of task arithmetic [1], applied across the language and task axes.

2.2.0.2 Activation-guided.

We replace the global \((\alpha,\beta)\) of Eq. 2 with per-layer scalars \((\alpha_\ell, \beta_\ell)\) computed from \(\lvert\mathcal{P}\rvert=50\) unlabeled probe sentences. The intuition is simple: if \(\Delta_L\) produces a larger activation shift than \(\Delta_T\) at a layer, that layer is more language-sensitive under the probes and should accept more of \(\Delta_L\); if \(\Delta_T\) dominates, the layer should accept more of \(\Delta_T\). Concretely, we measure the mean post-activation shift of layer \(\ell\) under the language delta as \[\bar{n}_\ell^{L} \;=\; \frac{1}{|\mathcal{P}|} \sum_{x\in\mathcal{P}} \left\lVert h_\ell^L(x) - h_\ell^0(x)\right\rVert_2,\] with \(\bar{n}_\ell^{T}\) defined analogously for \(\Delta_T\). We then set \[\alpha_\ell \;=\; \frac{\bar{n}_\ell^L}{\bar{n}_\ell^L + \bar{n}_\ell^T}, \quad \beta_\ell \;=\; \frac{\bar{n}_\ell^T}{\bar{n}_\ell^L + \bar{n}_\ell^T}, \label{eq:actguide}\tag{3}\] so \(\alpha_\ell + \beta_\ell = 1\) by construction. This rule requires two forward passes per probe sentence (one through \(\theta_0 + \Delta_L\), one through \(\theta_0 + \Delta_T\)), measures activation shifts at the post-activation hidden state, and does not require any optimisation step. It is not an optimal fit of the merged activation under a supervised loss; it is a per-layer relative-shift weighting, and we report it as such. Empirically, this heuristic performs worse than sparsity-aware and cross-axis TIES on every task family in our matrix (Table 2).

2.2.0.3 Sparsity-aware.

Sparsity-aware composition keeps only the top-\(k\%\) entries by the joint magnitude \(|\Delta_L| + |\Delta_T|\): \[f_{\text{sparse}}(\Delta_L,\Delta_T) \;=\; M_k \;\odot\; (\alpha\,\Delta_L \;+\; \beta\,\Delta_T), \label{eq:sparse}\tag{4}\] where \(M_k \in \{0,1\}^d\) is the top-\(k\) magnitude mask, \(\alpha,\beta\) are inherited from Eq. 3 , and \(k{=}20\%\) throughout. The rule plays a role similar to the trim step of TIES [2], but it acts across the language \(\times\) task plane and uses joint magnitude rather than per-delta magnitude.

2.2.0.4 Cross-axis TIES.

Cross-axis TIES applies the TIES steps of trimming, sign election, and merging [2] to the \((\Delta_L,\Delta_T)\) pair. We trim each delta independently to its top \(20\%\) magnitudes, elect a sign per coordinate from the surviving entries (ties broken by magnitude), and merge only entries whose sign matches the elected sign. The output is scaled by the activation-guided \(\alpha,\beta\) for a controlled comparison.

2.2.0.5 Why cross-axis TIES is not just two-task TIES.

Original TIES [2] elects signs over deltas trained on the same axis, such as multiple task deltas in one language; the election arbitrates between task gradients competing for one parameter. In our setting, \(\Delta_L\) and \(\Delta_T\) are trained on disjoint axes and corpora, so sign disagreement has a different interpretation: it marks coordinates where the language-fluency and task-extraction objectives conflict. The election step becomes a structural filter asking whether a coordinate helps both axes simultaneously. This predicts the empirical ordering SUM \(\gg\) QA \(>\) CLS/NER (Table 2): generation needs both axes at every output token, QA needs both for span localisation and extraction, while CLS/NER read in-language tokens but produce English-trained head decisions. This argument is consistent with our data but not independently testable from these experiments alone, so we treat it as the operative hypothesis. Sparsity-aware merging differs in two choices: it admits both signs (no election) and trims on \(|\Delta_L|+|\Delta_T|\) rather than per-delta.

2.2.0.6 Task-only baseline.

For comparison, we evaluate \(\theta_0 + \Delta_T\) alone, with no language delta. This is the natural reference for the central question: once a composition rule exists, does adding \(\Delta_L\) actually help?

2.3 Inference and deployment↩︎

After composition, the model has the same architecture and parameter count as \(\theta_0\); no adapter is plugged in at inference. The reproducibility artifact for every reported number is the per-pair trace JSON under Delta/results/, which records \(\ell\), \(t\), rule, \((\alpha,\beta,k)\), seed, score, and a delta-tensor hash.

3 Experimental Setup↩︎

3.0.0.1 Scope and design.

We compare five recipes that share the same \(\Delta_L\) and \(\Delta_T\) tensors and differ only in how they are combined: task-only (no \(\Delta_L\)), additive, activation-guided, sparsity-aware, and cross-axis TIES (§2.2). The contribution tested here is the composition rule, not whether merging beats fine-tuning in general. Holding the delta parameterisation, delta training data, training compute, and evaluation protocol fixed across rules makes observed differences attributable to the rule itself. This is the standard design for comparative studies of merging rules [1]–[5]; like that prior work, we do not run Full FT / LoRA / adapter / Continue-PT \(+\) LoRA controls under the same protocol in this version. We therefore separate the question this matrix answers (“which composition rule of \((\Delta_L,\Delta_T)\) is best?”) from the question it does not answer (“does merging beat fine-tuning?”). The latter requires a paired study at the same compute budget and is left to future work.

3.0.0.2 Bases.

We use XLM-R-base [6] for classification, NER, and QA, and mT5-base [7] for summarisation. All deltas are LoRA-style rank-\(16\) updates on attention and MLP projections; before composition, each dense delta tensor \(\Delta = BA\) is materialised.

3.0.0.3 Languages and corpora.

The encoder tasks cover Hausa (hau), Swahili (swa), and Yoruba (yor); summarisation additionally includes Amharic (amh). Monolingual text comes from public crawl-derived corpora [8].

3.0.0.4 Task families.

The four task families span a spectrum of language–task interaction, from input-only classification to per-token generation:

• Classification (cls): target-language news topic, MasakhaNEWS [9], supervision from AG News (App. 9), macro-F1. Language signal consumed at the input only.

• NER (ner): token-level entity tagging, MasakhaNER 2.0 [10], supervision from CoNLL-03 [11], entity-F1 (precision/recall in Table 4). Both signals run per token, small label set.

• Extractive QA (qa): AfriQA-style [12], supervision from SQuAD [13], F1 and exact match. Span localisation requires both signals to interact.

• Summarisation (sum): in-language abstractive summarisation, XL-Sum [14], supervision from English XL-Sum, chrF [15]. Generation requires both signals at every output token: fluent in-language continuation from \(\Delta_L\) and summarisation structure from \(\Delta_T\).

For each language–task pair, we evaluate on the released test split (sizes \(253\)–\(987\) examples; see Table 15). The gold training budget for the task delta is \(b{=}256\) examples drawn from the English-labeled source.

3.0.0.5 Composition recipes.

The five rules are defined by Eqs. 2 –4 and the cross-axis TIES variant in §2.2. The hyperparameters \(\alpha,\beta\) come from the activation-guided closed form on \(50\) unlabeled probe sentences, so no labeled in-language data is used for composition. The sparsity mask uses a fixed default \(k{=}20\%\).

3.0.0.6 Seeds.

For encoder tasks (cls, ner, qa), task-only, additive, sparse, and cross-axis TIES have three seeds (0, 1, 2), while activation-guided has two seeds (0, 1). For summarisation, sparse and cross-axis TIES have three seeds; additive and task-only have seed \(0\) only; activation-guided was not run on the encoder–decoder base. We report mean-over-seed numbers throughout and flag single-seed cells in every table. In total, the matrix contains \(158\) evaluated cells.

3.0.0.7 Statistical reporting.

Every evaluation file is accompanied by a \(10{,}000\)-sample paired bootstrap over the test split. Encoder tasks resample examples; summarisation resamples precomputed chrF statistics. Per-cell \(95\%\) CIs are reported in Tables 8–11 in the appendix, and mean CI half-widths per task are listed in Table 1.

3pt

3.0.0.8 Training and tuning.

All deltas are trained with AdamW at learning rate \(5\!\times\!10^{-5}\), batch size \(16\), and sequence length \(256\) (cls/ner/qa) or \(512\) (sum). The activation-guided closed form uses a \(50\)-sentence probe set per language. We did not run a per-cell sparsity sweep; the default \(k{=}20\%\) is fixed throughout.

3.0.0.9 Hardware and reproducibility.

Runs use A100 40GB and G6e (NVIDIA L40S) instances. Every numerical claim in the paper is backed by a released per-evaluation JSON trace and indexed in a claim ledger that maps each table and figure cell to its trace; see App. 8 for the artifact layout.

4 Results↩︎

We compare five recipes that share the same \(\Delta_L\) and \(\Delta_T\) and differ only in how they are combined: task-only, additive, activation-guided, sparsity-aware, and cross-axis TIES. The results are not a single monotone ranking. Section 4.1 reports aggregate task scores; §4.2 zooms in on summarisation and QA, where composition gives the clearest positive effects; §4.3 reports CLS and NER, where macro-F1 is statistically tied but secondary metrics move. A per-cell win matrix is in App. 5.

4.1 Main comparison↩︎

Table 2 reports per-task aggregate scores. The gains from cross-axis TIES concentrate on generation (SUM \(+4.79\)) and span extraction (QA \(+2.32\)); macro-F1 on CLS/NER remains statistically tied with task-only. The per-language standard deviations are large because they pool languages with very different absolute scores (for example, CLS macro-F1 ranges from \(38\) on yor to \(62\) on swa). Per-cell bootstrap CIs (Table 1) are therefore the relevant noise band for individual comparisons.

4.2 Composition wins: SUM and QA↩︎

4.2.0.1 Summarisation.

Figure 2 (a) plots per-language chrF. This is the setting where the two axes most visibly need each other: cross-axis TIES wins three of four languages (amh, hau, swa) by \(4\)–\(7\) chrF over additive and over task-only. The Amharic cell is the sharpest example. \(\Delta_T\) alone yields chrF \(2.14\) on mT5-base because the English-trained task delta does not transfer in any usable way to Ge’ez script without language adaptation. With cross-axis TIES, the score recovers to chrF \(11.58\), a \(5.4{\times}\) relative gain. Per-cell bootstrap CIs are about \(1\) chrF wide (Table 1), so each per-language win clears the test-set noise band by \(5{-}10\times\).

Important seed caveat. Bootstrap CIs quantify test-set sampling uncertainty only; they do not capture training-seed variance. On summarisation we have \(3\) seeds for cross-axis TIES and sparsity-aware but only seed \(0\) for task-only and additive. The headline chrF gap is therefore robust to test-set resampling but not yet checked against training-seed variance of the comparators. The conservative reading is that the summarisation result is a strong directional finding from a \(3{\times}1\) comparison rather than a fully matched \(3{\times}3\) one. We flag this in the tables; the key follow-up is \(\geq 3\) task-only/additive seeds.

4.2.0.2 Why Yoruba behaves differently.

Yoruba is the one summarisation cell where task-only leads cross-axis TIES, by \(1.07\) chrF. Two features explain the exception. First, additive merging hurts on Yoruba (\(10.62\) vs.\(18.91\) task-only), making it the only language where adding \(\Delta_L\) degrades chrF. Second, Yoruba has the smallest monolingual corpus (\(\sim\) \(5\)M tokens vs.\(\sim\) \(25\)M for swa), so \(\Delta_L^{\text{(yor)}}\) is plausibly the noisiest language delta. When \(\Delta_L\) is noisy and the base (\(\Delta_T\)-adapted mT5) is already producing useful Yoruba summarisation, the cross-axis TIES sign-election trims wrong-sign \(\Delta_L\) coordinates and recovers \(+7.22\) chrF over additive, but the rescued model still trails task-only. We read Yoruba as a stress test: cross-axis TIES rescues a degenerate additive merge but does not exceed an already-strong task-only baseline.

3pt

4.2.0.3 Extractive QA.

Figure 2 (b) reports F1 and EM averaged over the three encoder languages. Cross-axis TIES tops both F1 (\(+2.32\) over task-only) and EM (\(+2.91\)), with sparsity-aware merging close behind. We frame this as an aggregate trend, not a per-cell effect: the per-cell bootstrap CI half-width on QA is \(4\)–\(4.5\) F1 (Table 1), larger than the \(+2.32\) aggregate gain, so no individual (lang, seed) cell carries the result on its own. The signal comes from directional consistency rather than from cell-level statistical resolution: cross-axis TIES wins F1 in \(5/9\) aligned (lang, seed) cells and EM in \(7/9\) (Tables 10, 14). The Yoruba QA cell is the one place where activation-guided is the per-language winner (\(19.64\) F1 over \(14.15\) for task-only), at two seeds only.

4.3 Where macro-F1 ties: CLS and NER↩︎

On CLS and NER, the spread of the four non-degenerate rules and task-only is \(0.8\) macro-F1 and \(0.55\) entity-F1 respectively, both inside the per-cell bootstrap CI width. We therefore do not claim a headline improvement on these task families. The secondary metrics, however, show that composition is not inert.

4.3.0.1 NER trades precision for recall.

Sparse and cross-axis TIES lift entity recall by \(\sim\) \(3\) points over task-only (\(80.47\) and \(80.18\) vs.\(76.97\)) and pay \(\sim\) \(2\) points of precision (\(56.34\) and \(56.32\) vs.\(58.60\)); entity-F1 is unchanged (Table 4). The pattern is consistent across all three languages (Table 13). Our interpretation is that the language delta sensitises the model to in-language entity spans, raising recall, but does not improve boundary and type decisions enough to prevent additional false positives.

4.3.0.2 CLS Pareto-improves calibration.

Sparsity-aware merging cuts ECE from \(13.81\) (task-only) to \(8.86\) at parity macro-F1, a \(36\%\) relative reduction. Cross-axis TIES is close behind. Thus, even where accuracy does not move, the composition rule changes confidence quality. The Pareto plot and full calibration table are in §5.1.

4.4 Per-cell win matrix↩︎

3pt

Cross-axis composition wins where the task itself requires language-aware generation or span localisation (sum, qa); on CLS and NER it is approximately inert on the headline metric and active on secondary metrics (calibration, recall).

5 Analysis↩︎

The aggregate table shows where composition helps, but the mechanism is easier to see through diagnostics. We report three of them: the classification calibration Pareto-improvement (§5.1), the cell-by-cell paired win analysis (§5.2), and a scope clarification on what the data does and does not establish (§5.3).

5.1 Calibration: a Pareto-improvement↩︎

3pt

Sparsity-aware merging cuts ECE from \(13.81\) to \(8.86\) at parity macro-F1, the cleanest Pareto improvement in our study (Fig. 3; Table 6). Cross-axis TIES is close behind; Brier and NLL follow the same ordering. Mechanism. The top-\(k\) mask retains the dominant entries of \(|\Delta_L|+|\Delta_T|\) and zeros the low-magnitude residue. We expect the zeroed entries to be exactly the small positions where \(\Delta_L\) and \(\Delta_T\) are most likely to disagree in sign; without the mask, these conflicting updates can push logits toward mis-calibrated extremes. The same explanation is consistent with cross-axis TIES, whose sign-election step gives stronger conflict resolution and the second-best calibration.

5.2 Shared-seed paired analysis↩︎

3pt

Table 7 reports the cross-rule comparison restricted to triples present under both methods. The cross-axis TIES lead is concentrated on summarisation and QA: per-task paired deltas are SUM \(+4.79\) (\(3/4\) paired cells; single-seed comparators), QA \(+1.20\) (\(5/9\)), CLS \(-0.40\) (\(3/9\)), and NER \(-0.49\) (\(3/9\)). The CLS and NER deltas are mildly negative on macro-F1, which matches the earlier pattern: composition does its visible work on calibration and recall, not on the headline F1 metric (Tables 6, 4). The full per-task decomposition is in App. 10.

5.3 What the data does and does not establish↩︎

5.3.0.1 Supported.

(i) Cross-axis TIES is the strongest summarisation rule we tested (\(3/4\) per-language wins by \(4\)–\(7\) chrF, each \(5{-}10\times\) the per-cell CI); (ii) cross-axis TIES and sparsity-aware merging improve QA F1 and EM over task-only as an aggregate trend across the three encoder languages; (iii) sparsity-aware is a Pareto improvement on classification calibration at parity macro-F1; and (iv) composition shifts the NER profile toward recall while entity-F1 is unchanged.

5.3.0.2 Not supported by the current matrix.

The current matrix does not support claims that any rule dominates uniformly cell by cell; comparisons against LoRA, Full FT, joint FT, Continue-PT \(+\) LoRA, MAD-X-style adapter routing, or external merging baselines (out of scope, §¿sec:sec:limitations?); claims about activation-guided as a strong rule (two seeds, weak NER); best-rule-per-test-cell headlines (Table 5 is diagnostic); or generalisation to non-LoRA delta parameterisations and non-XLM-R/mT5 bases.

6 Related Work↩︎

6.0.0.1 PEFT and model merging.

LoRA [16], [17], adapters [18]–[20], prompt/prefix tuning [21], [22], and sparse fine-tuning [23]–[25] reduce trainable parameters but still optimise language and task jointly. Model merging takes a different route: task arithmetic [1] adds independently-trained deltas in weight space; TIES [2] resolves interference between deltas by trimming low-magnitude entries, electing a sign per parameter, and merging only agreeing entries; model soups [3], [26], Fisher-weighted [4], and dataless merging [5] are further variants. Most of this work targets composition across tasks within a single high-resource language. We reuse the TIES steps of trimming, sign election, and merging across the language \(\times\) task plane (cross-axis TIES), and we add a sparsity-aware rule on the union of a language and a task delta rather than two task deltas. This paper does not benchmark against task arithmetic, plain TIES, or model soups under the same evaluation protocol; the comparison in Table 2 is internal to recipes built on the same \((\Delta_L,\Delta_T)\) pair.

6.0.0.2 Cross-lingual transfer with modular components.

MAD-X [19], AdapterFusion [20], Hyper-X [27], and composable sparse fine-tuning [25] combine per-language adapters with task adapters at inference. DeltaMerge-LowRes differs in two respects: the language and task components live in the same parameter space as the base (composition is addition in weight space, not routing in activation space), and we ask whether sparsity-aware or cross-axis sign-election improves over plain additive merging. The answer is task-dependent (Table 2). Weight-space composition adds no inference parameters; activation-space routing trades extra parameters for the absence of in-weight interference.

6.0.0.3 Benchmarks.

We evaluate on African-language benchmarks where labels are scarce but monolingual text is reachable: MasakhaNEWS [9], MasakhaNER 2.0 [10], AfriQA [12], and XL-Sum [14], on XLM-R [6] and mT5 [7]. A natural single-model alternative is to start from an Africa-adapted encoder (AfroXLMR, AfriBERTa) and run task-only adaptation, folding the language signal into the base. That answers a different question; we instead ask whether a language signal trained after pretraining [8] can be added back into a generic multilingual base at composition time. An unmerged Continue-PT \(+\) LoRA pipeline at matched compute is the most direct fairness baseline and the recommended follow-up (§¿sec:sec:limitations?).

6.0.0.4 Positioning.

Our contribution is not a new PEFT module and not a new benchmark: we define cross-axis TIES, then run a tightly-controlled comparison of five recipes that share identical \(\Delta_L\) and \(\Delta_T\) tensors. This isolates the composition rule from the delta parameterisation within the parameterisation we test (LoRA rank-\(16\)). Where results overlap with single-axis merging studies, the comparison points the same way: sparsity helps when low-magnitude sign-conflicting entries interfere. Where they diverge, with cross-axis TIES much stronger than additive on summarisation but marginal on classification macro-F1, the pattern is informative about when language and task signals need to interact tightly.

7 Conclusion↩︎

We asked which weight-space rule best recombines a separately-trained language delta and task delta. Holding \((\Delta_L,\Delta_T)\) fixed, we find that the merge rule matters, not just the decision to merge. Cross-axis TIES applies the trimming, sign-election, and merging steps of [2] to a language–task pair, and it wins where the two axes must cooperate at every output step (SUM chrF \(18.59\) vs.\(13.80\), \(3/4\) languages by \(+4\) to \(+7\); QA F1 \(+2.32\), EM \(+2.91\)). Sparsity-aware merging also Pareto-improves classification calibration (ECE \(-36\%\) at parity F1). When the two deltas are trained on disjoint axes, the sign-election step becomes a structural filter asking whether a coordinate helps both objectives. This view predicts the ordering SUM \(\gg\) QA \(>\) CLS/NER and survives our cleanest stress test, where Yoruba summarisation has an additive loss of \(-8.29\) but TIES recovers \(+7.22\). The most useful follow-up our release enables is a matched-compute comparison against Continue-PT \(+\) LoRA.

Limitations and Threats to Validity↩︎

7.0.0.1 What the matrix cannot establish (baselines).

The five recipes share \((\Delta_L,\Delta_T)\) by design, so this paper cannot say whether cross-axis TIES is better than LoRA at \(b{=}256\) in-language gold, Continue-PT \(+\) LoRA, or joint FT at matched compute. This is the main boundary of the claim. The released \((\Delta_L,\Delta_T)\) pairs make such a fairness study a straightforward follow-up (App. 8); App. 11 places our absolute scores within the published range on the same benchmarks.

7.0.0.2 Seed coverage and multiple comparisons.

Encoder tasks use three seeds for additive/sparse/TIES/task-only and two for activation-guided. On summarisation, only sparse and TIES have three seeds; additive and task-only have seed \(0\) only. Bootstrap CIs quantify test-set uncertainty but not training-seed variance. We therefore treat the summarisation result as a directional finding from a partly \(3{\times}1\) comparison, with the \(3/4\) language win count as the conservative summary. We compare \(4\) rules against task-only on \(4\) tasks (\(16\) aggregate cells); the summarisation headline (\(p \ll 0.05\) per language under per-cell bootstrap, three independent languages) survives Bonferroni correction, but the QA aggregate (\(+2.32\) F1, below the \(\sim\) \(4\) F1 CI half-width) would weaken from “positive aggregate trend” to “directionally consistent” under Bonferroni, matching our framing in §4.2.

7.0.0.3 Other scope items.

We fix \(k{=}20\%\) and do not run a per-cell sparsity sweep. Three to four languages is a small typological sample. We report chrF [15] rather than human judgements; chrF correlates reasonably with human assessment on XL-Sum but is not a substitute, particularly below chrF \(20\). We use one base per task family (XLM-R-base on cls/ner/qa, mT5-base on sum) and one delta parameterisation (LoRA rank-\(16\)), so the rule ranking is conditioned on those choices. We do not verify it at \(7\)B+ scale, under African-pretrained encoders, or under non-LoRA deltas. The task delta also inherits English-style label inventories; the AG News \(\to\) MasakhaNEWS mapping is described in App. 9.

Ethics and Broader Impact↩︎

DeltaMerge-LowRes reduces the marginal cost of adapting a multilingual encoder to a new \((\ell,t)\) pair to one-shot weight composition. The released deltas are \(\sim\) \(6\)MB each and can be combined offline, which makes adaptation easier to reproduce but also easier to deploy without enough task-specific scrutiny.

7.0.0.4 Bias inheritance and interaction.

A delta inherits its training-data biases: \(\Delta_L\) encodes biases of the monolingual corpus, and \(\Delta_T\) inherits English-trained label boundaries (CoNLL-03 entities, AG News topics). The composition rule can interact with these biases in ways the individual deltas do not exhibit. We see three concrete risks. First, the recall-biased NER profile means false-positive entity attribution rises with \(\Delta_L\) strength, potentially over-tagging demographic or regional terms. Second, the AG News \(\to\) MasakhaNEWS class mapping (App. 9) excludes Health, Religion, and Entertainment classes; per-class evaluation of unmapped classes is needed before deployment. Third, Amharic summarisation chrF, while substantially improved, remains at a low absolute level (chrF \(11.58\)), where output may contain hallucinated content. Each \((\ell,t)\) deployment requires independent fairness assessment.

7.0.0.5 Data provenance.

Monolingual corpora are public crawl-derived datasets [8]. English task data uses CoNLL-03 [11], SQuAD [13], AG News [28], and XL-Sum English [14]. Low-resource evaluation uses publicly released MasakhaNER, MasakhaNEWS, AfriQA, and XL-Sum splits. We do not collect human-subject data; all test labels come from existing benchmarks. The artifact is intended for research and is not suitable for safety-critical deployment without domain-specific fairness, robustness, and human assessment.

8 Reproducibility↩︎

This appendix is the reproducibility map for the paper. It describes where each reported number is stored, which run settings are fixed across experiments, and how the uncertainty estimates in the tables are computed.

8.0.0.1 Artifact layout.

The submission package is organized so that every reported aggregate can be traced back to a machine-readable record:

• Delta/results/encoder/: per-evaluation JSON traces for cls, ner, qa on Hausa, Swahili, Yoruba. Each (lang, task, method, seed) tuple has an _eval.json file with the headline metric and an _bootstrap.json file with a \(10{,}000\)-sample paired bootstrap over the test split.

• Delta/results/seq2seq/: per-eval JSON traces for summarisation on Amharic, Hausa, Swahili, Yoruba.

• current_metric_summary.csv (under Delta/results/encoder/): row-per-eval CSV with lang, task, method, seed, metric, and value.

• Delta/results/claim_ledger.csv: the anchor-to-trace mapping used to audit manuscript claims.

• Delta/logs/: training logs for every delta and composition run.

• sections/, appendix/, main.tex, references.bib: manuscript sources.

8.0.0.2 Claim ledger.

Each row of the claim ledger links a manuscript anchor (table, figure, or in-text number) to the JSON trace files that support it. The audited anchors include the main results table, the per-language tables, the calibration table, the NER and QA breakdown tables, and the paired analysis table.

8.0.0.3 Hyper-parameter inventory.

For each run, the trace records the LoRA rank (\(r{=}16\)), \(\alpha,\beta\) from the activation-guided closed form, sparsity \(k\) (default \(20\%\)), learning rate (\(5{\times}10^{-5}\)), batch size (\(16\)), sequence length (\(256\) for cls/ner/qa, \(512\) for sum), and seed (\(\{0,1,2\}\) for the three-seed methods, \(\{0,1\}\) for activation-guided, \(\{0\}\) for summarisation additive/task-only).

8.0.0.4 Hardware.

Runs are split between A100 40GB and G6e (NVIDIA L40S) instances. The hardware used for each evaluation is recorded in the corresponding JSON trace.

8.0.0.5 Bootstrap procedure.

For each evaluation, we run a \(10{,}000\)-sample paired bootstrap over the test split, with random seed \(0\) for the bootstrap RNG. The bootstrap quantifies test-set sampling uncertainty only; it does not capture training-seed randomness. We report seed variability separately through mean-over-seed aggregation and the per-cell tables in App. 10.

8.0.0.6 Quick-start: reproducing Table 2 row by row.

Each row of the main table can be reproduced from a single CSV read with no additional modeling or hidden aggregation:

1. Open current_metric_summary.csv under Delta/results/encoder/.

2. For CLS / NER / QA: filter on task and method; the column is macro_f1 (CLS), entity_f1 (NER), f1 (QA). Multiply by \(100\) for CLS and NER. Take mean and SD across the rows.

3. For SUM: aggregate the matching per-language bootstrap JSON files, named by the pattern ; read observed from each.

The same logic produces the per-language tables (3, 4, 14) after adding the lang filter. The summarisation bar chart (Fig. 2a) plots the observed field from each per-(lang, method, seed) bootstrap JSON.

8.0.0.7 Recommended follow-up baselines.

The most useful experiments enabled by this release, but not run in this paper, are: (i) Target-language LoRA at \(b{=}256\): train a LoRA on \(256\) in-language gold examples per \((\ell,t)\) and score against the same test splits; this is the most direct fairness comparator. (ii) Continue-PT + LoRA: continue pretraining mT5/XLM-R on the same monolingual corpus used for \(\Delta_L\) at the same compute budget, then train LoRA on \(b{=}256\) gold examples; this asks whether the language signal is more useful as a base modification or as a delta. (iii) Sequential adaptation: load \(\theta_0 + \Delta_L\) as the base for \(\Delta_T\) training; this is a non-merging operationalization of the same modular intuition. The released \((\Delta_L, \Delta_T)\) pairs and per-(lang, task, seed) evaluation harness are designed so that each baseline can use the existing claim-ledger and bootstrap protocol without protocol changes.

9 Data sources, splits, and prompts↩︎

This appendix specifies the data interfaces behind the language and task deltas: the monolingual corpora used for language adaptation, the English supervision used for task deltas, and the target-language splits used only for evaluation.

9.0.0.1 Monolingual data per language.

• Swahili (swa): \(\sim\)​25M tokens from public crawl-derived sources [8].

• Hausa (hau): \(\sim\)​15M tokens, same sources.

• Amharic (amh): \(\sim\)​8M tokens.

• Yoruba (yor): \(\sim\)​5M tokens.

We use deduplicated document-level splits and apply standard cleaning: Unicode normalisation, length filtering, and language-ID filtering with fastText [29] at threshold \(0.9\). Estimated post-clean noise, defined as residual non-target-language lines that pass the fastText filter, is below \(3\%\) for swa/hau and below \(6\%\) for amh/yor, based on manual inspection of a \(200\)-line sample per language.

9.0.0.2 Task source data (English).

The task deltas are trained from English source data. Classification uses AG News [28], a four-class news topic dataset (World, Sports, Business, Sci/Tech). NER uses CoNLL-03 [11] with four entity types (PER, ORG, LOC, MISC). QA uses extractive spans from SQuAD v1.1 [13]. Summarisation uses the English split of XL-Sum [14], with chrF as the evaluation metric.

9.0.0.3 Target-language evaluation data.

For evaluation, classification uses MasakhaNEWS-style topic splits [9]; sentiment is not part of our evaluation. NER uses MasakhaNER 2.0 splits [10]. QA uses AfriQA [12]. Summarisation uses the XL-Sum target-language splits [14], evaluated with chrF [15]. We hold out the released test split in each case. The gold budget of \(b{=}256\) is drawn from the English-labeled source for the task delta, not from the target-language test data.

9.0.0.4 Prompts.

We do not use natural-language prompts for the encoder tasks: classification, NER, and QA use task-specific heads on XLM-R. Summarisation uses mT5 text-to-text targets with the prefix ‘‘summarize:’’ in the language-appropriate orthography, following the XL-Sum convention.

9.0.0.5 Label schema mapping.

For classification, AG News uses World/Sports/Business/Sci-Tech, while MasakhaNEWS uses a broader inventory that includes Politics, Sports, Business, Technology, Health, Religion, and Entertainment. We hand-map AG News \(\to\) MasakhaNEWS as World\(\to\)Politics, Sports\(\to\)Sports, Business\(\to\)Business, and Sci-Tech\(\to\)Technology. MasakhaNEWS classes absent from AG News (Health, Religion, Entertainment) are evaluated as zero-prediction for those test examples. This is the most conservative four-to-four matching available: it does not artificially inflate macro-F1 because the unmapped classes are scored against zero predictions. For NER, the inventory is identical (PER, ORG, LOC, MISC). For QA, spans are tokenised by language-specific BPE and aligned via offset matching.

9.0.0.6 Audit fields.

Each per-pair trace records the language ID, task ID, system, seed, composition rule, \((\alpha,\beta,k)\), test-set score, per-instance prediction (where space allows), and a hash of the merged delta tensor. These fields are sufficient to recompute every reported number.

10 Per-cell results↩︎

This appendix exposes the cells that are averaged in the main paper. It is intended to make the aggregate trends auditable: readers can check whether a result is stable across languages and seeds, and can see which comparisons have incomplete seed coverage. Headline metrics include a \(95\%\) paired bootstrap confidence interval from \(10{,}000\) samples over the test split for encoder tasks, or \(10{,}000\) samples over the precomputed chrF statistics for summarisation. Secondary tables aggregate per-cell diagnostics (classification calibration, NER precision/recall, and QA exact match) into per-language means over seeds.

10.1 Per-cell headline metrics with bootstrap CI↩︎

Tables 8–11 list the raw headline cells. A dash marks a cell that was not run under the current compute budget.

3pt

3pt

3pt

3pt

10.2 Classification secondary metrics per language↩︎

Table 12 breaks out the classification calibration numbers from Table 6 by target language. The aggregate pattern also appears at this level: sparse and TIES have the lowest ECE for almost every language. The gap is smallest on Yoruba, where macro-F1 itself is lowest.

3pt

10.3 NER secondary metrics per language↩︎

Table 13 breaks out the NER precision, recall, and token-accuracy numbers from Table 4 by target language. The recall lift from sparse/TIES is largest on Yoruba (\(+10\) recall over task-only), while the precision drop is largest on Hausa (\(-3\) precision). Swahili sits between these two cases.

3pt

10.4 QA secondary metrics per language↩︎

Table 14 breaks out F1 and exact match by language. The largest EM gain from cross-axis TIES is on Swahili (\(+1.92\) over task-only), and the largest sparse gain is on Hausa (\(+2.66\) over task-only). Activation-guided is strongest on Yoruba, but that comparison has only two seeds.

3pt

10.5 Test split sizes↩︎

For completeness, Table 15 lists the test split size for every language–task cell used in this study. The bootstrap CIs in Tables 8–11 are computed by paired sampling with replacement at the example level for cls/ner/qa and at the chrF-statistic level for sum.

3pt

11 Contextualising our numbers against published low-resource baselines↩︎

We do not run external baselines under our protocol (§¿sec:sec:limitations?). To help readers interpret the absolute scale of our scores, this appendix places our composition recipes next to comparable single-model numbers for the same benchmarks and base families. These are not our results, and the comparisons are qualitative rather than controlled head-to-head claims because the published systems use different gold budgets and seed protocols.

11.0.0.1 NER (MasakhaNER 2.0, XLM-R-base).

[10] report single-language fine-tuned XLM-R-base entity-F1 of approximately \(76\)–\(78\) on Hausa, Swahili, and Yoruba under the full training split, which is much larger than our \(b{=}256\) task-delta budget. Our task-only cells reach \(75.4\) (hau), \(77.4\) (swa), and \(46.7\) (yor). The Yoruba drop is consistent with the typological coverage gap that monolingual continued pretraining was reported to close in [10].

11.0.0.2 Classification (MasakhaNEWS, XLM-R-base).

[9] report macro-F1 in the high \(60\)s and \(70\)s for languages with substantial pretraining coverage, dropping to the \(40\)s for languages with little coverage. Our task-only cells reach \(58.5\) (hau), \(62.2\) (swa), and \(38.0\) (yor). Yoruba is again below the published full-supervision range, which is consistent with the much smaller \(b{=}256\) task-delta budget.

11.0.0.3 QA (AfriQA-style extractive QA).

[12] report F1 in the \(20\)s to mid-\(30\)s for African-language extractive QA on multilingual encoders under various adaptation regimes. Our cross-axis TIES F1 of \(22.05\), averaged across hau/swa/yor, sits within that range. Our task-only number of \(19.73\) is at the lower end, and the \(+2.32\) gain is consistent with the order-of-magnitude effect sizes reported for in-language adaptation in [12].

11.0.0.4 Summarisation (XL-Sum, mT5-base).

[14] report chrF in the high teens to mid-\(20\)s for mT5-base fine-tuned on the full XL-Sum split for these languages. Our cross-axis TIES chrF of \(11.58\) (amh), \(21.19\) (hau), \(23.76\) (swa), and \(17.84\) (yor) sits at the lower end of those published numbers. The gap is consistent with using an English-only \(\Delta_T\) rather than per-language full fine-tuning, while the gains show that cross-axis TIES recovers a meaningful part of the motivation for per-language tuning at zero per-pair training cost.

11.0.0.5 What this comparison does not show.

This appendix does not establish that cross-axis TIES is better than target-language LoRA at \(b{=}256\), or than Continue-PT \(+\) LoRA at matched compute. It only places our absolute scores within the published range, so readers can judge whether the composition-rule comparison happens in a useful operating region. A controlled fairness comparison against external baselines is the natural follow-up; the released \((\Delta_L,\Delta_T)\) pairs are designed to make it straightforward.

References↩︎