1 Introduction↩︎

Modern ASR systems achieve strong performance for high-resource languages such as English and Mandarin, which benefit from large-scale speech datasets [1]. However, the majority of the world’s approximately 7,000 languages remain under-represented in speech technology, restricting access to digital services, educational tools, and communication technologies for hundreds of millions of speakers [2].

Dhivehi (also called Maldivian), the national language of the Maldives [3], is spoken by roughly 335,000 to 410,000 people across the Maldives, Minicoy (India), and expatriate communities [4]. Mozilla Common Voice provides only 61 recorded hours of speech, of which 37 hours are validated for Dhivehi [5], far below the data volumes typically required for competitive ASR systems.

Self-supervised models such as Wav2Vec [6] and multilingual extensions such as XLS-R [7] have substantially reduced labelled data requirements through large-scale pre-training on raw audio. Fine-tuning on limited target data often underperforms because pretrained encoders lack exposure to the target language’s acoustic characteristics. Cross-lingual transfer leveraging data from a related, higher-resource language offers a complementary strategy that has shown promise across several language families [8]–[10].

Dhivehi and Sinhala both belong to the Insular Indo-Aryan subgroup, sharing phonological, lexical, and structural features documented in comparative linguistic work [4]. Figure 1 illustrates their genealogical relationship and contrasts it with Turkish, which is used in this study as a control experiment and belongs to a different language family.

Table 1 illustrates this through sentence-level comparisons, where cognates such as magu/maga (“road”), gamu/gama (“village”), and kan/kana (“ear”) appear in natural utterances. These shared properties may correspond to overlapping acoustic patterns that a pretrained encoder can leverage during cross-lingual transfer. Sinhala offers approximately 220 hours of openly available transcribed speech [11], roughly four times the validated Dhivehi data, providing the necessary resource to evaluate this hypothesis.

Despite growing interest in multilingual ASR, research on the Sinhala and Dhivehi languages remains extremely limited. Dhivehi is not adequately covered in existing multilingual ASR frameworks. Ahmed [12] had to use Sinhala-language tokens when fine-tuning Whisper for Dhivehi because the model did not provide explicit support for Dhivehi. As Sinhala and Dhivehi are both Indo-Aryan languages, their shared historical and structural characteristics may provide useful conditions for related-language transfer in low-resource ASR. Current research has not systematically examined this relationship [13], [14]. These gaps motivate the following research questions:

1. To what extent does pre-training on Sinhala speech improve Dhivehi ASR performance compared to a Dhivehi-only baseline?

2. How effectively can Sinhala and Dhivehi be represented within a shared latent space for cross-lingual ASR?

3. What role does linguistic relatedness play in cross-lingual transfer performance between Sinhala and Dhivehi?

4. How can ASR evaluation be standardised to ensure fair comparison across transfer learning strategies?

By addressing these research questions, this study offers several contributions:

• Strong baselines for Sinhala and Dhivehi that exceed existing results in the literature.

• A controlled comparison of three cross-lingual transfer paradigms for Sinhala to Dhivehi ASR, including continual pre-training (CPT), sequential fine-tuning, and multilingual fine-tuning with and without explicit language ID tokens.

• A control experiment using Turkish speech to isolate the effect of linguistic relatedness from generic multilingual data augmentation.

• A reproducible framework for future cross-lingual transfer learning studies.

2 Related Work↩︎

2.1 Self-supervised and Multilingual Speech Models↩︎

The transition from supervised acoustic modelling to self-supervised pre-training has significantly advanced low-resource ASR systems. Wav2Vec [6] learns contextualised speech representations through masked prediction and contrastive objectives over raw audio, enabling strong downstream ASR with as little as ten minutes of labelled data. XLS-R [8] extended this framework across 53 languages, while XLS-R [7] scaled to 128 languages and nearly half a million hours. MMS [2] further expanded coverage to over 1,000 languages through massive multilingual pre-training.

Whisper is a modern, large-scale, multilingual encoder-decoder model [15]. By relying on previously generated outputs, such models can capture language structure more naturally than pure CTC models, but they often require more data and may be less stable in very low-resource settings

2.2 Dhivehi and Sinhala ASR↩︎

The strongest published Dhivehi ASR work compares multilingual models based on XLS-R, MMS on Dhivehi Common Voice 13.0 [12]. Its most important finding is that subword modelling clearly outperforms a pure character vocabulary for Dhivehi. The best system combines MMS 1B fine-tuning with a language-aware subword vocabulary, 5-gram KenLM decoding, and automatic spelling correction, reducing WER to 14.26% and improving substantially over the earlier reported Dhivehi baseline of 20.95% [12].

For Sinhala, prior work has compared hybrid deep neural network-hidden Markov model (DNN-HMM) and end-to-end lattice-free maximum mutual information (LF-MMI) systems on 40 hours of speech, reporting 27.79% and 28.55% WER, respectively [16]. More recent work shows that pretrained e2e models can achieve 23.38% WER [17], while transfer learning with Mozilla DeepSpeech, KenLM decoding, and data augmentation has further reduced WER to 17.19% [18]. Taken together, these studies show that Sinhala ASR benefits from transferred representations. However, no prior work has investigated whether Sinhala can serve as a source language for targeted cross-lingual transfer to Dhivehi.

2.3 Transfer-learning↩︎

Yu et al. [19] showed that Chinese-to-Tujia transfer reduced the recognition error rate to 46.19%, improving performance by 2.11 percentage points over training on Tujia alone. Li [20] found that a Mandarin pretrained Wav2Vec-XLS-R model reduced Cantonese CER from about 0.30 to 0.20. Pillai et al. [21] showed that Tamil to Malasar multistage transfer reduced WER to 51.9%, and further to 47.3% after punctuation filtering, while Kim and Kang [22] reported over 20% relative WER improvement by further pre-training English Wav2Vec 2.0 on Korean speech. Černiavski [23] found that zero-shot transfer across Swedish, Danish, and Norwegian remained competitive, with relative degradation generally within 10–25% compared with fully supervised systems, and multilingual decoding further reduced WER with gains of up to 15–20% over monolingual Danish and Norwegian models. Nowakowski et al. [9] showed that CPT improved Sakhalin Ainu ASR from 42.3% WER to 29.8%, and that adding related Hokkaido Ainu yielded 30.5% WER, whereas unrelated English performed worse at 40.1% WER. Elamin et al. [24], reported WER of about 18–22% for Swahili, 20–25% for Kinyarwanda, and 22–28% for Luganda, with only 2–5% relative degradation under code switching. Anidjar et al. [25] demonstrated that semantically aligned multilingual datasets improve ASR performance, for example, reducing Russian Portuguese WER to 26.22% compared with 50.93% under low semantic similarity conditions. Getman et al. [10] showed that CPT and extended fine-tuning outperform direct fine-tuning for Northern Sámi, reducing XLS-R WER from 37.25% to 30.14% with 20 hours of data and to 28.84% WER and 7.36% CER.

3 Methodology↩︎

3.1 Data Collection and Preprocessing↩︎

Table 2 summarises the datasets used in this study. For Sinhala, we use the Large Sinhala ASR training dataset from OpenSLR (SLR52), originally introduced by Kjartansson et al. [11], collected through crowdsourcing in Sri Lanka. For Dhivehi, we use the Dhivehi subset of Mozilla Common Voice 24.0 [5]. Although the full Dhivehi subset contains approximately 61 h of audio, only the validated portion 37 h is used for supervised training. Common Voice data is collected through a volunteer-driven process in which speakers record prompted sentences, and other contributors validate the recordings for quality and accuracy. Turkish data from Common Voice, matched in volume to the Sinhala subset, is used in the control experiment.

All audio is converted to a common format, resampled to 16 kHz. Text transcriptions undergo Unicode normalisation (NFC), punctuation and spacing normalisation, non-linguistic symbol removal, lowercasing where needed, and language-specific script normalisation. For multilingual experiments, Sinhala is subsampled to 30 h or 60 h to reduce source language dominance, and <si> and <dv> tokens are prepended to each transcription.

3.2 Model Selection and Tokenisation↩︎

Wav2Vec was selected as the primary ASR model because it provides strong low-resource performance and a clear separation between self-supervised pre-training and supervised fine-tuning, enabling systematic comparison of transfer learning strategies. It also supports modular integration of external language models through CTC decoding. Compared with encoder-decoder models such as Whisper, it is more flexible for controlled experiments and more computationally efficient. Character-level tokenisation was used because limited text data and script differences make subword learning less reliable for Sinhala and Dhivehi.

3.3 External Language Model↩︎

To improve decoding quality, we incorporate an external language model during inference. Specifically, we use KenLM [26], a widely used toolkit for building efficient \(N\)-gram language models, together with pyctcdecode, a beam-search decoder designed for CTC-based speech recognition systems. A 5-gram KenLM model is trained on transcriptions and integrated into decoding through shallow fusion.

During inference, pyctcdecode combines the acoustic scores produced by the CTC acoustic model with the scores from the external language model in order to favour more plausible word sequences. The decoding score for a text hypothesis \(h\) given audio input \(X\) is defined as \[\text{Score}(h) = \log P_{\text{CTC}}(h \mid X) + \alpha \log P_{\text{LM}}(h) + \beta |h|,\] where \(P_{\text{CTC}}(h \mid X)\) denotes the score assigned by the acoustic model, \(P_{\text{LM}}(h)\) is the language model probability, and \(|h|\) is the hypothesis length. The hyperparameter \(\alpha = 0.5\) controls the contribution of the language model, while \(\beta = 1.0\) provides a length compensation term during decoding.

The beam width is set to 64. Empirical tuning showed that increasing the beam width to 128 degraded performance, likely because the larger search space retained more acoustically weak hypotheses during decoding.

3.4 Experimental Design↩︎

We conducted seventeen runs using six different experimental combinations. The experiment scripts are available in the project’s GitHub¹ repository.

(1) Dhivehi only baseline: Fine-tune the pretrained encoder on Dhivehi data only. This establishes the primary reference for all transfer comparisons.

(2) Sinhala only baselines: Fine-tune on 30 hours and 60 hours Sinhala subsets to validate the source language performance. The 60-hour model serves as the source checkpoint for sequential transfer.

(3) Sequential fine-tuning: First fine-tune on 60 hours of Sinhala, then continue fine-tuning on Dhivehi. This tests whether Sinhala provides a better initialisation for Dhivehi adaptation.

(4) Multilingual fine-tuning: Joint training on Sinhala + Dhivehi within a shared model, tested in four variants: with/without language ID tokens, and at 30/60 hours Sinhala proportions. This evaluates whether simultaneous cross-lingual exposure encourages beneficial shared representations.

(5) Continual pre-training: Continue the self-supervised pre-training objective on Sinhala audio using XLS-R, then fine-tune on Dhivehi. This targets representation-level adaptation rather than task-specific transfer.

(6) Turkish control: Replicate the no token multilingual setup by replacing Sinhala with an equivalent volume of Turkish data. Turkish, a Turkic language with no genealogical relationship to Dhivehi, controls for generic multilingual data augmentation effects.

3.5 Training Configuration↩︎

All experiments use a batch size of 16, gradient accumulation of 2 (effective batch size of 32), a learning rate of \(5\times10^{5}\) with 500 warm-up steps, 10 epochs, FP16 mixed precision, and gradient checkpointing.

3.6 Evaluation↩︎

System performance is evaluated on Dhivehi ASR using Word Error Rate (WER) as the primary metric and Character Error Rate (CER) as a secondary metric. \[\text{WER} = \frac{S + I + D}{N}\]

WER is the primary evaluation metric because it directly reflects word-level recognition quality and is the most widely used measure in ASR benchmarking. However, WER can treat minor spelling errors and completely incorrect words equally. To provide a more fine-grained analysis, we also report CER.

4 Experiments and Results↩︎

4.1 Main Results↩︎

3.5pt

Table ¿tbl:tab:results? presents all Dhivehi target results. The best system is continual pre-training with KenLM, achieving 12.89% WER and 2.70% CER, the only configuration that improves over the baseline in both metrics under both decoding conditions. Multilingual fine-tuning without language ID tokens 60 hrs Sinhala achieves 13.26% WER with KenLM, modestly improving WER over the 13.50% baseline. The 30 hrs Sinhala variant is similarly competitive, achieving 13.34% WER. Sequential fine-tuning and multilingual training with language ID tokens both degrade performance substantially.

4.2 Effect of KenLM Decoding↩︎

4pt

Table 3 quantifies the KenLM effect. Every configuration improves by 24-29 absolute WER points when KenLM is enabled. Without KenLM, all systems cluster in a high error band around 40-44% WER, making acoustic transfer differences nearly indistinguishable. Once KenLM is applied, the systems separate meaningfully, and the differences in transfer pathways become visible. This is the single largest and most consistent effect observed in the study, demonstrating that in this low-resource setting, the external language model is a dominant determinant of final transcription quality.

Reducing the beam width from 128 to 64 consistently improved performance by approximately 10 WER points across all evaluated configurations, highlighting the importance of decoder tuning.

4.3 Role of Language ID Tokens↩︎

3pt

Table 4 reveals an important finding: the multilingual model without language ID tokens substantially outperforms the variant with tokens for both Dhivehi 13.26% vs.% WER and Sinhala 6.49% vs.% WER when decoded with KenLM. This counterintuitive result suggests that in this bilingual setting, explicit language conditioning introduces harmful decoder separation rather than beneficial disambiguation.

4.4 Effect of Sinhala Data Proportion↩︎

3pt

Table 5 shows that Dhivehi performance is remarkably stable across the two Sinhala proportions \(\Delta\)WER \(= 0.08\) with KenLM in multilingual fine-tuning, while Sinhala baseline performance degrades when its training share is halved 6.49%\(\to\)​8.35% WER. This difference indicates that the multilingual benefit to the target language saturates at a relatively low level of source-language volume, suggesting that even modest amounts of related language data may suffice.

4.5 Unrelated Language Control Experiment↩︎

The Turkish control provides critical evidence for interpreting the multilingual results. Replacing Sinhala with Turkish degrades performance relative to both the Sinhala paired model (13.77% vs 13.26% with KenLM) and the Dhivehi-only baseline (13.50%). Without KenLM, the gap widens further 43.02% vs 42.09% vs 41.27%. This confirms that the modest multilingual improvements from Sinhala reflect genuine phonological and acoustic similarities between the two Insular Indo-Aryan languages, rather than a generic data-augmentation effect.

4.6 Sinhala Baseline Validation↩︎

Table 6 shows that the Sinhala source setup is reliable, achieving 7.00% WER with KenLM. This means the weaker sequential and multilingual transfer results are more likely due to limitations in the transfer approach, rather than a weak Sinhala source model.

4.7 Comparison with Published Benchmarks↩︎

Table ¿tbl:tab:comparison? compares our results with existing literature. Even the Dhivehi-only baseline 13.50% numerically surpasses the strongest published Dhivehi benchmark 14.26% [12]. The best transfer system, 12.89%, further improves upon this already competitive baseline. The Sinhala result 7.00% is substantially lower than the literature benchmark 17.19%, though direct comparison is limited by differences in evaluation conditions. Note that Ahmed [12] evaluated models using Common Voice 13.0, whereas this work uses Common Voice 22.0 (we use only the validated Dhivehi subset for supervised training);

5 Discussion↩︎

5.1 Transfer Strategy Matters More Than Relatedness Alone↩︎

The main finding is a qualified one: Sinhala improves Dhivehi ASR, but only through certain transfer strategies. Continual pre-training is the strongest approach because it adapts the encoder’s speech representations before introducing Dhivehi-specific supervision, thereby retaining reusable acoustic features. Multilingual fine-tuning without language ID tokens also gives a small gain. By contrast, sequential fine-tuning does not outperform the Dhivehi-only baseline, suggesting that stronger adaptation to the Sinhala supervised task may reduce the model’s transfer to Dhivehi.

With KenLM, the Dhivehi-only baseline achieves 13.50% WER, already numerically better than the strongest Dhivehi benchmark reported in the literature review; the transfer gains are therefore meaningful because they are achieved over a strong target-only baseline.

5.2 Language ID Tokens Are Not Universally Beneficial↩︎

The multilingual experiments show that shared modelling can help with Dhivehi, but the conditioning method matters. In this low-resource, bilingual, character-level setting, multilingual fine-tuning without language ID tokens performs much better than the version with them. This suggests that useful sharing is better supported when the model is allowed to learn common structure without explicit prefix-level separation. The weaker token-based result does not mean that language tokens are always harmful, but it does show that, in this setting, extra language conditioning may impose more separation than the learning problem can usefully support.

5.3 Decoding Support Is a First Order Effect↩︎

The effect of KenLM is too large to be treated as a minor implementation detail. In the Dhivehi-only system, WER falls from 41.27% to 13.50%, and similarly large reductions appear across the transfer settings, including the Turkish control. This shows that the best systems in the study should be understood not only as transfer learning results, but as transfer plus decoding results. Strong acoustic representations alone are not enough to explain the final performance. For low-resource Dhivehi ASR, decoder support is therefore essential, both for fair interpretation of model comparisons and for practical system design.

5.4 Linguistic Relatedness Alone Is Not Enough↩︎

The Turkish control provides the clearest evidence that relatedness matters. Under the same multilingual setup, Sinhala improves Dhivehi WER to 13.26% with KenLM, while Turkish reaches 13.77%, compared with 13.50% for the Dhivehi-only baseline. If the effect were only data augmentation, Turkish should have performed similarly. Instead, the Sinhala advantage suggests that phonological and acoustic overlap between the two related languages supports useful transfer. However, relatedness alone is not enough as sequential transfer still fails to beat the baseline. The most appropriate conclusion is therefore that relatedness helps, but the training pipeline determines whether that benefit is realised.

5.5 Limitations↩︎

Computational constraints limited the overall training scale, the number of repeated runs, and the extent of hyperparameter tuning; consequently, we did not evaluate models across multiple data splits or perform cross-validation. Continual pre-training used the XLS-R architecture because Wav2Vec2-BERT did not support the same CPT setup, creating an architectural inconsistency across experiments. Some pipeline instability affected intermediate estimates when integrating the pyctcdecode library, but final reported evaluations were manually verified.

6 Conclusion↩︎

We presented a controlled evaluation of Sinhala-to-Dhivehi cross-lingual transfer for low-resource ASR across five transfer paradigms and 17 combinations. Continual pre-training on Sinhala followed by fine-tuning to Dhivehi with KenLM achieves the best performance (12.89% WER and 2.70% CER), while multilingual fine-tuning without language ID tokens provides a competitive alternative. Sequential transfer and multilingual training with language tokens do not improve over the strong Dhivehi-only baseline. The Turkish control experiment confirms that observed gains can be attributed to linguistic relatedness. However, KenLM decoding is the single most impactful component, contributing far more than any transfer strategy.

These results show three key implications for low-resource ASR research. First, language-relatedness is necessary, but the transfer learning strategy determines whether shared linguistic features lead to measurable performance improvement. Second, design assumptions from large-scale multilingual systems, such as the benefit of language ID tokens, do not necessarily transfer to low-resource bilingual settings and must be validated empirically. Third, external language modelling remains essential and should not be treated as a secondary concern in the ASR pipeline.

Several directions for future research follow from this study. First, qualitative error analysis should examine which Dhivehi phonemes, word types, or linguistic structures benefit most from Sinhala-based transfer. Second, evaluation should be extended beyond the current benchmark to include a wider range of speakers, accents, domains, and recording conditions. Third, future work could compare Sinhala transfer with other related and unrelated source languages to better measure the value of linguistic relatedness. Fourth, stronger neural language models, especially Transformer-based approaches, could be explored for post-processing rather than relying solely on n-gram methods. Overall, this study provides a stronger benchmark and a reusable framework for future low-resource ASR research

References↩︎

---

1. https://github.com/lukmalilyas/From-Sinhala-to-Dhivehi-ASR↩︎