1 Introduction↩︎

The surge of mis- and disinformation has increased the demand for fact-checking [1]. As manual fact-checking is laborious [2], automated fact-checking (AFC) has emerged as a critical approach to support fact-checkers. A crucial early-stage task in AFC is claim check-worthiness detection (CWD), identifying claims that require verification based on domain-specific criteria [3].

On Wikipedia, CWD operationalizes as citation needed detection (CND) [3], [4]. We present examples of such claims in Figure 2 and Appendix 8. Wikipedia editors collectively judge and determine the check-worthiness of claims by applying Wikipedia’s verifiability policy.¹ When editors encounter claims that violate the policy, they tag them with the Citation Needed template to signal the need for an authoritative citation. This maintenance work is critical for preserving Wikipedia’s knowledge integrity, which has made Wikipedia an essential resource for the artificial intelligence community [5]. Yet, CND remains a largely manual task, motivating research that frames it as a supervised binary classification problem [4], [6], [7].

However, existing work has primarily studied CND in high-resource languages.² This is a critical limitation, as lower-resource language communities typically have fewer active editors and therefore face a disproportionate burden in tagging claims that require citations [9]. Moreover, recent AFC pipelines have often relied on large language models (LLMs) for claim detection [10], [11]. Their proprietary nature and high inference costs, however, make LLMs inaccessible to low-resource organizations [12].

In this work, we address these gaps by introducing Multilingual Citation Needed (MCN), a large-scale CND dataset derived from high-quality Wikipedia articles covering 18 linguistically diverse languages across multiple resource levels [8] (Figure 1). MCN advances prior work by extending coverage to lower-resource languages and by substantially increasing scale (Table [tab:data95comp]). On these data, we study small decoder-based language models (SLMs)³ as cost-effective, open-source alternatives to LLMs for CND. To this end, we specialise SLMs for CND through various supervised fine-tuning (SFT) [15] variants with QLoRA [16], to meet low-resource constraints.

We structure our study around three research questions for multilingual CND. First, initial work demonstrates strong performance of SLMs for English CWD [17]–[19]. However, their effectiveness in lower-resource languages remains unexplored, and existing work relies on standard SFT. This motivates: (RQ1) How do different SFT variants affect SLM performance across languages? Second, recent AFC work relies predominantly on LLMs [20], despite their prohibitive cost for low-resource organizations. We assess whether smaller models offer a viable alternative: (RQ2) How do fine-tuned SLMs compare to prompted LLMs and encoder-based PLMs across languages? Third, to our knowledge, no prior work has evaluated cross-lingual CND (X-CND). This setting is particularly relevant for lower-resource languages, where labelled data is scarce and supervised training is often infeasible. We therefore ask: (RQ3) How well do English-trained models transfer to lower-resource target languages under zero-shot and few-shot settings?

Our experiments show that encoder-style fine-tuning of SLMs [21] yields gains of up to 8 percentage points over generative SFT variants (RQ1). Across all resource levels, fine-tuned SLMs and PLMs substantially outperform prompted LLMs, with PLMs remaining a competitive baseline (RQ2). For cross-lingual transfer (RQ3), English-trained SLMs achieve strong zero-shot performance on parallel data but accuracy drops by about 20 percentage points under real-world conditions. Notably, encoder-style SLMs fine-tuned solely on English outperform prompted LLMs—even with little to no target-language supervision. We further demonstrate that our findings obtained on high-quality FA articles generalize to randomly sampled Wikipedia claims, underscoring the practical utility of our models.

Our findings have important implications for practitioners aiming to support lower-resource language communities. Crucially, fine-tuning SLMs on high-resource languages alone enables strong cross-lingual transfer, providing a viable path to deployment even when target-language data is scarce. Beyond Wikipedia, our findings reinforce emerging evidence [17], [22] that compact, task-specific models are often preferable to general-purpose LLMs.

max width=.8

Our contributions are as follows:

• We introduce Multilingual Citation Needed (MCN), a large-scale CND corpus covering 18 linguistically diverse languages across three resource levels sourced from high-quality Wikipedia articles. MCN extends prior datasets by including lower-resource languages and providing substantially larger samples.

• We study the impact of different SFT variants on adapting SLMs for multilingual CND, and benchmark them against encoder-based PLMs and LLMs. Our results show that fine-tuned models consistently outperform LLM baselines.

• To our knowledge, we present the first evaluation of cross-lingual CND. Our results show that SLMs can transfer effectively across languages in zero- and few-shot regimes, highlighting their practical potential for deployment in low-resource settings.

2 Related work↩︎

2.0.0.1 Citation Needed Detection (CND)

CWD is an early-stage task in AFC that aims to identify claims requiring verification based on domain-specific criteria [3]. Unlike verifiable claim detection (CD), CWD is inherently domain dependent [23], [24]. Accordingly, prior work has studied CWD across a range of domains, including rumor detection [25], politics [26], and Wikipedia [4]. [4] formalized the CND task for Wikipedia as a binary supervised classification problem that identifies English claims requiring citations. [6] extended CND to three smaller non-English Wikipedias, and subsequent work proposed a sample-efficient CND framework [27].⁴ Recently, the Wikimedia Foundation expanded CND to several high-resource languages, including Spanish and Italian [7]. However, existing work largely focuses on high-resource languages, leaving lower-resource language communities underexplored.

2.0.0.2 Multilingual Claim Detection

The majority of claim detection research has focused on English [28], [29]. The most prominent non-English claim detection efforts originate from the NLP4IF [30] and CheckThat! [31] shared task series. Although a small number of additional multilingual efforts exist (Table [tab:data95comp]), several studies have emphasized the need for multilingual claim detection resources [3], [20], [24]. Regarding cross-lingual approaches, only [32] investigate cross-lingual transfer on the NLP4IF dataset [30] and show that mBERT generalizes well to high-resource languages. Our work addresses the critical lack of multilingual CWD data and enables both multilingual and cross-lingual experiments across languages with varying resource levels.

3 Data↩︎

We introduce Multilingual Citation Needed (MCN), a CND corpus that spans 18 linguistically diverse languages across three resource levels [8], sampled from high-quality Wikipedia articles. MCN advances prior datasets by incorporating medium- and low-resource languages and by containing a substantially larger number of claims (see Table [tab:data95comp]).

max width=1

3.0.0.1 Language Selection

We select 18 languages that span linguistic diversity and represent high-, medium-, and low-resource levels [8] (Figure 1). Our selection is constrained by the availability of sufficiently high-quality Wikipedia articles from which we sample claims.

To ensure linguistic diversity, we sample languages from multiple families, groups, and scripts. We include several Indo-European (IE) language groups (e.g., Germanic, Slavic, and Albanian), covering three scripts (Armenian, Latin, and Cyrillic). To increase family-level diversity, we incorporate languages from the Austronesian, Austroasiatic, and Turkic families. Balancing linguistic diversity with resource availability, we select at least three languages per resource level from related language groups, including Germanic, Common Turkic, and Slavic.

3.0.0.2 Data Collection

For each Wikipedia language edition, we collect claims from featured articles (FAs), which editors designate as the highest-quality content on Wikipedia.⁵ One FA criterion states that "claims are verifiable against high-quality reliable sources and are supported by inline citations where appropriate."⁶ Prior analyses of Wikipedia’s citation quality confirm that FAs are consistently well sourced [33]. Consequently, we can derive binary CND labels, whether a claim requires a citation, by leveraging the presence or absence of inline citations. Following prior CND work [4], [6], we label a claim as 1 (Citation Needed) if it contains an inline citation or appears in a paragraph with a citation at the end; we label claims without inline citations as 0 (No Citation Needed). We further augment each claim with contextual information (the preceding and following sentences, when available) and the article’s topic (e.g., Health), which we obtain using Wikipedia’s ORES topic model [34]. We remove all citation markers from claims and apply extensive cleaning to ensure that no citation artifacts remain. Appendix 8 provides additional details on the data collection and cleaning procedures.

3.0.0.3 Dataset Statistics

Table [tab:desc] presents dataset statistics. As expected, lower-resource languages contain fewer FAs. However, even among languages within the same resource level, the number of available articles varies substantially. Across languages, claims that require citations tend to be longer and are more likely to include numeric expressions. These claims originate from articles that cover, on average, 76 unique topics (e.g., Health, Geography). Topic Similarity, computed as the cosine similarity between the topic distributions of claims with and without citations, remains relatively high. This pattern holds across languages except for Albanian, likely due to its considerably smaller number of available FAs. Appendix 8 provides additional descriptive statistics and example claims.

3.0.0.4 Data Quality

A potential concern is that parsing errors during data collection or incorrect citation retrieval may introduce bias. To address this issue, we conduct an extensive manual evaluation of data quality, which we report in Appendix 8.

3.0.0.5 Data Split

For each language, we split the data at the article level to ensure that test claims originate from articles unseen during training. Across all experiments, we use the same train/dev/test splits of 5k/500/500 claims, balanced by label. We report accuracy as the primary evaluation metric and provide F1 scores in Appendix 10.

4 Experimental Setup↩︎

We first describe the different SFT variants for adapting SLMs to CND, followed by a description of our experimental setup.

4.1 SFT Variants↩︎

We specialize SLMs via SFT [15] using QLoRA [16], making our experiments reproducible on a single A100 40GB. We explore three SFT variants, with generative and discriminative objectives:

4.1.0.1 SFT Variant 1: Full-token loss (FTL)

The FTL is the standard loss in most work, and the default in Huggingface’s SFTTrainer [35]. Training minimizes the autoregressive next-token loss over all tokens of the prompt (i.e., system, user, and assistant messages). This strategy has been adapted in prior claim detection work [18], [19].

4.1.0.2 SFT Variant 2: Target-only loss (TOL)

TOL differs from FTL by masking system and user tokens so that model weights update only on assistant (i.e., target) tokens. When fine-tuning models with QLoRA on the MMLU benchmark, [16] show that TOL outperforms FTL. The motivation for applying TOL to CND is to encourage the model to ‘focus’ its learning capacity on predicting the label rather than reconstructing the input prompt.

4.1.0.3 SFT Variant 3: Encoder-style (ES)

Rather than using generative SFT, recent work shows that SLMs can be effectively adapted as encoder-style discriminators [36], [37]. We implement this variant similar to [21] by feeding the last-token hidden state into a feed-forward classifier. We fine-tune this variant by concatenating the claim with its context and optimizing it using standard cross-entropy loss.

max width=

4.1.0.4 SFT Data

For generative SFT, we need to format each data sample into a chat-style format. Given our dataset \(d = \{(c_i, x_i, y_i)\}_{i=1}^{N}\) consisting of claim \(c_i\), context \(x_i\), and label \(y_i\), we convert each example into a chat-style prompt with system, user, and assistant messages. The system message contains task instructions; the user message concatenates the claim and its context (if available); and the assistant message contains the target label formatted as JSON (e.g., {"label": 1}) to enforce consistent output formatting.

4.1.0.5 Models

We use Llama-8B [38], Qwen3-8B [39] and Aya-8b [40]. We omit smaller sizes which we leave for future work.

4.2 Experiments↩︎

We design an experiment for each research question. Appendix 9 presents experimental details for full reproducibility.

4.2.0.1 (1) SFT Variants

To evaluate the effectiveness of different SFT variants for adapting SLMs to CND (RQ1), we fine-tune each model for every combination of SFT variant and language using a hyperparameter grid search. We use a simple instruction prompt across all runs, as preliminary experiments show little benefit from prompt engineering.

4.2.0.2 (2) Multilingual CND Benchmark

To evaluate the performance of different LM families for CND on MCN (RQ2), we benchmark SLMs against encoder-based PLMs and commercial LLMs. For PLMs, we fully fine-tune mBERT [41] and XLM-R [42] in both base and large variants. For LLMs, we evaluate GPT-5.2 [43] and Gemini 2.5 Flash [44], combining zero- and few-shot prompting with both instruction-style and verbose prompts, as LLMs may benefit from more detailed task explanations [17].

4.2.0.3 (3) X-CND

The ability of trained models to perform cross-lingual CND remains an open question (RQ3). We investigate this question by training mBERT and Llama with all three SFT variants on English as the source language. We then evaluate their zero-shot and few-shot transfer performance on all medium- and low-resource target languages in our corpus.

For zero-shot transfer, we consider two evaluation settings: (i) Parallel Test Data: Because claims in MCN are not parallel, we construct synthetic parallel test sets by translating the English test set into all target languages included in our study.⁷ To address potential translation quality issues [45], we perform extensive automatic quality checks, which we detail in Appendix 8. (ii) Non-parallel Test Data: Evaluation on non-parallel test data assesses models’ cross-lingual transfer under both language and claim distribution shifts (e.g., topics and semantics). This setting poses a more challenging yet more realistic evaluation scenario, as models deployed in practice must classify claims that diverge from their training data distribution.

For few-shot transfer, we follow the strategy by [46] for the non-parallel data setting: each model trained on English is further fine-tuned on a small number of target-language examples (\(N < 500\)). While this approach has proven effective for PLMs across a range of NLP tasks [46], its effectiveness for CND remains unclear and, to our knowledge, it has not previously been applied to SLMs.

5 Results↩︎

Experiment 1: SFT Variants↩︎

Encoder-style tuning mostly outperforms other SFT variants for multilingual CND across language resource levels. Table [tab:all95acc] reports CND accuracy for SLMs across SFT variants and languages of all resource levels. Across nearly all model-resource combinations, ES tuning achieves higher average accuracy than generative SFT variants. This effect is most pronounced for Aya, where ES tuning outperforms generative variants on almost all individual languages (e.g., by an average of 10 percentage points compared to FTL on high-resource languages). For Qwen and Llama, ES tuning also outperforms other variants on both high- and low-resource languages, although the gains are smaller (e.g., an average improvement of 3 percentage points for Llama on low-resource languages). On medium-resource languages, generative tuning variants achieve higher average performance. However, this result is driven by a single outlier (Uzbek) where generative tuning yields accuracy gains of up to 13 percentage points over ES tuning. For most other medium-resource languages, ES tuning remains competitive with generative SFT. Among the generative variants, TOL training yields slightly higher performance than FTL on high-resource languages across models. On medium- and low-resource languages, TOL and FTL perform comparably, with differences of only 1–2 percentage points.

max width=

Experiment 2: Multilingual CND Benchmark↩︎

SLMs considerably outperform LLMs on CND across most languages. Table [tab:all95acc] reports accuracy benchmarking results for all LMs in our study. For SLMs, we report results for the ES variant. Across resource levels, SLMs outperform LLMs by an average margin of 8–12%. Although LLMs achieve competitive results for Turkish and Bulgarian, ranking second best and best, SLMs consistently outperform LLMs even on most individual languages across all resource levels. LLM performance degrades as language resources decrease. The best-performing LLM, Gemini 2.5, achieves only 59.44% accuracy on low-resource languages, compared to 66.85% for Qwen. A similar trend appears for SLMs and PLMs when moving from high- to medium-resource languages. However, performance remains relatively stable from medium- to low-resource languages (e.g., Llama achieves 66.46% accuracy on medium-resource languages and 66.85% on low-resource languages). With detection accuracy ranging between 53–70% for medium- and low-resource languages, these results highlight the need for further research focused on low-resource language communities. We note that performance differences across languages stem from both language-specific factors and variations in claim distributions.⁸ We are able to analyze these effects in the X-CND experiments presented in the next section. Encoder-based PLMs remain a competitive baseline for multilingual CND. Encoder-based PLMs continue to provide strong baselines for multilingual CND. mBERT achieves the highest average accuracy on medium- and low-resource languages, with moderate gains of 1–2 percentage points over SLMs. Compared to XLM-RoBERTa, including its larger variants, SLMs achieve higher accuracy by 1–4 percentage points.⁹ mBERT’s strong performance may stem from its exclusive pretraining on multilingual Wikipedia data, which provides near-perfect domain alignment.

Experiment 3: X-CND↩︎

Fine-tuned models demonstrate strong zero-shot capabilities, but performance drops by up to 20 percentage points under real-world conditions. SLMs outperform PLMs by about 10 percentage points. Figure 3 reports accuracy averaged across test sets and language resource levels. On parallel data, fine-tuned models exhibit strong zero-shot generalization: SLMs achieve up to 87% accuracy, while mBERT reaches approximately 74% on medium-resource languages, with only marginal declines for low-resource languages. However, on non-parallel data, accuracy drops by 15–20 percentage points across all models. Across all settings, SLMs outperform mBERT by 5–10 percentage points and exhibit lower variance, as indicated by the error bars. Notably, performance differences across SFT variants remain small (within 2 percentage points), suggesting that the choice of SFT variant matters less for zero-shot transfer than for monolingual CND. QLoRA-fine-tuned SLMs require slower learning to achieve few-shot gains. For few-shot transfer, we continue fine-tuning the English-trained models on small amounts of target-language data, following [46]. However, we find that few-shot adaptation does not reliably improve SLM performance when using the higher learning rates commonly adopted in QLoRA fine-tuning [16]. We therefore treat the learning rate as a tunable hyperparameter during this second-stage adaptation. Appendix 10 reports the full results and shows that substantially lower learning rates are necessary to obtain consistent few-shot gains for QLoRA-fine-tuned SLMs.

max width=

Encoder-style SFT yields the most reliable few-shot gains among SLMs, while mBERT benefits the most from few-shot tuning overall. Table [tab:few95acc] reports few-shot accuracy results on non-parallel data. For each model, we report results obtained with the optimal learning rate, as described above. Among the SFT variants, only ES tuning exhibits consistent and substantial gains from few-shot adaptation. In contrast, continued fine-tuning with FTL and TOL proves largely ineffective on average, yielding only marginal improvements (less than 1% accuracy). By comparison, mBERT consistently benefits from few-shot tuning, particularly on lower-resource languages (e.g., a gain of 11.47 percentage points on Albanian with only 500 samples). Overall, few-shot tuning improves mBERT more than it improves ES-tuned SLMs.

English-only ES-fine-tuned SLMs surpass LLMs, even with zero or minimal target-language supervision. Compared to the monolingual evaluation of Experiment 2, ES-tuned Llama outperforms prompted LLMs on nearly all target languages, even under zero-shot or minimal target-language supervision. For example, on Uzbek, Llama (ES) achieves 64.73% accuracy in the zero-shot setting and 72.46% with \(k{=}500\) target-language examples, compared to 67.20% for the best-performing LLM (Gemini 2.5, few-shot). This result holds for all languages except Armenian. Notably, Llama is not only considerably smaller but also fine-tuned exclusively on English claims for this cross-lingual task.

Are results generalizable to non-FA articles?↩︎

max width=1

A potential concern is that models trained on claims from FAs may not generalize to non-FA articles, for example due to less standardized editorial styles. To assess this, we collected claims from randomly sampled Wikipedia articles (RA), selecting two languages per resource level, and constructed balanced test sets (\(n=100\) per language). Because non-FA articles are often under-referenced, leading to a high number of false negatives, we manually corrected all 600 labels.

Table [tab:general95acc] reports results for (i) the originally sampled claims, (ii) the label-corrected claims, and (iii) the FA baseline from Table [tab:all95acc]. We find that, for most languages, model performance on RA claims is comparable to, or slightly lower than, performance on FA claims. The discrepancy between results on uncorrected and corrected RA labels is explained by the high number of false negatives in the original annotations. After correction, the label distribution becomes more imbalanced; nevertheless, model performance remains close to FA-level detection. Overall, these results indicate that models trained on FA data generalize well to claims from randomly sampled Wikipedia articles, supporting their practical utility for the Wikipedia community.

6 Discussion↩︎

Choosing the right SFT variant for SLMs can yield substantial performance gains for multilingual CND (RQ1). Experiment 1 shows that ES fine-tuning consistently outperforms generative variants for monolingual CND, although TOL can benefit certain model–language combinations. Recent work explores adaptations of decoder-based models for encoder-like tasks [21], [47] and demonstrates that encoder-style training can be effective [36], [37]. Our results align with these findings and show that ES training performs well for CND across languages. For practitioners, ES tuning requires only minimal modifications to the base model while delivering reliable performance gains. In contrast, under generative SFT, the optimal variant depends on the specific model-language combination.

Supervised models substantially outperform LLMs (RQ2). Both fine-tuned PLMs and SLMs consistently outperform LLMs on multilingual CND. This result extends prior findings on English CWD [17]–[19] to the multilingual setting and aligns with evidence from broader NLP classification tasks [22]. Our findings show that supervised models are not only more compact and cost-effective than LLMs but also deliver substantially stronger performance for CND. Their relatively low hardware requirements make them particularly well suited for deployment in resource-constrained environments such as Wikipedia. Nonetheless, the substantial performance variation across resource levels (53-75%) indicates that multilingual CND remains an open and challenging research problem.

Encoder-style SLMs exhibit strong zero-shot cross-lingual transfer and outperform LLMs with little to no target-language supervision (RQ3). Across SFT variants, SLMs demonstrate robust zero-shot generalization on parallel data. However, performance drops by up to 20 percentage points under real-world conditions, exposing a critical limitation for practical deployment. This finding highlights the need for future X-CND research to develop more robust strategies that address simultaneous language and claim-distribution shifts.

Crucially, ES-fine-tuned SLMs trained exclusively on English claims outperform prompted LLMs even with no or minimal target-language adaptation. This result has direct implications for real-world deployment: cross-lingual transfer of specialized SLMs trained on high-resource languages emerges as a viable strategy when target-language data is scarce. For practitioners, this approach offers an affordable deployment pathway to support low-resource communities in CND.

7 Conclusion↩︎

We introduce MCN, a multilingual citation needed detection (CND) dataset that covers Wikipedia claims from 18 linguistically diverse languages across three resource levels. We study decoder-based small language models (SLMs) as alternatives to LLMs for CND, specialized through supervised fine-tuning. We demonstrate that SLMs outperform LLMs on monolingual CND across languages and that encoder-style tuning enables strong zero- and few-shot cross-lingual CND transfer. Our findings have important implications for supporting low-resource language communities in CND.

Limitations↩︎

Our work has several limitations.

7.0.0.1 Language selection concentrates on Indo-European languages.

Our selected languages are 70% Indo-European. While we vary language groups and also cover other families, future work can focus on language families such as Sino-Tibetan or Semitic.

7.0.0.2 Insights into SLM performance are limited to QLoRA-based SFT.

To ensure practical relevance, we fine-tune SLMs exclusively using QLoRA. As a result, our insights into the effects of SFT strategies and the cross-lingual capabilities of SLMs for CND are limited to this PEFT setting. Performance dynamics may differ under alternative parameter-efficient fine-tuning approaches. Nevertheless, QLoRA is particularly well suited to this study due to its simplicity and efficiency under quantisation. Moreover, we restrict our analysis to 8B-parameter SLMs. Future work should explore a broader range of model sizes and PEFT strategies to assess the generality of our findings.

7.0.0.3 Limited generalizability to other domains.

CWD is inherently domain-dependent, as domain-specific criteria determine what constitutes a check-worthy claim. Accordingly, our findings are limited to the Wikipedia domain and may not directly transfer to other common CWD settings, such as political debates. Nevertheless, the strong monolingual and cross-lingual CND performance observed on the challenging Wikipedia domain suggests that comparable gains may be achievable in other application domains.

7.0.0.4 Real-world citation practices deviate from labeling heuristics.

We label claims as requiring a citation or not based on the presence or absence of inline citations, a heuristic following prior CND work [4], [6], [7]. Nevertheless, in practice, real-world editorial practices deviate from that heuristic. For example, a citation may appear only in the final paragraph of a multi-paragraph passage when it is clear to support all paragraphs. We illustrate this challenge for German Wikipedia in Appendix 10. We believe that devising more robust labeling heuristics, for example through the use of soft labels, is an important direction for future work.

Ethics Statement↩︎

7.0.0.5 Biases

Although our language selection spans a wide range of language families and writing systems, it is skewed toward Indo-European languages. In addition, Featured Wikipedia articles, while covering diverse topics, tend to emphasize subjects that are of particular interest to the corresponding language communities at a given time, which may introduce topical biases.

7.0.0.6 Potential Harms

We are not aware of any direct harms arising from our dataset or experiments. Our work is motivated by the goal of supporting Wikipedia editors, particularly in low-resource language communities, by improving automated citation needed detection.

7.0.0.7 Licensing

We collect only publicly available Wikipedia content, which is licensed under CC BY-SA. We release all collected data, including translated text obtained via the Google Translate API, under the same CC BY-SA license to support future research.

7.0.0.8 Reproducibility

We provide full details required to reproduce our results in Appendix 9, including hyperparameters, hardware requirements, and fine-tuning procedures. All models are trained three times with different random seeds, and we report mean performance and standard deviations to enhance the reliability of our findings.

7.0.0.9 AI Assistants

We use LLMs to proofread our writing by prompting it to correct grammar and spelling.

width=.8

8 Data↩︎

8.1 Data collection↩︎

We collect claims from Featured Articles, which represent the highest-quality articles on Wikipedia.¹⁰ For languages other than German, English, and Russian, where Featured Articles are abundant, we additionally include Good Articles¹¹ to increase data coverage. Like Featured Articles, Good Articles are required to contain factually accurate and verifiable information, although they are typically less comprehensive.

For all selected articles across languages, we query the MediaWiki API¹² to obtain their full HTML representations. We then parse each article into paragraphs (excluding the lead section) and split paragraphs into individual claims (i.e., sentences) using Stanza.¹³ Cleaning primarily involves correcting formatting issues (e.g., spaces before punctuation), removing extraneous whitespace, and discarding claims shorter than 15 characters, among other steps. These procedures are applied consistently to all retrieved elements, including the topic, the claim itself, and the preceding and following sentences.

Because artifacts introduced during parsing or preprocessing may create spurious correlations and bias our results, we additionally conduct a manual data quality check.

8.2 Data Labelling↩︎

We largely follow prior work in labeling claims [4], [6], with one minor modification. Consistent with [4], we label a claim as not requiring a citation only if it appears in a paragraph that contains no citations. Conversely, if a target claim does not contain an inline citation but appears in a paragraph where other claims are cited, we label it as citation needed. This design choice is motivated by our observation that, in practice, claims without inline citations are often implicitly supported by citations in adjacent sentences, either preceding or following the claim, even when no explicit inline citation is present. We provide a qualitative analysis of the general limitations of this heuristic on the German Wikipedia in Appendix 10.

8.3 Descriptive Statistics↩︎

8.3.0.1 Dataset examples

Table [tab:data95examples] illustrates two representative examples in English and Italian from our dataset. In the English examples, the first claim contains multiple factual assertions (e.g., the naming of a city and the reason for its name) related to a historical event, each of which requires a citation to verify that these events occurred. The second English example represents a canonical case of a claim that does not require a citation under Wikipedia’s verifiability policy, as readers can reasonably infer that the source of the information is the film itself.¹⁴

For the Italian examples, the first claim translates to “In the fall, he began recording a new album, and thus, eleven months after Due anni dopo, L’isola non trovata was released.” This claim refers to the recording and release of a new album and therefore requires a reliable external source to substantiate the information. The second Italian example introduces a citation to Gianluca Magi. While the cited text itself requires a citation, the surrounding sentence does not, illustrating a case where the claim is not citation-worthy.

8.3.0.2 Descriptive statistics

Figure 4 presents descriptive statistics on syntactic distance (a), cross-lingual topic similarity (b), and the ten most frequent topics across languages (c). Our language selection covers a broad range of linguistic diversity, as measured by syntactic distance derived from lang2vec vectors.¹⁵ We cover syntactic distance relationships ranging from distant language pairs (e.g., Indonesian and Albanian) to closely related ones (e.g., Ukrainian and Russian).

Panel (b) illustrates cross-lingual topic similarity measured via cosine similarity between topic vectors. Overall, Wikipedia language editions exhibit relatively high topical overlap. A notable exception is Albanian (sq), for which topic similarity is lower, likely due to the substantially smaller number of high-quality Albanian Wikipedia articles in our sample.

Finally, Panel (c) shows the ten most frequent topics across languages, indicating that claims span a diverse set of Wikipedia categories, including Regions, STEM, and Media.

8.4 Manual Quality Evaluation of MCN↩︎

width=

To ensure the quality of our data, we conduct a manual evaluation to assess the correctness of the labels, extracted claims, and context on a representative sample of our data. We randomly sample 540 instances, balanced across languages and labels, and manually evaluate the following: (1) whether the claim is correctly extracted and free from parsing errors (e.g., truncated sentences); (2) whether the assigned citation label is accurate based on our heuristic (see above); and (3) whether the surrounding context is correctly retrieved and free from parsing errors. We conduct these checks by identifying the original claim and its surrounding context on the source Wikipedia page and then manually verifying whether the original instances match our retrieved instances. If they do not match, we mark the corresponding metric as incorrect.

Table [tab:data95eval] presents the data quality evaluation results. Overall, data quality is consistently high across all three criteria and across language resource levels. Most common sources of errors are missing conclusion of certain HTML tags which lead to incorrect claims and labels, and incorrect sentence splitting. However, we also observe discrepancies between the labeling heuristic, also used in prior work [4], [6], and real-world editorial citation practices. We discuss this issue in more detail below.

8.5 Translation Quality Evaluation↩︎

width=

We conduct an extensive automatic evaluation of the translated parallel test sets used for zero-shot X-CND. Specifically, we back-translate claims that were translated from English into the target languages back into English in order to compute standard translation quality metrics. Automatic evaluation via back-translation is a common procedure when gold reference translations are unavailable [48]. We use the DeepL API for back-translation.¹⁶ We report standard machine translation metrics, including BLEU [49] and ROUGE [50] as n-gram overlap measures, and BERTScore [51] to assess semantic similarity.

Table [tab:trans95qual] presents the results of the automatic translation quality evaluation. Across all languages, all metrics indicate strong semantic preservation, with BERTScore values ranging from 0.94 to 0.98. BLEU and ROUGE scores likewise suggest successful reconstruction, although they exhibit greater variance. In particular, Macedonian and Uzbek show lower BLEU and ROUGE scores, likely reflecting greater typological distance from English rather than translation errors, as semantic similarity remains high. Overall, these results provide strong evidence that translation quality is high across languages of all resource levels.

9 Experimental Details↩︎

We design all experiments to be runnable on a single A100 GPU (40 GB). All experiments are implemented using PyTorch and the Hugging Face transformers library¹⁷. While each experiment fits on a single GPU, we run experiments on an HPC cluster with multiple GPUs to accelerate training.

9.1 SLMs↩︎

width=.8

We use SFT [15] with QLoRA [16] to adapt SLMs to CND. For each language–model combination, we perform a hyperparameter grid search using the hyperparameters shown in Table [tab:hps95slm]. In cases where we encounter out-of-memory errors, we reduce the batch size to 8 and use a gradient accumulation step of 2. We use the peft¹⁸ library to implement QLoRA.

Preliminary experiments with different prompts revealed only marginal performance differences. Therefore, we restrict our experiments to a single prompt. We use the simple instruction prompt shown below for all experiments involving SLMs:

Instruct Prompt
----------------
You are a multilingual classifier.
Decide whether the given {lang} claim requires a citation.
Use 1 if the claim needs a citation. Use 0 if the claim does not need a citation.

Return only JSON in the format: {"label": 0} or {"label": 1}.
No explanations or extra text.

To enhance the reliability of our results, we select the optimal hyperparameter configuration and repeat training twice using different random seeds. We report the mean performance and standard deviations across these runs.

9.2 PLMs↩︎

width=.8

For PLMs, we fully fine-tune each model-language combination with hyperparameter grid search using the hyperparameters displayed in Table [tab:hps95plm]. We run reliability checks identical to the SLM setup.

9.3 LLMs↩︎

For LLMs, we use the same instruct prompt applied to SLMs as well as a more verbose prompt, as [17] show that prompt verbosity can influence performance. The verbose prompt provides additional task instructions and incorporates the citation reason taxonomy proposed by [4]:

Verbose Prompt
----------------
You are a multilingual Wikipedia citation classifier.
You are provided with a {lang} claim and its context.
Your task is to analyze the claim and the context to decide whether the claim needs a citation.
On Wikipedia, there are various reasons why a claim may or may not require a citation.
The reasons are listed below:

Reasons why citations are needed (Label 1):
Quotation - The statement is a direct quotation or close paraphrase of a source.
Statistics - The statement contains statistics or quantitative data.
Controversial - The statement makes surprising or potentially controversial claims.
Opinion - The statement expresses a person's subjective opinion or belief.
Private Life - The statement contains claims about a person's private life.
Scientific - The statement includes technical or scientific claims.
Historical - The statement makes general or historical claims that are not common knowledge.
Other (Needs Citation) - The statement requires a citation for other reasons.

Reasons why citations are not needed (Label 0):
Common Knowledge - The statement contains well-known or widely established facts.
Plot - The statement describes the plot or characters of a work that is the subject of the article.
Other (No Citation Needed) - The statement does not require a citation for other reasons.

Based on these reasons, think step-by-step to decide in which category the claim falls.
Return only JSON in the format: {"label": 0} or {"label": 1}.
No extra text.

We use OpenRouter¹⁹ to run inference with the selected LLMs. All LLM experiments are conducted with a low temperature (0.1) which is why we do not perform additional reliability runs.

For few-shot prompting, we randomly sample two examples per label. As shown in Table [tab:all95acc], the inclusion of random shots often improves performance, particularly for the instruct prompt variant.

10 Results↩︎

In this section, we report the results of our main experiments using F1 score instead of accuracy. We further present additional analyses on the effect of different learning rates for few-shot tuning of SLMs.

10.1 Experiment 1: SFT Variants↩︎

Table [tab:atl95f1] presents the results of our SFT variant experiments using F1 score as the main evaluation metric.

max width=.9

10.2 Experiment 2: Multilingual CND Benchmark↩︎

Table [tab:all95f1] presents the multilingual CND benchmarking results for all LM families, using F1 score as the main evaluation metric.

max width=.9

Error Analysis of the German Wikipedia↩︎

We observe comparatively low evaluation scores for the German Wikipedia. To better understand this effect, we manually analyze model misclassifications. We find that real-world citation practices of Wikipedia editors occasionally diverge from the labeling heuristic we adopt, which is also used in prior work [4], [6]. We illustrate these discrepancies with two representative examples.

10.2.0.1 Example 1

The first deviation arises from citations that appear only at the end of multiple consecutive paragraphs. In the German Wikipedia article on the city of "Augsburg, the section "Gewässer" consists of six paragraphs. While the first five paragraphs contain no citations, the final paragraph includes a single citation that appears to support the entire multi-paragraph. We observe several such cases, suggesting that editors sometimes choose to include only one citation to cover a broader textual span.

Under the established labeling heuristic, claims in all but the final paragraph would be incorrectly labeled as not requiring a citation. One potential remedy would be to treat all claims preceding the cited paragraph as citation-worthy. However, this assumption does not always hold, as the final citation does not necessarily reference all preceding paragraphs in every multi-paragraph setting.

10.2.0.2 Example 2

A second issue arises when a citation attached to an earlier claim implicitly supports subsequent claims without being repeated, provided that the connection is obvious. For example, in the article Useless, section Video and Vinyl:

2004 veröffentlichte das Bochumer Label Dirty Faces Records das Album Useless als Schallplatte und zusätzlich auf farbig gesprenkeltem Vinyl in limitierter Auflage von 525 Stück. [18] Eine weitere Ausgabe des Albums in pinkfarbenem Vinyl erschien im Jahr 2017 bei Drumming Monkey Records.

(English translation) In 2004, the Bochum-based label Dirty Faces Records released the album Useless on vinyl and additionally on colored splatter vinyl in a limited edition of 525 copies. [18] Another edition of the album on pink vinyl was released in 2017 by Drumming Monkey Records.

In this context, the citation [18] clearly also supports the final sentence. To account for this editorial practice, we modify existing labeling heuristics [4], [6] to also label the final sentence as citation needed, which is correct in this case. However, this assumption does not hold for all sentences without explicit citations.

Real-world editorial citation practices are difficult to capture automatically, as they vary across Wikipedia language communities. Overall, these examples highlight the limitations of current heuristic labeling approaches for CND on Wikipedia. Designing more robust labeling strategies therefore remains an important direction for future research.

10.3 Experiment 3: X-CND↩︎

Zero-shot X-CND↩︎

Figure 5 presents zero-shot X-CND results with F1 score as the main metric.

Few-shot performance by learning rates↩︎

max width=.9

max width=.9

For SLMs fine-tuned with QLoRA, the optimal learning rate is typically relatively higher (2e-4) compared to most other fine-tuning tasks [16]. This is also the case for CND, where we observe that across nearly all model-language combinations, hyperparameter search consistently identifies high learning rates as optimal. However, when applying continued fine-tuning for few-shot transfer following [46], we quickly observe that this strategy is detrimental to the performance of SLMs.

Therefore, for the second-stage fine-tuning on the target language shots, we treat the learning rate as the sole hyperparameter to optimize. While we could also include other hyperparameters, the connection between high learning rates, small amounts of training shots, and poor model performance seems most obvious to remedy.

Tables [tab:lr95acc] and [tab:lr95f1] report the results of our experiments analyzing the effect of learning rate choice on few-shot X-CND. We observe a consistent pattern: Llama (ES) and mBERT benefit from relatively higher learning rates, whereas generative SFT variants (FTL and TOL) exhibit improvements from few-shot tuning only when using substantially lower learning rates, down to 1e-7. Even then, the gains achieved by generative SFT through additional few-shot fine-tuning remain modest. Overall, these results indicate that learning rate selection is critical for continued fine-tuning on small amounts of target-language data for QLoRA-based SLMs, particularly under generative SFT. In contrast, mBERT performs optimally with learning rates similar to those identified during hyperparameter.

Few-shot Performance↩︎

max width=.9

Table [tab:few95main95f1] reports the results for few-shot CND with F1 score as the main metric on non-parallel data.

Generalizability↩︎

max width=1

References↩︎