1 Introduction↩︎

Language models can acquire factual knowledge during pre-training, enabling them to answer factual questions without explicit access to external resources [1]. However, this knowledge is stored implicitly in model parameters, making individual facts difficult to inspect, correct, or selectively update. This creates a fundamental limitation for factual generation: models may hallucinate [2], become temporally misaligned as world knowledge changes [3], [4], or over-rely on memorized facts even when external evidence provides a conflicting update [5]. Furthermore, their factual recall is uneven across relations and entities, and degrades for long-tail entities [6]. Finally, relying on parameters to store factual knowledge does not scale efficiently: factual knowledge capacity increases with model size, and conversely, memorizing large-scale knowledge bases may require extremely large models and training budgets [7].

Retrieval-augmented generation (RAG) methods [8]–[10] and tool-use approaches [11], [12] allow pulling in factual information from external textual resources or tools. These methods improve factual grounding and make some knowledge updates possible through changes to the retrieval corpus or tool backend. However, retrieval alone does not eliminate parametric factual reliance. The model may ignore retrieved evidence, blend it with memorized knowledge, or generate claims that are not attributable to the retrieved source [13]–[15].

In this paper, we propose KARLA, a method that entices a language model to query factual knowledge from a knowledge base (KB, also called knowledge graph), i.e., a structured repository of subject-relation-object triples.

This endeavor is challenging for several reasons: First, the model must issue correct queries, i.e., it must identify the subject and relation of a query that has a chance to deliver a result. Second, the model must learn to issue correct queries also for entities that were not seen during training, and about which it has no parametric knowledge. (While it appears easy to determine that population size is a reasonable query for “Paris”, what is a reasonable query for “9-hydroxyrisperidone”?) Finally, our method has to work without any manual supervision or intervention.

Our key idea is to generate a synthetic corpus directly from the KB, interleaved with relation-specific query tokens. The model is then fine-tuned on this corpus to issue these tokens whenever it is about to generate factual information (Figure 1). During inference, we replace these tokens with the results of a query to the KB. Our goal is to separate linguistic competence from factual knowledge: the language model handles interpretation and generation, while the KB provides the atomic factual values. This brings several advantages:

1. Factual knowledge can be updated in the KB at virtually no cost, without retraining the model. Any change is effective immediately.

2. Knowledge that was pulled in from the KB can be marked as such, thus leaving a provenance that serves transparency and explainability. Every KB-sourced span is verifiable against a specific fact in the KB, a guarantee that neither vanilla generation nor RAG can deliver.

3. The model can be much smaller, as all factual knowledge resides in the KB and not in the parameters. KARLA models scale not with the number of facts (as parametric knowledge does), but with the number of relations (which is typically small), meaning that our approach allows even small models to achieve high factual accuracy with KBs of near-arbitrary size.

Indeed, our experiments on factual question answering, long-form generation, and counterfactual KB-update settings show that (1) the model will generate updated information from the KB even when this contradicts its own parametric outdated knowledge, (2) a substantial part of generated long-form text can be sourced to the KB, and (3) small models such as Qwen0.6B comfortably beat larger models in factual accuracy when enhanced with our method.

2 Related Work↩︎

2.0.0.1 Language models as KBs.

Pretrained models store facts implicitly [1], but their capacity is limited to \(\approx\)​2 bits/parameter [16], [17] and uneven for long-tail entities [6]. Theoretical work suggests that external lookup outperforms memorization under these constraints [18], which motivates our shift of factual storage outside the model.

2.0.0.2 Retrieval-augmented generation

\(\!\!\!\!\)(RAG) places passages retrieved from external documents in the context window of the model [8]–[10]. However, the model can choose to ignore this information, or hallucinate information that was not given [13]–[15], as we also show in our experiments.

2.0.0.3 Tool-use approaches

\(\!\!\!\!\)let the model issue calls to external systems [11], [12], [19], sometimes via learned special tokens per tool [20], but rely on annotated demonstrations of tool use that do not scale to the relation inventory of a typical KB. KARLA instead generates training data synthetically from the KB itself. KBLaM [21] integrates a knowledge base into a language model through continuous key-value representations that support dynamic updates without retraining. However, this method holds the entire KB via an attention mechanism and attends over all knowledge tokens at query time. Hence, it is applicable only to small KBs. KARLA, in contrast, is designed for large KBs such as YAGO [22] with millions of triples.

2.0.0.4 Large Memory Language Models

\(\!\!\!\!\)(LMLMs) [23] are pre-trained from scratch with retrieved factual values masked from the loss, so that the model learns lookup behavior rather than memorization. The method takes as input a corpus of text, and builds up the KB from there. Our method, in contrast, is designed for the scenario where the user already possesses a KB, and wants the language model to answer with factual information from this source.

3 Methodology: KARLA↩︎

3.1 Problem Setup↩︎

3.1.0.1 Approach.

We are given a pre-trained language model and a knowledge base (KB). Formally, a KB is a set of triples of the form \(\langle{}s, r, o\rangle{}\), where \(s\) is a subject entity, \(r\) is a relation (or predicate) from a closed set \(\mathcal{R}\), and \(o\) is the corresponding object, as in \(\langle\)Paris, populationTotal, 2,047,602\(\rangle\). Our objective is to fine-tune the model so that it interleaves natural-language generation with inline queries of the form \(\langle{}s, r, ?\rangle{}\). These inline queries are then replaced by an object \(o\) for which \(\langle{}s, r, o\rangle{}\) is in the KB (see Figure 1).

3.1.0.2 Inline queries.

In practice, the inline query \(\langle{}s, r, ?\rangle{}\) with result \(o\) is expressed as \[\langle{}r\rangle \, \langle\texttt{subj}\rangle{}s\langle/\texttt{subj}\rangle \, \langle\texttt{KB}\rangle{}o : o_{\text{desc}} \langle/\texttt{KB}\rangle\] In this sequence, \(\langle{}r\rangle\) is a relation-specific trigger token. The tag \(\langle\texttt{subj}\rangle\) marks the subject, which is used to query the KB. The model will be trained to identify the subject here from the preceding part of the sentence. The tag \(\langle\texttt{KB}\rangle\) marks the answer to the query that was retrieved from the KB. It is included in the output sequence of tokens as soon as the inline query is generated, so as to allow the model to condition the following tokens on \(o\). The text \(o_{\text{desc}}\) is a short description of the object as retrieved from the KB (e.g., by help of the relation schema:description). It provides information about \(o\) that the model may not have in its parameters, in particular for long-tail or unseen entities. If the query has several answers in the KB, one of them is picked at random. This is motivated by the use case of long-form generation, where a single object will fulfill the expected role (as in “Elvis Presley won the \(\langle\)award\(\rangle\langle\texttt{subj}\rangle\)Elvis Presley\(\langle/\texttt{subj}\rangle\langle\texttt{KB}\rangle\)Grammy Award\(\langle/\texttt{KB}\rangle\)”, where the Grammy Award is one of the awards of Elvis).

3.1.0.3 Predicate Representation and Initialization.

We represent each relation as an atomic special token \(\langle{}r\rangle\) rather than as a plain-text label. This has the advantage that the model cannot generate relations that do not exist in the KB. It also reduces per-call inference cost, because each retrieval action emits one token rather than the several sub-tokens of a plain-text predicate label. To speed up convergence, we initialize the new embeddings from the base tokenizer. Let \(E \in \mathbb{R}^{|\mathcal{V}| \times d}\) denote the embedding matrix of the original tokenizer where \(\mathcal{V}\) is the vocabulary set and \(d\) is the embedding dimension. For a relation \(r\), we define the embedding of the corresponding predicate token \(E_{\langle r \rangle}\) as the mean of its constituent sub-token embeddings [19]. The added vocabulary introduces \(|\mathcal{R}| \cdot d\) embedding parameters, a negligible fraction of the base model size. We provide an ablation against a plain-text predicate representation in Appendix 11.

3.2 Generating the training corpus↩︎

To train the model to interact with the KB, we require a corpus in which natural language is interleaved with KB queries. One could, of course, take an existing corpus of text, and weave in the KB queries. However, this would require jointly solving several information extraction tasks, including named entity recognition [24], entity linking and entity disambiguation [25], [26], and relation extraction [27]. In addition, naturally occurring text provides highly imbalanced coverage of KB relations, with long-tail predicates appearing too rarely to support robust learning [28]. We thus generate the training corpus synthetically from the KB.

For this purpose, we first have to sample facts from the KB. A naive uniform sampling would generate a corpus of disconnected facts, in which each sentence expresses a single fact and the phrasings in which the facts appear are severely limited. Therefore, we generate texts that talk about several facts of a single entity before moving on to the next entity. If we sample entities instead of facts, though, we cannot ensure that all relations appear equally often. We therefore formulate the sampling as an iterative greedy process designed to satisfy a target distribution across the relational schema. Let \(\mathcal{R}\) be the set of KB predicates and \(T\) the target frequency for each relation. At each iteration \(i\), we identify the least used relation \(r^* \in \mathcal{R}\). We then sample an entity \(e\) uniformly at random from the set \(\{e \mid \exists (e, r^*, o) \in \text{KB}\}\). For the selected entity \(e\), we retrieve its fact neighborhood \(\mathcal{F}(e)=\{\langle{}e, r, o\rangle \mid \langle{}e, r, o\rangle \in \text{KB}\}\), i.e., the set of all KB triples with \(e\) as subject. When a relation \(r\) admits several objects, we sample one uniformly at random. We remove from the neighborhood all facts with relations \(r\) that have already appeared \(2\times{}T\) times in previous samples. We then sample \(k\) facts from this reduced neighborhood. The complete sampling algorithm is provided in Appendix 8. We also prove in Proposition 1 in the same appendix that our method yields per-relation counts bounded between \(T\) and \(2\times{}T\) regardless of the KB’s natural relation distribution (as we sample with replacement).

Once the facts are sampled, we prompt a teacher LLM (GPT-5 mini [29] in our experiments) to produce the training corpus. For this purpose, we iteratively feed it with each sampled entity subgraph, together with natural-language descriptions of the relations, and prompt it to generate an encyclopedia-style paragraph (approximately 250 tokens) with inline queries. Prompts are provided in Tables 8 and 9 (Appendix 14). We validate each generated example using two checks: (i) for every annotated object mention, the text span must match exactly the referenced object \(o\); and (ii) every sampled relation in \(\mathcal{F}(e)\) must appear in the output. If either check fails, the teacher receives structured feedback describing missing relations and parsing errors, and generation is retried up to two times, after which the subgraph is discarded. This affects less than \(0.5\%\) of subgraphs in our experiments.

3.3 Training↩︎

We then train our input model on the generated corpus. Let \(\tilde{x} = (\tilde{x}_1, \ldots, \tilde{x}_M)\) denote a sequence of tokens with inline queries and their KB-returned objects. Since training covers only a small subset of the KB, the model must learn to handle cases where the KB contains no matching fact. In 10% of inline queries (sampled independently), we replace the successful lookup result with the failure token \(\langle\texttt{KB\_FAIL}\rangle\). This teaches the model to recover from empty or unresolved KB queries by falling back to its parametric knowledge.

To enforce the separation of natural language and KB knowledge, we employ a masked next-token prediction loss. The loss is defined as: \[\mathcal{L}(\theta) = -\sum_{t=1}^{N} m_t \cdot \log p_\theta\left(\tilde{x}_t \mid \tilde{x}_{<t}\right)\] The binary mask \(m_t \in \{0, 1\}\) is constructed to gate the gradient flow: \[m_t = \begin{cases} 0 & \text{if } \tilde{x}_t \in \text{span}(\langle\texttt{KB}\rangle, \dots, \langle/\texttt{KB}\rangle) \\ 1 & \text{otherwise} \end{cases}\] By setting \(m_t = 0\) for tokens within the retrieval-result span, the model is not penalized for failing to predict the returned object \(o\) from the context alone. Instead, supervision is concentrated on the surrounding natural language and on the retrieval query that obtains the relevant KB evidence. This reduces direct supervision pressure to store KB facts in the model parameters and encourages the model to condition factual generation on the retrieved KB object.

3.4 Inference↩︎

Before generation begins, we pre-process the prompt to bring any entities into the form that KARLA-models can digest. More precisely, any entity mention \(e\) in the prompt is replaced by ⟨KB⟩\(e^*\): \(e^*_{desc}\)⟨/KB⟩, where \(e^*\) is the canonical KB entity of \(e\). To determine \(e^*\), we use a two-stage pipeline inspired by [26]:

1. Candidate Retrieval: A bi-encoder (all-MiniLM-L6-v2 [30]) retrieves the top 10 candidate entities from a FAISS [31] index built over entity embeddings.

2. Contextual Re-ranking: A small language model (Qwen3 1.7B) then re-ranks these candidates using the surrounding generation context to resolve ambiguities and determine \(e^*\).

This allows the model to condition any following tokens on \(e^*\).

The generation then proceeds auto-regressively. A KB query is triggered when the generation terminates in a sequence of the form: \[\langle{}r\rangle\, \langle\texttt{subj}\rangle \, s \, \langle/\texttt{subj}\rangle\] where \(r\) is a relation-specific token. When detected, the model generation is paused to resolve \(s\) to a canonical entity \(s^*\) of the KB, in the same way as before. The system then attempts to fetch an object \(o\) such that \(\langle{}s^*, r, o\rangle \in \text{KB}\). The generation continues by appending either the retrieved object or a failure signal to the sequence: \[\text{Output} = \begin{cases} \langle\texttt{KB}\rangle \, o : o_{\text{desc}} \, \langle/\texttt{KB}\rangle & \text{if } \langle{}s^*, r, o\rangle \in \text{KB} \\ \langle\texttt{KB\_FAIL}\rangle & \text{otherwise} \end{cases}\] In the event of a \(\langle\texttt{KB\_FAIL}\rangle\), the model falls back to its internal parametric weights to complete the sequence. This makes it possible to distinguish content that comes from the KB from content that comes from parametric memory.

4 Experiments↩︎

4.1 Experimental setup↩︎

4.1.0.1 Knowledge bases.

We instantiate KARLA on two knowledge bases with complementary domains: YAGO [32], a general-domain encyclopedic KB, and PrimeKG [33], a biomedical KB. Using both resources allows us to evaluate whether the method generalizes across open-domain and specialized scientific knowledge. Detailed statistics for the KBs are in Appendix 7.

4.1.0.2 Training data.

For both KBs, we construct the training corpus as explained in Section 3.2. We sample \(T = 1000\) triples per relation with \(k = 7\) relations per subject, over \(|\mathcal{R}| = 99\) relations for YAGO (after filtering non-semantic predicates such as yago:url) and \(|\mathcal{R}| = 18\) for PrimeKG. As shown in Figure 2, this yields a near-balanced distribution of relations in the corpus. The resulting training corpus contains \(36{,}929\) paragraphs for YAGO and \(10{,}470\) paragraphs for PrimeKG (example outputs are given in Tables 8 and 9 in Appendix 14). We also generate a test corpus with the same procedure, with subjects different from the ones in the training corpus. This yields \(1{,}389\) YAGO and \(382\) PrimeKG paragraphs.

4.1.0.3 KARLA configurations.

All KARLA-models are based on Qwen 3-Base [34] and fine-tuned with LoRA [35] applied to all attention and MLP projection matrices, following [36]. We keep the optimization setup fixed across model sizes: learning rate \(10^{-5}\), LoRA rank \(64\), and scaling factor \(\alpha{=}128\) (higher rank and \(\alpha\) improve convergence, see Fig. 5 Appendix 10). We train on YAGO for 3 epochs and on PrimeKG for 5 epochs, holding out 10% of each corpus for validation; full training details are in Appendix 9. We evaluate under several configurations that share the same KB, and the same training corpus, differing only at inference:

4.1.0.4 Competitors.

We compare against four families of competitors:

4.1.0.5 Experiments.

First, we study the impact of KARLA on the perplexity of the generated tokens (Section 4.2). Second, we study the impact of KARLA on the factuality of the generated text (Section 4.3). Finally, we study how KARLA fares when the KB contradicts the LLM’s parametric knowledge (Section 4.4).

4.2 KARLA reduces perplexity↩︎

To analyze how KARLA benefits from executing queries over the KB, we measure the perplexity of KARLA on our test corpus. We ask whether the reduction in uncertainty from conditioning on KB evidence is large enough to compensate for the cost of generating the query. For the competitors, we report standard perplexity: \[\mathrm{PPL}(x) = \exp\left( -\frac{1}{N} \sum_{i=1}^{N} \log p_\theta(x_i \mid x_{<i}) \right)\] For KARLA models, we report target-normalized masked perplexity. Let \(m_j \in \{0,1\}\) denote a scoring mask over the augmented sequence. The mask is set to zero for KB-returned objects, since these tokens are supplied by the external executor rather than predicted by the model. It is set to one for all tokens generated by the model, including test tokens and query tokens. We define: \[\mathrm{PPL}_{\mathrm{aug}}(\tilde{x}) = \exp\left( -\frac{1}{N} \sum_{j=1}^{M} m_j \log p_\theta(\tilde{x}_j \mid \tilde{x}_{<j}) \right)\] This is not the ordinary perplexity of the augmented sequence: the denominator is the number of tokens \(N\) in the non-augmented sequence, not the number of scored tokens nor the full augmented length \(M\). The metric thus measures the description length of the original passage while charging the model for the additional inline query tokens. A lower score means that the evidence obtained through KB execution reduces uncertainty enough to outweigh the overhead of producing the KB query.

5pt

Table 1 shows that KARLA improves this target-normalized score across both KBs and all model sizes. Compared with the Parametric SFT baseline, KARLA models lower perplexity for every evaluated Qwen 3 model. The mean score over the four model sizes decreases from \(7.03\) to \(5.89\) on YAGO and from \(4.15\) to \(3.28\) on PrimeKG, corresponding to relative reductions of \(16.2\%\) and \(21.0\%\). These gains hold despite the fact that KARLA models are penalized for generating queries, indicating that learned KB access provides useful evidence beyond what is captured by parametric-only fine-tuning. In the evaluation corpus, \(22\%\) of YAGO tokens and \(13\%\) of PrimeKG tokens are supplied directly by KB retrieval. A comparable share of the generated text is grounded in an explicit triple.

KARLA-empty-KB degrades performance from \(5.89\) to \(7.40\) on YAGO and from \(3.28\) to \(4.46\) on PrimeKG on average, showing that the improvement is not explained merely by tool syntax, sequence formatting, or tokenization artifacts, but by the actual values returned by the KB queries. KARLA-raw falls below Parametric SFT, indicating that KARLA trades standard text performance for inline-query specialization.

4pt

We next assess inline-query accuracy on the test corpus, measured separately for the relation token, the subject span, and their joint prediction. Table 2 shows that KARLA can indeed predict the subject and the relation with high accuracy. The accuracy is slightly lower for the relations on YAGO, likely because its relations are more heterogeneous and because multiple predicates can be plausible from similar textual contexts. The relation accuracy on PrimeKG reaches \(97.8\%\), indicating that its biomedical schema provides a more learnable relation-selection problem.

Together with the empty-KB ablation, this diagnostic supports the interpretation that KARLA’s gains come from learned use of external evidence rather than from sequence-format artifacts. The model reliably learns to ground subjects and to issue executable queries.

4.3 KARLA improves factuality↩︎

We next evaluate whether learned KB execution improves factuality, using YAGO as the KB. We consider both short-form question answering on the long-tail subset of PopQA [6] and long-form generation on the first 100 entities from FActScore [37]. Following [38], we replace ambiguous PopQA subject mentions with their canonical Wikipedia titles to isolate factual recall from entity disambiguation. Remaining setup details are reported in Appendix 13.

We compare KARLA against base models, 1-hop graph RAG, prompted tool use, and LMLM [23] where applicable. Table 3 reports the results.

The base LM performs poorly on both datasets regardless of size, confirming that parametric knowledge alone is insufficient, especially for the long-tail questions of PopQA. Tool-schema prompting improves over the baseline only modestly: the model sees the relation schema at inference time but is not trained to decide when to call the KB, which relation to select, or how to format an executable retrieval path. A single well-placed query suffices for PopQA, which the prompted model can partially manage zero-shot; long-form generation, however, requires chaining several calls, and the model typically issues one before reverting to parametric memory.

LMLM outperforms the prompted baselines but still falls behind KARLA. Several factors plausibly contribute: (1) LMLM is pre-trained from scratch on Wikipedia, whereas KARLA inherits the general linguistic competence from Qwen3; (2) KARLA queries a curated KB with a closed schema, whereas LMLM’s database is built from GPT-4o annotations of Wikipedia, so annotation errors propagate into retrieval; and (3) LMLM stores entities and relations as free-form surface strings resolved by fuzzy matching, whereas YAGO returns canonical typed values directly.

1-hop graph RAG performs strongly, as expected: the target entity’s one-hop neighborhood already contains the evidence needed to answer the query, though the model becomes noticeably more conservative (fewer extracted claims on FActScore Table 7 in Appendix 13). KARLA-no-desc surpasses graph RAG on PopQA and falls only slightly behind on FActScore, suggesting that lightweight entity-level context is especially useful in the multi-relation long-form setting. The full KARLA outperforms graph RAG on both datasets. KARLA also generates more atomic claims, which means that the gains do not come from being more conservative. Rather, the difference is structural: in graph RAG the model selects from the retrieved neighborhood and can still fall back on parametric memory, whereas in KARLA the answer is supplied directly by the KB.

5pt

4.4 KARLA can deal with updated facts↩︎

Finally, we evaluate how KARLA and the competitors fare when the KB is updated. In our previous evaluations, the factual information expected from the model was largely in line with what was seen at pretraining time. Here we move to a setting where this alignment no longer holds, as is the case in practice. To this end, we introduce Counterfactual YAGO, a controlled update benchmark derived from YAGO. Unlike unlearning benchmarks such as TOFU [39] or RESTOR [40], our goal is not to remove knowledge from the model or recover from training contamination. Rather, we evaluate factual overriding, where the system has to reply with updated information even if the old information is strongly represented in the pretrained model’s parametric memory. We construct the benchmark by sampling 400 entities across four Wikipedia popularity quartiles. For each target entity, we keep the subject fixed but replace all of its facts by all of the facts of another entity of the same type (top-level class in YAGO). For example, the city of Paris could receive its population, location, area, and other properties from London. In this way, internal consistency is preserved (e.g., population size and area correlate).

We compare three update mechanisms on a shared test set of QA prompts (1072 in total covering 52 relations) whose correct answers are also determined by the counterfactual KB. We report Exact Match accuracy against the counterfactual object. First, for the parametric baseline, we continue LoRA fine-tuning on a separate training set of QA pairs whose answers are determined by the counterfactual KB. Training and test sets cover the same entities and facts but use disjoint templates, so improvements reflect learned facts rather than memorized surface forms (Table 13 Appendix 14). This baseline therefore measures how many gradient steps are needed to absorb the revised facts into the model weights, even when the parametric updater is trained directly on the update task. Second, for KARLA, we keep the model parameters fixed and replace only the inference-time KB; evaluation is zero-shot. Third, we evaluate a 1-hop graph RAG baseline, where the local counterfactual KB neighborhood of the linked entity is placed in the context window; evaluation is also zero-shot.

6pt

Figure 3 shows that LoRA baselines require continued optimization to approach acceptable performance. Larger models adapt faster, but all parametric variants still require hundreds of update steps. KARLA, in contrast, updates factual behavior immediately by replacing the KB, reaching \(96.1\%\) accuracy without any additional gradient steps.

The 1-hop graph RAG baseline reaches only \(87.3\%\) overall accuracy despite being given the local KB neighborhood in context (prompt in Table 10 Appendix 14). While this number is sensitive to prompt phrasing, the popularity-stratified results in Table 4 isolate a failure mode that prompt engineering is unlikely to close: accuracy drops from \(89.1\%\) on Q1 to \(77.8\%\) on Q4, precisely where the model is likely most certain about its own knowledge, and is unlikely to accept conflicting information from its prompt. The model may ignore the retrieved field and revert to its memorized answer for the original entity (Example 14 Appendix 14), or fall back to parametric knowledge that is itself wrong, producing a value matching neither the KB nor the real world (Example 15 Appendix 14). These results support the central claim of factual externalization: once the model has learned query formulations, factual values can be changed outside the model. The remaining errors are primarily incorrect relation selection or entity-resolution mistakes, rather than failures to store the revised facts in the parameters.

4.5 Ablations↩︎

4.5.0.1 Special-token vs.free-form predicates.

KARLA matches its free-form counterpart on inline-query accuracy (Fig. 7 Appendix 11) and slightly outperforms it on PopQA, while eliminating the up to 11% out-of-schema predicate emissions (Table 6 Appendix 13).

4.5.0.2 Empty KB.

KARLA-empty-KB degrades both perplexity (Table 1) and factuality on PopQA and FActScore (Table 3), confirming that KARLA’s gains come from the interaction with the KB.

4.5.0.3 Entity descriptions.

KARLA-no-desc loses 13–23 points on PopQA relative to KARLA and a comparable margin on FActScore (Table 3), indicating that lightweight semantic context about the queried entity is needed for the model to plan useful lookups even when the atomic values themselves come from the KB.

4.5.0.4 Training corpus size.

Figure 6 Appendix 10 shows that inline-query accuracy is near zero for \(T \leq 100\), jumps sharply at \(T = 500\), and converges across all four Qwen3 sizes at \(T = 1000\).

5 Conclusion↩︎

KARLA shows that pretrained language models can be post-trained to pull factual information from a KB, allowing smaller models to match or surpass larger ones on factual questions, and to be redeployed without retraining when KB facts (or even the entire KB) change as long as the relation schema is unchanged. Retrieval and parametric knowledge are entangled rather than substitutable: issuing a useful query requires the model to identify the right subject from context and select the right relation from a closed inventory, both of which draw on linguistic and world knowledge inherited from pretraining. Future work can investigate how KARLA can be extended to multi-hop queries, which require composing several KB lookups. All our code and data is available in the supplementary material and will be made available publicly.

6 Limitations↩︎

6.0.0.1 Predicate vocabulary is closed by construction.

Each relation is represented as an atomic special token whose embedding is learned during fine-tuning. KARLA models can therefore generalize to new entities, but extending the schema to new predicates requires additional training.

6.0.0.2 Entity disambiguation is required.

KARLA requires the disambiguation of named entities. This is an open domain of research, with no perfect approaches. Yet, the problem is orthogonal to KARLA, which can work with any approach in a drop-in fashion. KARLA-no-disambig, which replaces the two-stage entity resolver with top-1 bi-encoder retrieval, changes PopQA accuracy by around one point at every model size (Table 6 in Appendix 13). This modest effect reflects the pre-disambiguated nature of PopQA, whose ambiguous mentions have been substituted with canonical Wikipedia titles; we expect disambiguation to matter substantially more in a real-world deployment.

6.0.0.3 Coverage inherited from the KB.

Since factual values come from the KB, queries to relations or entities that are not in the KB fall back to parametric memory, where our grounding guarantees no longer hold. This dependence is the price of treating the KB as the source of truth.

7 KB Details↩︎

Table 5 summarizes the two knowledge bases used in our experiments. YAGO provides broad encyclopedic coverage over general-domain entities, while PrimeKG provides a biomedical graph organized around diseases, drugs, genes, proteins, biological processes, phenotypes, and related clinical concepts. For each KB, we report the published graph scale together with the relation inventory used by KARLA after preprocessing.

5pt

8 Predicate-Balanced Sampling↩︎

We sample entities using a greedy procedure that prioritizes under-represented predicates. Let \(\mathrm{KB}\) be the knowledge base, \(\mathcal{E}\) its entities, and \(\mathcal{R}\) its predicates after filtering non-semantic metadata predicates. \(T\) denotes the target number of occurrences, and \(C = \alpha T\) is a saturation cap with \(\alpha > 1\). The parameter \(k\) is the maximum number of predicates sampled per entity.

We define: \[\mathcal{E}(r) = \{e \in \mathcal{E} \mid \exists o,\;(e,r,o) \in \mathrm{KB}\},\] the entities having predicate \(r\), and \[\mathcal{R}(e) = \{r \in \mathcal{R} \mid \exists o,\;(e,r,o) \in \mathrm{KB}\},\] the predicates attached to entity \(e\).

Here, \(r^*\) is the under-represented anchor predicate, \(\mathcal{A}\) is the set of eligible additional predicates attached to the sampled entity, \(\mathcal{S}_{\mathrm{add}}\) is the set of additional predicates sampled from \(\mathcal{A}\), \(\mathcal{S}_e\) is the final predicate set used for entity \(e\), and \(\mathcal{F}_e\) is the set of retrieved facts added to the dataset. Entities are sampled with replacement, while additional predicates are sampled without replacement for each entity.

8.0.0.1 Balance guarantee.

We show that Algorithm 4 produces a corpus whose relation frequencies are bounded, independently of the natural distribution of the KB.

8.0.0.2 Proof.

Since \(\mathcal{R}\) is constructed from the KB, every relation has at least one supporting triple, so \(\mathcal{E}(r) \neq \emptyset\) for all \(r \in \mathcal{R}\).

Lower bound. The selection step (line 5) picks \(r^* \in \arg\min_{r \in \mathcal{U}} \mathrm{count}(r)\), where \({\mathcal{U} = \{r \in \mathcal{R} : \mathrm{count}(r) < T\}}\). The uniform sample in line 6 is always well-defined since \(\mathcal{E}(r^*) \neq \emptyset\). Each iteration increments \(\mathrm{count}(r^*)\) by exactly one (line 12). The loop terminates only when \(\mathcal{U} = \emptyset\), i.e., when \(\mathrm{count}(r) \geq T\) for every \(r \in \mathcal{R}\). Sampling entities with replacement means relations with \(|\mathcal{E}(r)| < T\) are still pushed to \(\mathrm{count}(r) \geq T\) by repeatedly resampling from the same small entity set.

Upper bound. A relation \(r\) can be incremented in two ways: (i) with \(r^*\), which requires \(\mathrm{count}(r) < T\) (via membership in \(\mathcal{U}\)), or (ii) as an expansion relation in \(\mathcal{S}_{\mathrm{add}}\), which requires \(\mathrm{count}(r) < C\) (line 7). Since \(T \leq C\), in both cases \(\mathrm{count}(r) < C\) before the increment, so \(\mathrm{count}(r) \leq C\) after it. A single iteration increments \(\mathrm{count}(r)\) by at most one hence \({\mathrm{count}(r) \leq C}\).

Termination. The total count \(\sum_{r \in \mathcal{R}} \mathrm{count}(r)\) strictly increases by at least one per iteration (via the anchor \(r^*\)) and is bounded above by \(\sum_{r \in \mathcal{R}} C = \alpha T |\mathcal{R}|\), so the algorithm terminates in finitely many iterations.

9 Training details↩︎

We fine-tune Qwen3-Base [34] at four scales (0.6B, 1.7B, 4B, 8B) with LoRA [35] applied to all attention and MLP projection matrices within each transformer block (rank \(r=64\), scaling factor \(\alpha=128\)). The base model weights are kept frozen. In addition to the LoRA adapters, we train the embeddings of the newly introduced predicate tokens and the output projection, since both are randomly initialized for tokens absent from the base vocabulary. We optimize with AdamW (learning rate \(10^{-5}\), no weight decay) using a linear decay schedule without warmup, at a per-device batch size of 16, maximum sequence length of 512 tokens, and bf16 mixed precision. Each model is trained on a single NVIDIA L40S (48GB).

10 Scaling Laws / LoRA search↩︎

Performance improves consistently with both model size and LoRA capacity, as we sweep the rank \(r \in \{16, 32, 64\}\) with \(\alpha = 2r\) (Figure 5).

11 Special-Token vs. Free-Form Predicate Representation↩︎

We compare the special-token predicate representation used in the main paper against a free-form alternative in which relations are not special tokens.

11.0.0.1 Free-form variant.

The free-form variant preserves every other aspect of KARLA: masked retrieval-query objective, synthetic data pipeline, two-stage entity resolution, modifying only how predicates are encoded. Instead of an atomic predicate token \(\langle r \rangle\), the relation is wrapped between two special tokens \(\langle \texttt{KB\_query} \rangle\) and \(\langle \texttt{/KB\_query} \rangle\).

Predicate decoding is left free-form, and a KB query is triggered upon emission of a well-formed \(\langle\texttt{KB\_query}\rangle \, r \, \langle\texttt{/KB\_query}\rangle \langle\texttt{subj}\rangle \, s \, \langle\texttt{/subj}\rangle\) block. If \(r\) does not match any predicate in \(\mathcal{R}\), the executor returns \(\langle\texttt{KB\_FAIL}\rangle\) and the model falls back to parametric generation.

11.0.0.2 In-distribution inline-query accuracy.

Figure 7 reports exact-match accuracy across four Qwen3 model sizes, with both variants trained under the same LoRA configuration. The two representations reach comparable accuracy at convergence. The free-form variant converges slightly faster, which we attribute to its trigger being shared across predicates rather than tied to a per-predicate token.

11.0.0.3 Out-of-distribution behavior on PopQA.

The two representations diverge under distribution shift. Table 6 reports PopQA accuracy alongside the rate of out-of-schema emissions for the free-form variant: predicate strings emitted between \(\langle\texttt{KB\_query}\rangle\) and \(\langle\texttt{/KB\_query}\rangle\) that do not correspond to any \(r \in \mathcal{R}\). Such emissions trigger \(\langle\texttt{KB\_FAIL}\rangle\) and force a parametric fallback, regardless of whether an in-schema predicate could have answered the query. By construction, the special-token variant cannot produce them.

12 Baseline Details↩︎

12.0.0.1 1-hop graph RAG.

The 1-hop graph RAG baseline tests whether placing the full KB neighborhood of the target entity in the context window is sufficient to ground generation. Given an input prompt, we first identify the target entity using KARLA’s two-stage entity linker, then retrieve its complete one-hop neighborhood. Each fact is verbalized through a fixed per-relation template. The full verbalized neighborhood is then prepended to the user query (full template in Table 10 Appendix 14).

12.0.0.2 Tool-schema guided.

The tool-schema guided baseline tests whether a base LM, given the KB schema in context, can issue executable retrieval calls without fine-tuning. The full YAGO relation inventory is enumerated in the system prompt together with a description per relation, followed by two to three in-context examples illustrating the expected output format (Tables 11 and 12 Appendix 14). The model is instructed to emit calls wrapped in [GRAPH_QUERY]...[/GRAPH_QUERY] tags, with the subject and relation separated by <SEP>.

At inference, we scan the generated text for well-formed GRAPH_QUERY blocks. The subject is resolved through the same two-stage entity-linking pipeline used by KARLA (Section 3.4); the relation must match a string in \(\mathcal{R}\) verbatim, including the angle brackets. Each resolved block is replaced by the retrieved KB object, and generation is resumed conditioned on the inserted value. If the relation is not in \(\mathcal{R}\), or no triple \(\langle s^*, r, o\rangle\) exists in the KB, the block is replaced with an empty string and the model continues without correction.

13 Experiments setup↩︎

All KARLA models are evaluated using greedy decoding.

13.0.0.1 PopQA.

We evaluate on the long-tail entity subset of PopQA, following [41], which contains 1,399 entities with fewer than 100 weekly Wikipedia views. We measure performance with Exact Match, checking whether the gold answer appears in the generated output. To be consistent with [23], and because we evaluate base (non-instruction-tuned) models, we avoid probing instruction-following ability by re-formulating each question as a completion prompt. Table 6 contains the detailed results for the KARLA models.

13.0.0.2 FActScore.

FActScore [37] is an open-domain biography generation benchmark. It evaluates factuality by extracting atomic claims from the generation and verifying each one against Wikipedia using an LLM-based RAG pipeline. We follow the official evaluation protocol with ChatGPT, reporting both the factuality score and the number of extracted atomic claims. We prompt the model on the first 100 biography queries with a maximum generation length of 256 tokens, using the same fixed template as [23]: “Tell me a bio of \(\langle\)name\(\rangle\). \(\langle\)name\(\rangle\) is”. Table 7 contains the detailed results for the KARLA models and baselines.

5pt

5pt

14 Prompts and Outputs↩︎

This contains the prompt templates referenced in the main paper: verbalizer prompts (Tables 8, 9), the 1-hop graph RAG prompt (Table 10), the tool-schema prompts (Tables 11, 12), example train/test pairs from Counterfactual YAGO (Table 13), and illustrative RAG failure cases on Counterfactual YAGO (Tables 14, 15).

6pt

3pt

6pt

6pt

6pt

6pt

6pt

6pt

References↩︎