1 Introduction↩︎

Knowledge base question answering (KBQA) aims to answer natural-language questions using facts stored in a structured knowledge base. The challenge is especially pronounced for complex questions, which may require multi-hop reasoning, constrained relations, multiple subjects, or numerical operations [1]. Prior work distinguishes two main approaches: information retrieval-based (IR-based) methods build a question-specific graph and derive answers by ranking entities or generating from the retrieved evidence, whereas semantic parsing-based (SP-based) methods map the question to a symbolic logical form and execute it against the knowledge base. The distinction concerns the intermediate representation and final output, not whether retrieval is used: an SP-based system may retrieve entities, relations, and subgraphs while still producing an executable logical form.

We focus on SP-based KBQA because an explicit query provides a verifiable reasoning artifact and supports compositional operators such as joins, aggregation, comparison, and Boolean queries. Classical SP-based systems comprise four modules—question understanding, logical parsing, KB grounding, and KB execution [1]—where grounding aligns entities and predicates with the target schema. Large language models (LLMs) have recently made these modules interactive rather than sequential. Interactive-KBQA [2] views the LLM as an agent and the KB as an environment, alternating between a thought, a tool action, and an observation to search nodes, retrieve graph patterns, and execute SPARQL, and SPINACH [3] adopts a related ReAct-style process on Wikidata. We refer to this family as interactive agentic semantic parsing: logical-form construction and KB grounding are interleaved over multiple turns.

This interaction reduces dependence on a single unsupported generation, but it also makes the agent–KB interface a central part of the parser: at every turn the agent can ground the next predicate only from the candidates exposed by its tools. Interactive-KBQA semantically ranks local graph patterns, whereas SPINACH prunes an entity neighborhood with an additional LLM call. For a complex question, each added relation, constraint, or intermediate variable enlarges the grounding search space, so a locally plausible property choice can send the entire partial query down an incorrect branch.

We identify a systematic weakness in this grounding stage, which we call type-blind grounding. Existing interactive parsers assess a candidate property from its label, description, or observed triples, without consistently conditioning on four pieces of symbolic state: the type of the current entity or variable, the property’s declared domain, its declared range, and the expected answer type. A property may therefore be lexically relevant—or observed on a noisy instance—yet be incompatible with the semantic role assigned to the variable, producing a syntactically valid triple pattern that executes to an empty answer set, after which the agent spends extra turns repairing the wrong branch. Type blindness is thus not merely a prompting problem but a KB-grounding problem caused by an unconstrained action space.

Existing uses of schema information do not directly solve this problem. Schema-in-context methods place OWL, SHACL, or ShEx descriptions in the prompt [4], [5], post-hoc validators such as OBQC [6] detect violations only after a query is generated, and instance-level constrained decoding [7] restricts token generation using KG connectivity but requires access to the decoder. None of these approaches turns graph-pattern retrieval itself into a schema-aware KB-grounding operation.

Our key idea is to use schema as a construction-time search-space constraint. Many KGs expose entity types through instance assertions and property semantics through domain, range, or type-constraint declarations. Although incomplete, this metadata provides a high-precision compatibility signal: a known domain conflict can eliminate a property before it reaches the controller, while an unknown property is retained rather than incorrectly rejected. Schema thus acts as a grounding prior, not an assumption that the KB is complete.

Building on this idea, we propose SAGA (Schema-Aware Grounding for Agentic Text-to-SPARQL Generation), an interactive SP-based framework that modifies the property-grounding interface while preserving executable SPARQL as the final output. SAGA maintains a persistent bidirectional type state—entity and variable types accumulate forward from KB observations, while an expected answer type propagates backward from the question—and at each lookup uses this state and a KG-specific schema index to remove properties with known domain conflicts, retain candidates whose schema is unknown, and display the remainder with inline domain and range annotations, replacing a separate LLM pruning call. Empirical schema induction and trace-local discoveries let the same logic operate across Wikidata and Freebase.

We evaluate SAGA on nine benchmark settings spanning Wikidata and Freebase. Under a common execution-based protocol, SAGA achieves the highest F1 on all nine settings and the highest EM on eight. Relative to the strongest non-SAGA baseline on each benchmark, SAGA improves F1 by 2.4–20.3 points on Wikidata and by 0.2–11.2 points on Freebase. It also lowers the empty-result rate on all five Wikidata settings for which this diagnostic is reported.

Our contributions are threefold:

• We position iterative text-to-SPARQL as interactive agentic semantic parsing and identify type-blind KB grounding as a distinct failure mode at the interface between logical parsing and the KG.

• We introduce SAGA, a training-free schema-aware grounding framework that combines persistent forward and backward type signals with construction-time candidate filtering, compact typed graph-pattern observations, and permissive handling of incomplete schemas.

• We evaluate SAGA against comparable SP-based systems across nine benchmark settings on Wikidata and Freebase and show consistent gains in answer F1 together with lower empty-result rates and fewer LLM pruning calls.

2 Related Work↩︎

2.1 Retrieval- and Semantic-Parsing-Based KBQA↩︎

Complex KBQA methods are commonly grouped into information retrieval-based (IR-based) and semantic parsing-based (SP-based) approaches [1]. IR-based methods retrieve question-specific subgraphs and derive answers through ranking, graph reasoning, or generation; recent LLM-based examples include Think-on-Graph [8] and Reasoning on Graphs [9]. SP-based methods instead produce executable logical forms that support compositional operators and verifiable execution. Because retrieval may also serve as an intermediate grounding step within semantic parsing, the key distinction is the output artifact. SAGA is SP-based, so we compare it with systems that generate executable SPARQL under compatible protocols rather than IR-based graph-reasoning methods.

2.2 LLM-Based Interactive Semantic Parsing↩︎

LLMs have shifted SP-based KBQA toward iterative logical-form construction. Direct generation must resolve query structure and KG grounding in one pass, which can yield hallucinated identifiers, incompatible predicates, or empty results. Interactive-KBQA [2] instead treats the LLM as an agent with tools for node search, graph-pattern retrieval, and query execution. SPINACH [3] similarly searches entities, inspects neighborhoods, and revises partial queries through execution feedback, while mKGQAgent [10] and GRASP [11] provide other agentic routes to executable logical forms. Their grounding tools, however, select or prune predicates mainly by lexical relevance, retrieved instances, or LLM judgments, without jointly using entity or variable types, property domains and ranges, and the expected answer type. SAGA complements these controllers by making the agent–KB grounding interface schema-aware.

2.3 Schema-Aware Grounding and Query Validation↩︎

Schema information has been used through prompt augmentation, constrained generation, and post-hoc validation. Data Shapes Prompting [4] provides SHACL or ShEx descriptions, and SPARQLGEN [5] supplies an RDF subgraph, but the model must still apply this context during generation. Graph-constrained Reasoning [7] restricts decoding using instance-level topology, which does not guarantee schema-level type compatibility and may require decoder access. OBQC [6] checks OWL domain and range violations after a query is formed. SAGA instead applies schema compatibility during graph-pattern grounding, before candidate predicates reach the controller.

3 Method↩︎

3.1 Overview↩︎

SAGA is an interactive semantic parser that helps an LLM construct an executable SPARQL query through repeated access to a knowledge graph (KG). Its key idea is simple: before candidate properties are shown to the LLM, SAGA checks whether their declared domain is compatible with the type of the entity or variable currently being explored, preventing clearly invalid property choices while retaining candidates whose schema is missing.

The workflow has three stages. First, SAGA builds a lightweight schema index that stores the available domain and range information for each property. Second, during an interaction, it maintains the types discovered for entities and SPARQL variables, together with the expected answer type inferred from the question. Third, each lookup call uses this information to filter and annotate the local neighborhood before returning it to the LLM, which then continues constructing, executing, and revising the query as in prior agentic KBQA systems. Thus, SAGA changes the agent–KG interface rather than the LLM or its token-level decoding process. Figure 1 illustrates this pipeline using the question “Where are both The Call of the Wild and White Fang set?”; the three lower panels correspond to the schema index, lookup-time type-constrained grounding, and persistent type state described in the following subsections.

3.2 Problem Setup and Interactive Query Construction↩︎

We represent a knowledge graph (KG) as \(\mathcal{G}=(\mathcal{E},\mathcal{P},\mathcal{T})\), where \(\mathcal{E}\) is the set of entities (e.g., Wikidata’s Q212 for the United States), \(\mathcal{P}\) is the set of properties—the labeled edges that connect a subject to an object in a triple (e.g., P17, “country”)— and \(\mathcal{T}\) is the set of subject–property–object triples. Triple objects may be entities or literal values such as numbers or dates.

Every entity belongs to zero or more types (e.g., human or sovereign state). We write \(\mathcal{C}\) for the set of all types and \(\tau(e)\subseteq\mathcal{C}\) for the types declared for entity \(e\) (e.g., \(\tau(\texttt{Q212})=\{\text{sovereign state}\}\)). Each property \(p\) may additionally declare a domain \(\delta(p)\subseteq\mathcal{C}\) and a range \(\rho(p)\subseteq\mathcal{C}\): the domain lists the types an entity must have in order to appear as the subject of a \(p\)-triple, and the range lists the types required of the object. For instance, \(\delta(\texttt{P607})=\{\text{character, battle, sabotage}\}\) means only entities of those types are valid subjects of a P607 (“conflict”) triple. Either \(\delta(p)\) or \(\rho(p)\) may be empty when the KG does not declare this information for \(p\); the Schema Index and Type State subsection below explains how SAGA handles that case.

Given a natural-language question \(q\), the system constructs a SPARQL query \(\sigma\). Executing the final query on the KG produces the predicted answer set \(\hat{A}_q\): \[\hat{A}_q=\llbracket\sigma\rrbracket_{\mathcal{G}}, \label{eq:query-execution}\tag{1}\] where \(\llbracket\sigma\rrbracket_{\mathcal{G}}\) denotes the result of executing \(\sigma\) on \(\mathcal{G}\), i.e., the set of entities or literals returned by the SPARQL endpoint.

Following Interactive-KBQA [2], an LLM controller builds the query through a sequence of tool calls. The controller observes the question and the previous actions and observations, then chooses one of five actions: search for retrieving entities from a text query, lookup for inspecting the neighborhood of an entity, examples for viewing example uses of a property, execute for running a partial or complete SPARQL query, and stop for ending the interaction.

The main grounding bottleneck is lookup. For an entity \(e\), its raw outgoing neighborhood is \[\mathcal{N}(e)=\{(p,v)\mid(e,p,v)\in\mathcal{T}\}, \label{eq:raw-neighborhood}\tag{2}\] where \(p\) is a property and \(v\) is the corresponding object value; that is, \(\mathcal{N}(e)\) simply collects every (property, value) pair reachable by a single outgoing edge from \(e\)—the candidate triples that lookup would hand to the controller. A large entity may expose hundreds of such properties. Existing interactive parsers mainly prune them using lexical relevance or observed instances, without systematically checking whether a property’s schema is compatible with the current type. SAGA performs this check before the candidates are presented to the controller.

3.3 Schema Index and Type State↩︎

3.3.0.1 Schema index.

Before inference, SAGA creates one metadata entry per property, recording its known domain \(\delta(p)\) and range \(\rho(p)\) (defined above) so that they can be looked up without querying the KG at run time. For Wikidata, formal entries come from P2302 type constraints; for Freebase, they come from the curated fb_roles schema. When a property lacks a formal declaration, SAGA instead samples several subject–object pairs from the KG and stores the most frequently observed subject and object types as soft domain and range evidence. A property for which neither source is available is left unconstrained (\(\delta(p)=\rho(p)=\emptyset\)) rather than removed from the index.

3.3.0.2 Persistent type state.

During query construction, SAGA maintains a type state \(\Theta\): a running record that maps every entity or SPARQL variable encountered so far to the types currently known for it (formally, \(\Theta[x]\subseteq\mathcal{C}\) for entity or variable \(x\)). Looking up an entity adds its declared types \(\tau(e)\) to the state. Likewise, an explicit type pattern in a partial query, such as ?x wdt:P31 wd:Q5, records that the variable ?x is a human. Because \(\Theta\) persists across the whole interaction, information discovered in an early step can guide property selection later in the same interaction—this is what we mean by persistent.

Before the first tool call, the controller also predicts the expected answer type (Supplementary Appendix F.1): the type the final answer is expected to belong to (e.g., for “Where are both The Call of the Wild and White Fang set?”, the expected answer type is location). This expected type is used only as guidance, not as a hard filter: it can increase the priority of properties whose range matches it, but an intermediate variable may legitimately have a different type from the final answer, so it is never used to discard a property outright. SAGA additionally retains execution outcomes and newly observed type evidence in the interaction history so that the controller does not repeatedly make the same failed choice.

3.4 Type-Constrained Grounding↩︎

When lookup\((e)\) is called, SAGA retrieves the types \(\tau(e)\) of \(e\) (introduced above under Persistent type state) and the set \(P(e)\) of properties occurring in \(e\)’s raw neighborhood \(\mathcal{N}(e)\). A property \(p\in P(e)\) survives the filter unless it has a declared domain that conflicts with every one of \(e\)’s known types: \[P_{\tau}(e)=\{p\in P(e)\mid \delta(p)=\emptyset\;\lor\; \delta(p)\cap\tau(e)\neq\emptyset\}, \label{eq:type-compatible-properties}\tag{3}\] Intuitively, Eq. 3 keeps a property \(p\) in one of two situations: either \(p\)’s domain is unknown (\(\delta(p)=\emptyset\), so there is no basis for rejecting it), or \(p\)’s domain and \(e\)’s types overlap (\(e\) is a plausible subject for \(p\)). The only remaining case—a known domain that shares no type with \(\tau(e)\)—is a demonstrated type conflict, and it is the sole reason a property is removed. \(P_\tau(e)\) denotes the properties that survive this check, and every pair \((p,v)\in \mathcal{N}(e)\) whose property is not in \(P_\tau(e)\) is dropped from the observation returned to the controller. Consequently, a known domain conflict is rejected before generation, whereas missing schema information never causes rejection. When no type is known for \(e\) (i.e., \(\tau(e)=\emptyset\)), Eq. 3 would incorrectly reject every property that does have a declared domain, since intersecting any non-empty set with \(\emptyset\) is always empty; SAGA therefore skips the filter in this case and falls back to the same LLM-based neighborhood pruning used by the underlying interactive parser.

Figure 1 shows that this filtering is entity-type-dependent rather than property-specific: the same property can be retained for an entity of a compatible type and removed for one that is not. In the illustrated example, SAGA processes both literary-work entities Q476871 (The Call of the Wild) and Q152267 (White Fang) symmetrically: 12 of 24 and 12 of 22 properties are removed from each—those whose declared domain (e.g., human or country) conflicts with literary work—while work-compatible properties such as P136 (genre) are retained with inline domain annotations. The same property P136 would be removed for a human entity, since its declared domain does not include human. Recognizing both novels as the same type also enables the controller to construct a unified multi-entity SPARQL using VALUES and a GROUP BY/HAVING intersection to retrieve their shared narrative settings.

For each retained property, SAGA returns its identifier, label, short description, and available domain and range information, rendered in a compact entry of the form:

This representation makes the schema evidence visible to the controller at the moment of property selection. Because filtering and annotation are performed by the lookup tool itself, SAGA does not require a separate LLM call to prune the same neighborhood.

SAGA also accumulates schema evidence during execution. When a property with no indexed range returns non-empty results, the system inspects the types of a small sample of returned entities and records them as trace-local evidence. This evidence can guide later steps of the current interaction, but it does not overwrite the offline schema index or act as a hard constraint.

3.5 Agentic Loop↩︎

At each step, the controller selects one of the five actions described above. A lookup\((e)\) call runs the type-retrieval-and-filtering process of Eq. 3 ; an execute\((\sigma)\) call runs the (partial) query and updates the persistent type state \(\Theta\) with any newly observed types; and stop ends the interaction and returns the final query \(\sigma^{*}\). Every action and its resulting observation are appended to the interaction history \(H\), which the controller conditions on at the next step. Algorithm 2 formalizes this loop: \(H\) contains the action–observation history, \(a\) and \(o\) denote the current action and observation, \(\Theta\) is the persistent type state, and \(\sigma^{*}\) is the final query returned by the controller. TypeFilter and CompactFormat implement the type check and compact neighborhood rendering described in the Type-Constrained Grounding subsection, while LLMPrune is the fallback used when no entity type is available. The controller and answer-type prompts are provided in Supplementary Appendix F. SAGA operates at the tool-call level rather than during token decoding, so it is compatible with both open and closed LLMs and requires no access to model internals; unlike instance-level constrained decoding [7], its main constraints come from KG schema and can remain useful when local instance evidence is sparse.

4 Experiments↩︎

4.0.0.1 Datasets.

We evaluate SAGA on nine benchmark settings over Wikidata and Freebase. For Wikidata, QALD-7 [12] contains 45 test questions, QALD-9-plus [13] provides a 127-question Wikidata test split, and QALD-10 [14] contains 383 test questions. WikiWebQuestions (WWQ) is the Wikidata adaptation of WebQuestionSP [15], with 450 development and 1,409 test questions. The SPINACH dataset [3] contains 149 development and 156 test questions collected from complex in-the-wild questions. LC-QuAD 2.0 [16] contributes 826 Wikidata test questions, including multi-hop and Boolean queries.

For Freebase, WebQSP [17] contains 1,628 test questions with annotated topic entities and SPARQL queries. ComplexWebQuestions (CWQ) [18] contains 3,273 compositional test questions involving conjunctions, comparisons, and superlatives. GrailQA [19] contains 1,000 test questions covering i.i.d., compositional, and zero-shot generalization settings.

4.0.0.2 Baselines.

We compare SAGA with Zero-shot SPARQL, Entity Linking [1], Data Shapes Prompting [4], OBQC Repair [6], mKGQAgent [10], GRASP [11], Interactive-KBQA [2] and SPINACH [3]. These baselines produce executable SPARQL queries and are evaluated using execution-based metrics. SPINACH shares SAGA’s multi-turn process of entity search, property inspection, query construction, and execution, while SAGA adds schema-aware grounding to the interaction process.

4.0.0.3 Metrics.

Let \(\hat{A}\) be the predicted and \(A^*\) the gold answer set for a question. We report two execution-based metrics averaged over all questions. Macro-averaged F1 is the harmonic mean of set precision and recall: \[\mathrm{F1} = \frac{1}{|Q|}\sum_{q \in Q} \frac{2\,|\hat{A}_q \cap A^*_q|}{|\hat{A}_q| + |A^*_q|},\] where \(|Q|\) is the number of test questions and an empty prediction yields \(\mathrm{F1}_q = 0\).

Exact Match (EM) is the fraction of questions for which the prediction is a perfect set match with the gold: \[\mathrm{EM} = \frac{1}{|Q|}\sum_{q \in Q} \mathbf{1}\!\left[\hat{A}_q = A^*_q\right] = \frac{1}{|Q|}\sum_{q \in Q} \mathbf{1}\!\left[\mathrm{F1}_q = 1\right].\]

3pt

3.5pt

We additionally report the empty-result rate, the proportion of generated queries that return no answer from the SPARQL endpoint.

4.1 Implementation Details↩︎

All experiments are conducted with a 120B-parameter open-weight instruction-tuned LLM (gpt-oss-120B [20]) served through a local vLLM endpoint. We use greedy decoding (temperature \(= 0\)) and cap each response at 2,048 tokens to obtain deterministic outputs. SAGA is additionally evaluated with LLaMA-3.1-70B [21] across all nine benchmark settings under the same decoding configuration. For all baseline systems, we use gpt-oss-120B with identical inference settings so that differences in results reflect only the grounding strategy.

5 Results↩︎

5.1 Results on Wikidata↩︎

Table 1 reports execution-based F1 and EM on six Wikidata benchmarks. SAGA with gpt-oss-120B achieves the highest F1 on all six datasets.

Relative to SPINACH, SAGA with gpt-oss-120B improves F1 by 2.4–20.3 points across the six datasets (Table 1), with the largest gains on QALD-7 and WWQ-test, where query construction depends strongly on selecting valid properties over multiple turns. The gap between SAGA and SPINACH indicates that exposing schema constraints during property grounding is more effective than an otherwise similar multi-turn interaction without the type-aware interface. A manual error analysis on QALD-7 and QALD-9-plus confirms that 36.9% of SPINACH failures (41 of 111) are caused by type-blindness—selecting a property whose declared domain conflicts with the entity type—which SAGA corrects through construction-time filtering (Supplementary Appendix A). SAGA achieves these gains largely without increasing the interaction budget: across the four reported datasets it uses the same number of actions as SPINACH on QALD-7, fewer on QALD-9-plus and SPINACH-test (a 29% reduction on SPINACH-test), and slightly more on QALD-10 (Supplementary Appendix B).

Table 3 separates LC-QuAD 2.0 into Boolean and factoid questions. SAGA raises Boolean F1 from 25.6 to 55.6, a gain of 30.0 points, while its factoid F1 is 1.0 point lower than SPINACH. The Boolean improvement is large enough to raise overall F1 from 25.83 to 28.25 and EM from 23.58 to 25.39. Thus, the overall LC-QuAD gain is concentrated in questions whose logical form requires a Boolean answer rather than an answer set.

4pt

The empty-result analysis in Table 4 provides complementary evidence: SAGA lowers the empty-result rate on every reported setting, most notably on WWQ-test (75.5% to 27.0%), QALD-9-plus (38.6% to 25.8%), and QALD-10 (32.4% to 21.9%). This is consistent with the intended role of schema-aware grounding, in which properties with known type conflicts are removed before they can enter a partial query.

3pt

5.2 Results on Freebase↩︎

Table 2 reports F1 and EM on WebQSP, CWQ, and GrailQA. Direct SPARQL generation remains weak on all three datasets, while the multi-turn SPINACH baseline performs substantially better, showing the importance of interacting with the KG during logical-form construction. SAGA further improves over SPINACH by 0.2–11.2 F1 across the three datasets (Table 2), though the margin on CWQ is small. These improvements show that schema-aware property grounding transfers beyond Wikidata and remains effective under a different KG schema and relation inventory.

5.3 Ablation Study↩︎

Table 5 evaluates the three grounding mechanisms defined in the ablation protocol on the five Wikidata benchmarks with complete ablation results, reporting the unweighted macro average across them. Full SAGA obtains the highest average F1 of 49.14. Removing persistent type state causes the largest average decrease (5.43 points), followed by removing type-constrained filtering (4.97 points), showing that both construction-time filtering and cross-turn type memory contribute substantially.

The contribution is not uniform across datasets. Type filtering is particularly important on QALD-7, QALD-9-plus, and SPINACH-test, where its removal lowers F1 by 6.84, 5.69, and 7.96 points; persistent type state has the strongest effect on QALD-7 and QALD-9-plus, with drops of 11.52 and 7.08 points. The answer-type prior has the smallest average effect of the three mechanisms: removing it lowers F1 on all five reported datasets, with the largest drop on SPINACH-test (4.52 points) and the smallest on WWQ-test (0.89 points). This is consistent with the prior’s role in biasing the controller toward Boolean (ASK) queries before any entity lookup (Supplementary Appendix D), while Full SAGA still achieves the best macro average.

5.4 Robustness to Incomplete Schema↩︎

Table 6 evaluates SAGA after retaining different fractions of Wikidata P2302 domain constraints. Full coverage obtains the highest average F1 (48.55) and the best result on QALD-7, QALD-9-plus, and SPINACH-test. At 0% coverage the average falls to 46.08, with drops of 8.92 points on QALD-7 and 2.35 on QALD-9-plus, supporting the usefulness of formal schema constraints.

4pt

The trend is not monotonic: QALD-10 performs best at 25% coverage, and several intermediate masks underperform both endpoints, showing that robustness depends not only on the amount of schema but also on which constraints are retained. Importantly, SAGA degrades gracefully when schema is absent because unknown properties remain available rather than being rejected. The best average at 100% coverage indicates that the complete schema is beneficial in aggregate, while the dataset-level exceptions motivate more selective or confidence-weighted constraint use in future work. Detailed qualitative examples and case studies, including both successful recoveries and a representative failure, are collected in the supplementary material.

6 Conclusion↩︎

We introduced SAGA, a schema-aware grounding framework for agentic SPARQL construction. SAGA maintains persistent entity and variable types together with an expected answer type, filters predicates with known domain conflicts, and annotates retained graph patterns with domain and range information while preserving candidates with incomplete schema through permissive fallbacks. Across Wikidata and Freebase benchmarks, SAGA improves answer F1, reduces empty-result queries, and removes a separate LLM pruning call. These results show that KG schema should shape the LLM’s grounding action space during logical-form construction, rather than serving only as prompt context or a post-hoc validator. Its main limitation is dependence on explicit type annotations, motivating richer type inference for incompletely typed entities.

Supplementary Material↩︎

7 SPINACH Error Distribution↩︎

Table 7 breaks down SPINACH’s failures on QALD-7 and QALD-9-plus into three mutually exclusive categories. Type blindness covers cases where the generated SPARQL applies a property whose declared domain is incompatible with the subject entity’s type (e.g., applying a relation whose domain is taxon to a human entity). Entity linking covers cases where the entity search returns a wrong QID that no type check could correct (e.g., the correct entity does not appear in the candidate list). Structure error covers cases where the entity and properties are correct but the SPARQL form is wrong (e.g., a missing path expression or wrong aggregation).

5pt

Type blindness is the single largest error category on QALD-7, accounting for 64.3% of failures. On QALD-9-plus the picture is more balanced: entity linking errors are slightly more frequent (42.2%), though SAGA’s type-constrained grounding also indirectly helps entity linking by signalling to the LLM that a resolved entity is the wrong type (see Appendix 11.2). Across both datasets, 36.9% of failures are type-blindness errors that SAGA’s construction-time filtering is designed to address.

8 Agent Action Efficiency↩︎

Table 8 compares the average number of agent actions per question between SPINACH and SAGA (both using gpt-oss-120B). SAGA uses the same number of total actions on QALD-7, fewer on QALD-9-plus and SPINACH-test, and slightly more on QALD-10. The average lookup calls per question increase slightly on QALD-7 and QALD-10 (SAGA retries entity lookups more often when a type mismatch signals the wrong entity), but decrease on SPINACH-test where SAGA’s tighter grounding reduces dead-end exploration. Overall, schema-aware grounding adds no meaningful interaction overhead.

4pt

9 Neighborhood Filtering Statistics↩︎

Table 9 reports neighborhood filtering statistics measured across the SAGA evaluation sets that contain gold SPARQL annotations. For each dataset, Edge reduction is the fraction of candidate property-value pairs removed by type-constrained filtering. Gold-prop recall is the fraction of gold SPARQL properties that survive the filter (i.e., still appear in the schema-compact neighborhood shown to the LLM).

5pt

On average, SAGA removes 25.8% of raw candidate edges per entity lookup while retaining 62.1% of the gold properties needed to answer each question. The 37.9% of gold properties that do not survive the filter represent cases where the Wikidata P2302 domain constraint is either too strict or uses a different class hierarchy path than the entity’s declared types. Two factors mitigate this loss. First, the agent can still discover a filtered property if another entity in the same interaction turn exposes it without a conflict. Second, SAGA retains all properties whose schema is absent (see Type-Constrained Grounding in the main paper), so the filter conservatively passes anything it cannot classify. The net gain indicates that removing clearly wrong candidates outweighs the occasional loss of a correct one.

10 Boolean Question Analysis↩︎

Table 10 reports Boolean results for four Wikidata datasets, extending the LC-QuAD 2.0 breakdown in the main paper. SAGA improves Boolean F1 on every dataset, with the largest gain on QALD-9-plus (33.3% \(\to\) 100.0%, \(+\)​66.7 points), followed by QALD-7 (25.0% \(\to\) 62.5%, \(+\)​37.5 points), the already-reported LC-QuAD 2.0 gain (25.6% \(\to\) 55.6%, \(+\)​30.0 points), and QALD-10 (27.9% \(\to\) 52.5%, \(+\)​24.6 points). The 100% Boolean F1 on QALD-9-plus reflects that SAGA correctly identifies the answer type for all three Boolean questions in that test set.

5pt

The answer-type prior (see Agent Loop in the main paper) is the primary driver of the Boolean improvement: it biases the LLM toward ASK queries before any entity lookup is issued. The high accuracy of this prior (100% on QALD-7, QALD-9-plus, and QALD-10 Boolean questions; 91% on LC-QuAD 2.0) confirms that the expected answer type can be reliably inferred from the question surface form for Boolean queries.

11 Additional Qualitative Examples↩︎

This appendix consolidates all qualitative results and case studies from the paper. It provides four Type-A case studies (SPINACH wrong, SAGA correct) that illustrate different manifestations of type-blind grounding, a compact set of additional prediction-level snapshots drawn directly from the saved result files, and a structured failure analysis of several Type-B cases (SAGA also fails).

11.1 Entity-Type Recovery: “Was U.S.president Jackson involved in a war?”↩︎

11.2 Entity Type Mismatch: “Who is the author of WikiLeaks?”↩︎

11.3 Tiny-Neighborhood Signal: “What is the highest place of Karakoram?”↩︎

11.4 Property-Level Type Error: “In which films directed by Garry Marshall was Julia Roberts starring?”↩︎

11.5 Additional Prediction Snapshots↩︎

Unlike the manually inspected figures above, the stored result files do not preserve full neighborhood renderings for every question. Table 11 therefore reports only query-level differences that are directly observable from the saved SPINACH per_example.csv outputs and the matching SAGA predictions.json files.

5pt

11.6 SAGA Failure Cases and Discussion↩︎

SAGA’s schema constraints reduce type-incompatible search, but they do not guarantee correct query composition or predicate interpretation. Table 12 collects five failures from the saved predictions. The examples cover Boolean polarity, missing joins, ontology-level predicate confusion, answer type propagation, and a controller output that is unrelated to the question.

4pt

11.6.0.1 Discussion.

The first two examples show that schema-aware grounding does not enforce the question’s logical form. In the Frank Herbert example, the entity and death property are both correct, but the requested negation is lost when the agent switches from an ASK condition to a non-empty SELECT. In the Australia example, the first hop is also correct, but the controller terminates before reaching the requested numeric answer. These errors suggest that a separate answer-type and query-completeness check is needed after schema-guided property selection.

The Anthony Bourdain example exposes a different limitation: domain and range compatibility cannot by themselves distinguish semantically adjacent predicates such as “cause of death” and “manner of death.” Predicate selection needs finer-grained relation descriptions, demonstrations, or execution-based verification. Similarly, the Montenegro example shows that typing the intermediate entity is not enough when the answer type applies to a later variable; type constraints should be propagated along the planned path, not only attached to the first grounded entity.

Finally, the Big Bang Theory example is a controller-level failure rather than a localized grounding error. SAGA’s schema tools cannot recover when the controller emits a query for a different task. A lightweight consistency guard could compare the generated query’s entities, predicates, and answer shape against the question, then trigger a fresh attempt when the mismatch is large. Because the stored result files do not include a causal trace for this failure, the proposed explanation is a mitigation hypothesis rather than a confirmed diagnosis.

11.7 Failure Case: “Give me all communist countries.”↩︎

11.7.0.1 Discussion.

The failure in Figure 7 is representative of SAGA’s primary limitation: it can only filter on types that are explicitly declared via P31 (Wikidata’s instance-of property). Class-level entities (e.g., sovereign state, film) frequently lack P31 annotations because they are the classes, not instances of them. When such entities appear as answer-class constraints in a SPARQL query, SAGA’s type state is empty and filtering is skipped. Potential remedies include using P279 (subclass-of) to derive implicit types, building a complementary class-level schema index, or augmenting type inference with KG embedding-based entity typing.

12 Prompts↩︎

SAGA issues two LLM calls per question: an answer-type inference call (once, before the agent loop) and a controller call (once per agent step). When an entity has no type information (\(\tau(e)=\emptyset\)), a third neighborhood pruning call is used as a fallback (Section 12.3); this prompt is inherited unchanged from SPINACH [3]. All templates use Jinja2 syntax: {{variable}} is a runtime value substituted at each call and {% … %} is control flow expanded before the prompt is sent.

12.1 Answer-Type Prompt↩︎

Called once before the first tool call (temperature \(=0\), max_tokens \(=100\)). Returns a JSON object with two fields: answer_type_label (a short English type name) and answer_type_qid (a Wikidata QID, or an empty string for scalar and Boolean answers). These values seed the persistent type state \(\Theta\) and bias the controller toward ASK queries for Boolean questions (Appendix 10). The Freebase variant is identical except that answer_type_qid holds a Freebase type string (e.g. people.person) instead of a Wikidata QID.

# instruction
Given a natural language question, identify the expected answer type.
Return ONLY a JSON object with two fields:
- "answer_type_label": a concise English label (e.g. "country", "person",
  "date", "number", "organization", "place", "film", "award")
- "answer_type_qid": the most relevant Wikidata QID for this type
  (e.g. "Q6256" for country, "Q5" for human/person, "" if unknown)

If the answer type is unclear or the question asks for a boolean (yes/no),
return {"answer_type_label": "boolean", "answer_type_qid": ""}.
If the answer is a number or quantity, return
{"answer_type_label": "quantity", "answer_type_qid": ""}.

Common mappings:
- person / human -> Q5          - country      -> Q6256
- city           -> Q515        - film / movie -> Q11424
- organization   -> Q43229      - award        -> Q618779
- language       -> Q34770      - university   -> Q3918
- sports team    -> Q12973014

# distillation instruction
Return the answer type JSON.

# input
Question: Who is the president of France?
# output
{"answer_type_label": "person", "answer_type_qid": "Q5"}

# input
Question: In which country was Albert Einstein born?
# output
{"answer_type_label": "country", "answer_type_qid": "Q6256"}

# input
Question: How many films did Stanley Kubrick direct?
# output
{"answer_type_label": "quantity", "answer_type_qid": ""}

# input
Question: {{ question }}

12.2 Controller Prompt↩︎

Called at each agent step (temperature \(=1.0\), top-\(p=0.9\), max_tokens \(=700\), stop token "\nObservation"). Template variables are filled at runtime: {{answer_type}} is the label returned by the answer-type prompt (e.g.person (Q5)); {{type_env}} lists entity-to-type mappings accumulated in \(\Theta\) (at most the five most-recent entries); {{knowledge_log}} summarizes the last six action outcomes; and {{action_history}} is the recent ReAct trajectory (at most the last five steps). Prior conversation turns are prepended in multi-turn settings.

Lines 20–31 (“Schema-Guided Property Selection”) and lines 33–42 (“Current Type Context” and “Knowledge Log”) are the SAGA-specific additions to the SPINACH controller prompt; the surrounding step-by-step instruction text is adapted from SPINACH. The Freebase variant replaces Wikidata QID/PID conventions with Freebase MID and dotted-property conventions; the structure is otherwise identical.

After the controller produces free-form text, a second lightweight LLM call (temperature \(=0\)) re-parses the output into a structured JSON object with fields thought, action_name, and action_argument.

# instruction
Your task is to write a Wikidata SPARQL query to answer the given question.
Follow a step-by-step process:

1. Start by constructing very simple fragments of the SPARQL query.
2. Execute each fragment to verify its correctness. Adjust as needed based on
   observations.
3. Confirm all your assumptions about the structure of Wikidata before
   proceeding.
4. Gradually build the complete SPARQL query by adding one piece at a time.
5. Do NOT repeat the same action, as the results will be the same.
6. The question is guaranteed to have an answer in Wikidata, so continue
   until you find it.
7. If the user is asking a True/False question with only one answer, use
   ASK WHERE to fetch a True/False answer at the very end.
8. In the final SPARQL projections, ask for the actual entities whenever
   needed.
9. If the final result was contained in last round's get_wikidata_entry and
   you are ready to stop, use execute_sparql to retrieve that result.

## Schema-Guided Property Selection (SAGA)
When selecting a property (PID) from search_wikidata results:
- Results now include [domain: X | range: Y] type constraints where available.
- Domain = the required type of the subject (the entity on the left).
- Range  = the required type of the object (the entity on the right).
- Prefer properties whose domain matches the known type of your subject entity
  and whose range matches the expected answer type.
- If domain/range info is shown, use it to disambiguate between
  similarly-named properties.
- When get_wikidata_entry is called, entity types are shown -- use these to
  check domain compatibility before committing to a property.
- If execute_sparql returns [BLOCKED], the property domain is incompatible
  with the subject type. Use a different property matching the subject's type.
- If execute_sparql returns an empty result, check the Knowledge Log and try
  a different property or entity.

## Current Type Context
Expected answer type: {{ answer_type }}

Known entity types (from explored entities):
{{ type_env }}

## Knowledge Log (recent steps)
{{ knowledge_log }}
([ok] = found results; [fail] Empty/BLOCKED = failed;
 [info] = entity type discovered; avoid repeating failed approaches)

Form exactly one "Thought" and perform exactly one "Action", then wait for
the "Observation".

Possible actions are:

- get_wikidata_entry(QID): Retrieves all outgoing edges (linked entities,
  properties, and qualifiers) of a Wikidata entity using its QID. Also
  reports the entity's types.
- search_wikidata(string): Searches Wikidata for entities or properties
  matching the given string. Property results include type constraints
  [domain: ... | range: ...] when available.
- get_property_examples(PID): Provides a few examples demonstrating the use
  of the specified property (PID) in Wikidata.
- execute_sparql(SPARQL): Runs a SPARQL query on Wikidata and returns a
  truncated result set. If the result is empty with a [WARNING], check
  property type compatibility.
- stop(): Marks the last executed SPARQL query as the final answer and ends.

# distillation instruction
Think and perform the next action.

# input
[Prior conversation turns prepended here in multi-turn settings;
 each turn renders as "User Question: ...\n<action history>\n--"]

User Question: {{ question }}
{{ action_history }}

Output one "Thought" and one "Action":

12.3 Fallback Neighborhood Pruning Prompt↩︎

When no type information is available for an entity (\(\tau(e)=\emptyset\)), SAGA falls back to the LLM-based pruning prompt inherited from SPINACH [3]. The prompt is given the raw Wikidata JSON neighborhood and the current question and asks the LLM to select the relevant subset. The two-shot version is shown below (few-shot examples abbreviated for space; the full examples are in the released code).

# instruction
At each turn, you are given a Wikidata entry and a question.
You want to write a SPARQL query that answers the question.
As the first step, remove the parts of the Wikidata entry that could not
be potentially helpful when writing the SPARQL.
The output should be a JSON object containing part of the Wikidata entry.

# distillation instruction
Prune this Wikidata entry.

# input
Wikidata entry for OneRepublic (Q1438730, 'OneRepublic' is an American pop
rock band formed in Colorado Springs, Colorado, in 2002):
{
  "instance of (P31)": "musical group (Q215380)",
  "genre (P136)": ["pop rock (Q484641)", "alternative rock (Q11366)", ...],
  "record label (P264)": "Columbia Records (Q183387)",
  "discography (P358)": "OneRepublic discography (Q935670)",
  ...
}
Question: "What is the title of the second single on OneRepublic's
           third album Native?"
# output
{
  "instance of (P31)": "musical group (Q215380)",
  "record label (P264)": "Columbia Records (Q183387)",
  "discography (P358)": "OneRepublic discography (Q935670)"
}

# input
Wikidata entry for Barack Obama Sr. (Q649593, economist and father of
Barack Obama Jr.):
{
  "educated at (P69)": ["Maseno School (Q6782972)",
                         "University of Hawaii (Q217439)",
                         "Harvard University (Q13371)"],
  "occupation (P106)": "economist (Q188094)",
  ...
}
Question: "where did barack obama sr. attend school?"
# output
{
  "educated at (P69)": ["Maseno School (Q6782972)",
                         "University of Hawaii (Q217439)",
                         "Harvard University (Q13371)"]
}

# input
Wikidata entry for "{{ entity_and_description }}":
{{ outgoing_edges }}

[Context from prior turns if in multi-turn setting]

Current Question: "{{ question }}"

References↩︎