1 Introduction↩︎

Regulated organizations operate under strict, layered frameworks of statutes, enforcement decrees, ministerial rules, notices, and operational manuals. Since non-compliance carries severe risks, compliance staff routinely navigate complex procedural questions whose grounds span these multiple authority tiers [1]. While Large Language Models (LLMs) offer a promising avenue to alleviate this burden, deploying them to assist with such inquiries requires rigorous traceability [2], [3]. To establish a verifiable audit trail, models should quote controlling articles and explicitly map every generated claim back to its source.

In this paper, we formalize this objective as Regulatory Compliance Question Answering (QA), where a model addresses a practitioner’s query by retrieving the exhaustive set of governing articles and detailing the precise claims derived from each rule. As modern regulatory systems evolve into highly complex cross-reference networks across multiple documents [4], [5], this task structurally diverges from both standard multi-hop and legal QA paradigms. Unlike standard multi-hop QA paradigms focusing on entity resolution [6], [7], hop transitions in this QA task follow typed citation rules, and search termination relies on complete evidence-set closure rather than finding a single final entity. Furthermore, while traditional legal QA tasks [8]–[10] largely center on judicial interpretation and case-law reasoning, recent regulatory NLP has expanded into procedural clause retrieval [11] and corporate sustainability extraction [12]. However, these existing benchmarks predominantly operate on flat rule structures or isolated reports, failing to capture the mechanics of hierarchical compliance. In contrast, Regulatory Compliance QA demands navigating multi-tiered, cross-document delegations to achieve deterministic evidence-set closure.

These rigorous demands expose two critical failure modes in existing RAG systems. The first is the inability to execute precise structural retrieval. Current graph-based RAG approaches [13]–[17] typically flatten explicit regulatory citations into generic entity-relation edges, stripping away the semantic distinctions necessary for procedural navigation. Because they rely on global signal propagation across surface-level entity overlaps, standard entity-centric graphs inevitably fail to resolve the rigid, multi-tiered delegation pathways inherent in regulatory networks. Moreover, because regulatory queries often omit specific situational constraints, systems must exhaustively identify all potential conditional branches and legal exceptions. Attempts to resolve this via query expansion or decomposition [18]–[21] typically rely on surface-level paraphrasing, produce disjoint sub-queries that fragment rather than unify the required evidence set.

The second major limitation emerges during answer generation, where attribution is typically treated as a post-hoc afterthought [22]. In regulatory compliance, the risk of a hallucinated citation outweighs the benefit of a broad response. However, rather than enforcing a schema-level binding between individual claims and their sources, generative models generally append citations as free-form footers, leading to systemic attribution failures [3], [23], [24]. Ultimately, this lack of structural binding between claims and their sources fails to provide the rigorous traceability required, leaving a significant gap in the reliability of regulatory AI.

To systematically diagnose these limitations, we introduce RegOps-Bench, an evaluation framework serving as a testbed for multi-tier procedural navigation, instantiated via a highly structured Korean national R&D regulatory corpus. This corpus features a five-tiered authority structure that exemplifies the nested complexities inherent in real-world administrative regulations. We model this corpus into an Operational Knowledge Graph (OKG) and construct 250 high-quality QA pairs grounded in real-world inquiries from the Institute of Information & Communications Technology Planning & Evaluation (IITP). By expanding these inquiries using a novel axis-decoupling principle, we independently control the substantive intent of a query and the structural complexity of its required reference set, spanning from straightforward lookups to exception-heavy procedural branching. This orthogonal design precisely isolates whether a system fails due to navigating a dense regulatory hierarchy or resolving complex procedural logic.

To overcome the identified retrieval and generation bottlenecks, we propose RefWalk, a structural traversal framework that navigates regulatory citation pathways guided by a shared topic anchor. RefWalk mitigates structural retrieval failures by exploring the OKG through three distinct semantic views, restricting hop expansion strictly to cross-document citation edges to eliminate internal containment noise. To preserve the specialized signals required across different difficulty tiers, candidates are fused using Reciprocal Rank MAX (RRM) rather than standard sum-based aggregation [25], which otherwise dilutes crucial specialist cues. During generation, RefWalk tackles systemic attribution failures by injecting the same anchor alongside a per-rule schema. Instead of appending citations as a post-hoc afterthought, we structure the model to generate claims directly as attributes of their governing rules. This approach inherently mitigates attribution hallucinations and binds generation to its source, advancing the traceability required for professional practice. Finally, we demonstrate the broader applicability of this evaluation framework and RefWalk by validating both on a HIPAA-derived dataset.

In summary, our main contributions are threefold. First, we formalize the task of Regulatory Compliance QA and release RegOps-Bench, the first benchmark for deterministic traversal of multi-tier regulatory hierarchies. Second, we propose RefWalk, a unified RAG framework that navigates explicit structural delegations, establishing a robust baseline for complex procedural lookups. Third, we outperform state-of-the-art RAG methods on complex cross-reference tasks. Furthermore, ablation studies confirm the impact of our schema-level binding, while contrastive experiments on HIPAA validate the necessity of multi-tiered evaluation.

2 Related Work↩︎

2.1 Regulatory NLP and Structural Retrieval↩︎

Legal NLP has increasingly shifted from coarse classification [8], [26] toward reasoning- and retrieval-intensive evaluation [10], [24]. Within this space, regulatory compliance presents unique challenges distinct from case-law analogy, requiring the navigation of complex, multi-document networks [4], [5], [27] to meet the demands of RegTech [1]. While De Jure [28] focuses on structuring raw regulations into rule sets, we leverage its evaluation pipeline for our HIPAA data generation. However, existing benchmarks still fail to formalize QA across layered, cross-document procedural chains.

Standard multi-hop QA [6], [7], [29] and graph-based RAG [13]–[16], [30] typically attempt complex retrieval by propagating through entity-centric relations. Similarly, query decomposition [18]–[21], [31]–[33] divides queries into independent sub-facts.

2.2 Attributed Generation and Citation Faithfulness↩︎

Verifiable grounding is widely recognized as a strict deployment prerequisite for legal AI [2], [24]. Consequently, citation faithfulness has emerged as a central evaluation axis, driving the development of specialized attribution benchmarks [3], [23], [34], [35] and methods that fold retrieval or revision decisions into the generation policy [36], [37]. Despite these advances, audits of off-the-shelf LLMs and domain-specific legal systems [3], [24] consistently reveal that a substantial fraction of claims remain uncited or misattributed. This occurs because existing systems treat attribution as post-hoc annotation, thereby lacking structural guarantees.

3 RegOps-Bench: Axis-Decoupled Construction for Compliance QA↩︎

RegOps-Bench is designed around three core properties of this domain: (1) ground truth is a typed citation closure, (2) difficulty is defined by the structural complexity of the reference set rather than lexical phrasing, and (3) evaluation operates at the regulatory unit where authority delegates.

3.1 Operational Knowledge Graph (OKG)↩︎

https://www.iitp.kr/web/lay1/bbs/S1T46C59/A/13/view.do?article_seq=4331&sort=latest&cpage=1&rows=10

3.1.0.1 Corpus.

To anchor the benchmark in real-world compliance scenarios, we curated a corpus of 12 Korean R&D regulatory documents covering the scope of 56 seed FAQs from an official FAQ document¹ of the Institute of Information & Communications Technology Planning & Evaluation (IITP). These documents span five authority tiers (1: statute – 5: manual), from statutory acts down to operational manuals (Table [tab:corpus]). The seven in-domain documents form a largely self-contained delegation network. While certain high-level mandates inevitably delegate to external authorities, cross-document references, including all FAQ lineages, are mostly resolved within this set, where each lower tier procedurizes the open-textured mandates of the tier above it. A single compliance answer therefore requires composing a statutory obligation with its decree-level conditions and notice-level numeric thresholds. Two auxiliary VAT documents cover the tax-side cross-references recurring in cost-eligibility questions. The remaining three Science & Technology Basic Act documents serve as distractors, sharing lexical overlap with the in-domain corpus but remaining legally out of scope.

3.1.0.2 OKG Construction.

We construct the OKG via deterministic rule extraction. By leveraging the formulaic citation patterns of legislation, this design effectively mitigates the generative hallucinations inherent in LLM-based indexing and provides deterministic edge typing—a prerequisite for evaluating closure. Furthermore, this approach eliminates heavy LLM computational overhead, making the index highly sustainable and easily updatable under frequent regulatory amendments. Each article (조) forms a node carrying its authority tier. Inter-article relations are typed into six classes: PART_OF (hierarchy), REFERENCES (citation), the DELEGATES_TO/SPECIFIES pair (downward delegation/upward realization), DEFINES (term-to-article), and REQUIRES_FORM (article-to-form). For instance, “as prescribed by the Presidential Decree” yields a DELEGATES_TO edge. Retrieval and citation are strictly evaluated at the article level (조), the atomic unit of authority; finer paragraph (항) structure is retained within nodes for condition evaluation, but is not a scoring target.

3.2 QA Construction with Axis Decoupling↩︎

3.2.0.1 Axis-Decoupled Augmentation via LLM.

To ensure the benchmark comprehensively evaluates the structural complexity defined in our design principles, we expand the 56 seed FAQs into a final set of 250 expert-grounded questions using a high-capacity LLM (Gemini-3-flash [38]). Our augmentation follows an axis-decoupling principle with two distinct axes:

• Question Type — The substantive intent of the query (e.g., single-clause lookup, exception-heavy, or sanction-bearing condition).

• Difficulty Level — The structural complexity of the required reference set (L1 to L4), strictly bounded by the quantitative triggers in Table 1.

The conditions are sampled and combined independently, so that a single question type can realize any of L1–L4 depending only on the anchor and injected facets, minimizing the confounding effect of surface-level phrasing.

For each seed, the LLM is given the governing articles and instructed to re-synthesize a situational, first-person practitioner inquiry matching the sampled axes, injecting facets—actors, temporal constraints, and institutional variables—that necessitate the target difficulty. For instance, to elevate a straightforward seed to L4, the model constructs a scenario in which an actor’s specific condition triggers a cross-document citation to an external disqualification provision. Difficulty thus remains an intrinsic property of the regulatory logic (the reference set) rather than an artifact of linguistic complexity. To verify that our augmentation induces the targeted structural depth, we compare the corpus characteristics of the LLM-generated splits against the original human FAQs in Appendix 8.5.

3.2.0.2 Reference Rules and Closure.

To reduce human annotator variance and ensure reproducibility, we standardize the ground-truth reference mapping via four deterministic expert rules, activated by formal structural markers identified during synthesis:

• R1 (Domain Anchor): binds the query to the top-priority controlling document of its domain—the Cost-Use Standards by default, or the domain-specific instrument otherwise (the Mgmt.Regulation for institutional-IT queries, the Standard Guide for facility/equipment, the VAT Act for tax-side, the Innovation Act for statutory-procedure).

• R2 (Parallel & Deemed-Application Expansion): expands the set across the four institutional slots (government-funded, university, non-profit, for-profit) via parallel groups, and follows deemed-application (의제 준용) forward edges so that a deemed article additionally pulls in the provisions it incorporates by reference.

• R3 (Pre-Approval Exception): binds exception-heavy inquiries to their governing pre-approval clause (Cost-Use Standards Art. 73).

• R4 (Sanction Exhaustion): closes sanction-bearing questions onto their corresponding settlement and disqualification provisions (Enforcement Decree Art. 26 and Cost-Use Standards Art. 83).

The external-law case is handled by absorbing it into the in-corpus VAT domain rather than as a separate rule, since the VAT Act and its Rules are already part of the corpus. This rule-based mapping turns each QA pair from an ambiguous retrieval task into a verifiable, deterministic traversal of the regulatory hierarchy.

3.2.0.3 Quality Assurance Protocol.

Each generated QA pair passes a layered validation cascade before merging. A regex runner gate first rejects forbidden formats, such as multi-institution enumerations and textbook-style summary or comparison requests. Survivors are scored by an LLM judge against a four-criterion rubric evaluating practitioner voice, concrete context, situational framing, and a single-institution viewpoint, yielding around a 50% acceptance rate. This is supplemented by heuristic checks (script leakage, meta-patterns, degenerate length) and a difficulty re-validation step that discards items whose realized reference set diverges from the target level. A closure validator then verifies and auto-completes any missing R3 or R4-triggered authorities. We also conduct a manual curation pass over the accepted pool to filter residual low-quality items. Finally, ground-truth reference sets undergo a deterministic post-merge audit that corrects a clause-attribution artifact (cross-article paragraph-token leakage) present in 38% of pre-audit items. The ground-truth reference sets are detailed in the Appendix 8.4.

4 RefWalk: Traversing Reference Paths for Structural Attribution↩︎

The rigorous properties of regulatory compliance QA—typed citation closures and article-level structural traversal—demand a framework where retrieval and generation operate as a traceable pipeline. RefWalk organizes this framework around the topic anchor, a structured abstraction distilling the query into its core procedural intent and conditional facets. Rather than treating query processing, graph expansion, and answer generation as isolated, fragmented stages, this single anchor propagates through the entire framework (Figure 1). It simultaneously drives multi-view query construction, seeds OKG traversal, and re-enters the generation prompt to structurally constrain attribution.

4.1 Topic-Anchored Multi-View Retrieval↩︎

4.1.0.1 Anchor Extraction and Views.

For a given query \(q\), a frozen LLM \(\tau(q)\) extracts the topic anchor: a structured tuple of the core topic and facet conditions (actor, temporal, magnitude, and situational context). Because regulatory QA requires matching both explicit granular constraints and implicit procedural exceptions, we construct three targeted views from this shared anchor. \(q_{\text{narrow}}\) directly uses the original question for dense semantic matching of explicit details. Conversely, \(q_{\text{wide}}\) drops the raw question, relying solely on the abstracted topic and conditions to capture structural multi-hop references where surface phrasing is uninformative. \(q_{\text{mid}}\) combines both for balanced retrieval.

4.1.0.2 OKG Expansion and RRM Fusion.

Retrieval begins by fetching an initial candidate pool \(\mathcal{S}\) of size \(N\) using \(q_{\text{mid}}\) via dense retrieval. We expand this pool one hop along citation-bearing edges (REFERENCES, DELEGATES_TO, SPECIFIES) in the OKG, isolating explicit delegations. To penalize indirect evidence, 1-hop neighbors receive a decay factor \(\delta\). For nodes discovered via multiple pathways, we take the maximum decayed score to preserve the strongest signal without inflating it via sum-based aggregation. The expanded pool is then scored by a cross-encoder across our three semantic views and fused via Reciprocal Rank MAX (RRM): \[\text{score}(d) \;=\; \max_{v \in \{narrow,\, mid,\, wide\}} \frac{1}{k + \text{rank}_v(d)},\] where \(k\) denotes a smoothing constant. Unlike sum-based aggregations (e.g., RRF) that demand consensus, RRM ensures that candidates highly ranked by a single specialist view retain their priority. Finally, we introduce an authority-aware decay \(\mu\) for candidates sourced from lower-tier operational manuals. Applied post-fusion, \(\mu\) injects a domain-specific inductive bias into the ranking: it intrinsically prioritizes primary statutes, yet permits strongly matched manual passages to surface when they contain critical procedural details. This allows RefWalk to maintain citation closure while navigating multi-tiered delegations, circumventing the fragmentation of standard query decomposition.

4.2 Per-Rule Attribution Schema↩︎

In regulatory compliance, the risk of a hallucinated citation outweighs the benefit of a broad response. Generative models typically treat attribution as a post-hoc afterthought, appending free-form footers that frequently mismatch the generated claims. RefWalk structurally mitigates this vulnerability through a per-rule attribution schema. The extracted topic anchor re-enters the framework by being injected into the generation prompt alongside the retrieved OKG passages and a strict JSON schema that maps specific rule_id keys to arrays of generated claims. Rather than generating free-form text and retroactively appending citations, the model must emit procedural claims exclusively as array values bound to their governing rules. Under this strict schema, every generated claim is inherently bound to its source. This ensures that the structural precision achieved during OKG retrieval directly translates into the rigorous traceability required for regulatory compliance.

5 Experiments↩︎

5.1 Setup↩︎

5.1.0.1 Dataset & Models.

To demonstrate the effect of our framework, we conduct our primary evaluations on RegOps-Bench. We also evaluate on a HIPAA-derived QA dataset (\(n=100\)) constructed following the De Jure procedure [28]. For retrieval, we use Qwen3-Embedding-0.6B and Qwen3-Reranker-0.6B [39] to ensure our gains stem from our structural design rather than brute-force embedding scale. For generations, we employ Qwen3.5-4B [40] and Qwen3.6-35B [41] to observe scale-dependent behaviors, alongside Gemini-3.1-Pro [42] for frontier model validation.

5.1.0.2 Metrics.

For retrieval, we report Recall@10, nDCG@10, and FullCov@10. We introduce FullCov@10 (Full Coverage) as the fraction of queries where the entire ground-truth reference set is successfully retrieved within the top 10 candidates, effectively capturing the exhaustiveness required for compliance QA. End-to-end generation is evaluated using Claim F1 and Citation F1. Claim F1 applies an LLM judge [43] to label each predicted claim against the reference set as {match, partial, none}, with partials weighted 0.5 before bipartite resolution [44]. Citation F1 compares predicted and ground-truth references after rolling sub-article ids (항/호) up to their 조-level ancestor, so credit is given for citing the correct provision regardless of granularity [35]. In regulatory compliance, Citation Precision is prioritized. While a missed citation merely results in an incomplete answer, hallucinating a legal citation poses severe operational risks. Thus, our analysis focuses on a model’s ability to mitigate attribution hallucinations.

5.1.0.3 Baselines.

We compare RefWalk against strong baselines across two stages. For retrieval, we evaluate standard architectures (BM25, Dense, Dense+Rerank) alongside state-of-the-art graph and query-based retrievers, including LightRAG [14], HippoRAG-2 [45], Query Decomposition [21], and PIKE-RAG [30]. For end-to-end RAG, we evaluate NativeRAG with a Dense+Rerank pipeline as well as the four aforementioned systems equipped with generation capabilities. To ensure a fair comparison, all baselines share the exact same embedding, reranking, and generation backbones as RefWalk.

5.1.0.4 Hyperparameters.

For RefWalk, we set the retrieval pool size to \(N=50\), the OKG seed count to \(M=10\), and the RRM fusion constant to \(k=60\). Further details are represented in Appendix 8.1.

5.2 Retrieval Results↩︎

Table 2 presents the retrieval performance. RefWalk establishes a state-of-the-art on RegOps-Bench, achieving the most substantial gain in FullCov@10. While these absolute metrics leave an unsolved gap for future work, this steep difficulty highlights the necessity of RegOps-Bench as a non-saturated stress test rather than a limitation of our method. Ablation studies confirm that removing multi-view reranking, OKG expansion, or anchor enrichment each degrades performance, demonstrating that these components jointly address the structural failures of dense retrieval. To address potential concerns regarding extraction imperfections during OKG construction, we further validate the robustness of our approach against graph-level noise in Appendix 8.8. On the HIPAA dataset, retrieval metrics generally saturate across most baselines. This suggests that while baseline retrieval is highly effective on flat-structure benchmarks, simple semantic matching is insufficient for the complex cross-reference environments.

5.3 End-to-End RAG Results↩︎

Table [tab:rag] compares the overall end-to-end performance on RegOps-Bench. While Claim F1 remains relatively consistent across systems—since powerful generators can infer similar claims once partial evidence is retrieved—RefWalk achieves substantial gains in Citation F1 over Native RAG and graph-based baselines. This improvement is structurally enforced by our per-rule attribution schema. We validate this through the w/o schema ablation, which effectively isolates the generation bottleneck by pairing RefWalk’s advanced retrieval with a Native RAG-style free-form prompt. When the schema constraint is removed, Citation F1 drops sharply. This result shows that without schema-level structural binding, generative models treat citations as post-hoc footers, failing to accurately map claims to their corresponding rules even when the correct evidence is retrieved.

5.4 Difficulty-Stratified Analysis↩︎

5pt

5pt

To understand exactly where existing systems fail, we stratify performance by difficulty (L1–L4). As shown in Table 4, all methods succeed at single-anchor lookups (L1). However, at L3 (multi-hop/cross-doc) and L4 (conditional multi-hop), baseline retrieval significantly collapses. Lexical or simple dense retrieval cannot navigate explicit, multi-tiered delegations. This bottleneck directly cascades into generation, as demonstrated in Table 5. While baseline models attempt to answer L3/L4 queries, their Citation F1 plummets because they fail to bind claims to the correct cross-document sources. In contrast, RefWalk demonstrates significant relative improvements in Citation F1 across these complex tiers, effectively mitigating the severe drop-offs caused by fragmented retrieval.

5.5 Mechanism Analysis: RRM vs RRF↩︎

As shown in Table 6, RefWalk’s resilience at L3/L4 is driven by the RRM fusion. For complex cross-document references, the \(q_{wide}\) view carries the structural signal. Standard sum-based aggregations (e.g., uniform or weighted RRF) dilute this specialist signal by demanding consensus across all views, which leads to a degradation in L3 performance. RRM preserves the single strongest rank, allowing the framework to maintain citation closure without compromising the retrieved evidence set.

5.6 Mitigating Attribution Hallucination via Schema Constraints↩︎

To validate whether per-rule schema alleviates attribution hallucination (Table [tab:generation95reliability]), we conduct a ceiling analysis using an Oracle with Controlled Noise setting (\(k=10\)). While Native RAG’s scores are artificially inflated under a Pure Oracle setting (without distractors), padding the context with hard negatives from the 1-hop citation neighborhood exposes a systemic eager-citing bias, causing a sharp decline in Citation Precision. Conversely, RefWalk’s conservative mapping strategy filters out these hard negatives, securing a highly competitive Citation Precision. Applying our schema constraint to PIKE-RAG also boosts its native Claim Precision and F1. This highlights the transferability of our schema-binding strategy in steering LLMs toward more precise operational claims. However, even with this structural enforcement, PIKE-RAG’s Citation F1 remains substantially lower than RefWalk’s. Furthermore, evaluating Gemini-3.1-Pro reveals that even frontier models with advanced reasoning capabilities are highly susceptible to attribution hallucination under Native RAG.

6 Conclusion↩︎

Assisting regulatory compliance with LLMs requires a paradigm shift from broad answer generation to verifiable, structure-bound attribution. In this work, we formalized this challenge through RegOps-Bench, a benchmark designed to capture the intricate, multi-tiered delegation networks inherent in real-world regulatory ecosystems. By decoupling procedural complexity from surface-level phrasing, our evaluations revealed that current retrieval-augmented systems—despite their proficiency on flat-structure tasks—struggle significantly to navigate complex legal citations. To overcome this, we introduced RefWalk, a framework that unifies Operational Knowledge Graph traversal with per-rule attribution. Our findings demonstrate that preserving specialist signals via multi-view RRM fusion effectively resolves cross-document chains, while structural schema-binding curtails the persistent attribution hallucinations prevalent even in frontier LLMs. Ultimately, RefWalk successfully mitigates the vulnerabilities of free-form generation, providing a robust and generalizable foundation for deploying fully traceable AI in high-stakes regulatory domains.

7 Limitations↩︎

While RefWalk establishes a robust framework for verifiable compliance QA, its design philosophy fundamentally prioritizes safety over comprehensiveness, introducing an inherent precision-recall trade-off in attribution. In high-stakes domains like regulatory compliance, hallucinating a false citation is far more dangerous than missing a valid one. Consequently, our per-rule attribution schema enforces a highly conservative mapping strategy. As demonstrated in our oracle experiments, this strictness successfully pushes citation precision to highly reliable levels under noisy conditions, but it inevitably sacrifices recall compared to the eager-citing behavior of Native RAG. Future work should explore adaptive schema constraints that dynamically balance recall without compromising strict hallucination boundaries.

Beyond generation constraints, the framework’s retrieval architecture relies on the deterministic extraction of an Operational Knowledge Graph (OKG). By leveraging the highly formulaic citation patterns of Korean regulatory documents, we achieved high-precision, low-cost rule extraction without the heavy LLM-based indexing overhead seen in other graph-based RAGs. However, scaling this purely deterministic graph-building process to less structured jurisdictions, such as heavily case-law-driven domains or entirely different languages, remains a challenge. Adapting RefWalk to such environments will likely require transitioning to hybrid (rule and LLM-assisted) extraction pipelines.

Finally, navigating the OKG accurately requires preserving expert signals during complex multi-hop retrieval, which RefWalk achieves via multi-view cross-encoding and RRM fusion. While highly effective for targeted candidate pools (e.g., \(N=50\)), applying deep cross-attention independently across three distinct semantic views scales linearly with the pool size, which may eventually encounter computational bottlenecks in massive-scale deployments. Addressing these latency challenges—such as optimizing prompt topology for cross-view KV cache sharing, or introducing dynamic view routing to conditionally bypass redundant cross-encoder passes—remains an essential direction for scaling RefWalk without sacrificing its rigorous matching precision.

8 Appendix↩︎

8.1 Implementation Details↩︎

All open-source models were served using vLLM [46] on RTX A6000 (48 GB) GPUs in bfloat16 precision with a maximum sequence length of 32,768 tokens. We employed Qwen3.6-35B-A3B-FP8 and Qwen3.5-4B as the 35B and 4B backbones, respectively, for the cross-scale comparison. Dense retrieval was conducted using Qwen3-Embedding-0.6B, while the cross-encoder utilized Qwen3-Reranker-0.6B with the official yes/no template. For all text generation, the temperature, top-\(p\), and top-\(k\) parameters were set to 0.1, 0.95, and 20, respectively. We disabled the thinking mode and fixed the random seed to 42 across all runs. For the retrieval stage of RefWalk, we utilized a 50-candidate rerank pool over three views fused via Reciprocal Rank MAX (k=60). This was further augmented by a 1-hop OKG expansion over REFERENCES, DELEGATES_TO, and SPECIFIES edge types, applying a decay factor of \(\delta=0.7\) and up to \(M=10\) seed nodes. Here, \(\delta\) down-weights 1-hop neighbors as indirect evidence; co-discovered neighbors take the max decayed score, not the sum. In addition, an authority-aware manual-node decay of \(\mu=0.7\) is multiplied into the fused rerank score of any candidate sourced from a manual (매뉴얼) rather than a statute (법령/시행령/시행규칙). Because it is applied after multi-view fusion, it acts as a soft tie-break that favors higher-authority sources while still allowing a strongly-matched manual passage to outrank a weakly-matched statute. All hyperparameters were kept fixed across both the RegOps and HIPAA datasets. End-to-end generation is evaluated using Claim F1 and Citation F1. Claim F1 is computed via LLM-as-judge [43] bipartite matching between predicted and reference atomic claims, where exact semantic matches receive full credit and partial matches—those preserving the core proposition but missing a condition, exception, or numeric scope—receive half credit. Citation F1 measures the overlap between cited and ground-truth references at the article (조) level, rolling up sub-article citations (항/호) to their parent article so that an answer is credited whenever it points to the correct legal provision regardless of granularity.

8.2 Knowledge Source and OKG Properties↩︎

8.2.0.1 Statistics.

The underlying corpus of RegOps-Bench comprises 718 procedural articles containing roughly 478K subword tokens. As detailed in Table 7, the article length distribution is heavily right-skewed. However, because only a small fraction of the articles exceed the standard 8,192-token embedding context window, article-level indexing remains lossless for the overwhelming majority of the text. Furthermore, the corpus includes a non-trivial amount of structured data, with approximately 10% of articles embedding tabular content and 3% acting as appendix forms (별표).

8.2.0.2 OKG Composition.

The Operational Knowledge Graph (OKG) instantiated from this textual corpus consists of 2,572 nodes (primarily operational article/provision units) and 3,942 typed edges. As Table 8 illustrates, the graph’s edge distribution is dominated by the structural containment hierarchy (Part_Of) and citation-bearing relations. Notably, the upward realization (Specifies) and downward delegation (Delegates_To) edges are perfectly balanced. This symmetry reflects the paired multi-tier legal structure of the corpus, forming the exact structural signature that retrieval systems must traverse to resolve complex regulatory queries.

8.2.0.3 Reference Topology of the OKG.

Beyond local edge distributions, we analyze the global connectivity of the OKG to substantiate that the in-domain documents form a largely self-contained delegation network. Considering only the citation-bearing relations (excluding the within-document containment hierarchy), the graph fragments into 1,878 components but is dominated by a single referential backbone of 654 nodes spanning all twelve documents; the remaining components have a median size of one, indicating that most articles delegate into the shared backbone rather than isolated islands. When the containment hierarchy is added, the graph consolidates into 148 components. This includes a giant component of 2,078 nodes that again spans all twelve documents, leaving only 81 truly orphaned nodes. Of the 148 components, only nine bridge two or more documents, mostly representing auxiliary tax pairings. Because the giant component co-mingles the containment hierarchy with citation edges, we further apply greedy-modularity community detection to it, recovering 48 distinct communities. The largest communities align tightly with intuitive procedural themes: a statutory-procedure cluster anchored on the Innovation Act (314 nodes across 9 documents), a cost-use cluster anchored on the Cost-Use Standards (300 nodes, 6 documents), and a self-contained tax cluster anchored on the VAT Act (264 nodes, 1 document). This empirical decomposition confirms that the benchmark’s multi-hop reference closures are not artifacts of a single dense hub but rather span semantically coherent, multi-document procedural neighborhoods.

8.3 Benchmark Details↩︎

RegOps-Bench comprises 250 question–answer pairs, combining 194 instances generated via axis-decoupled augmentation with 56 verbatim queries retained from the IITP practitioner board. As outlined in Table 9, the benchmark spans diverse substantive intents—ranging from single-clause lookups to multi-facet institutional inquiries. The Cost-Use Standards serves as the primary domain anchor for nearly half of the generated questions, supplemented by procedural chains from the Innovation Act, institutional IT regulations, and auxiliary tax documents.

We balance the dataset toward higher complexity, with the advanced L3 and L4 tiers jointly accounting for nearly half of the benchmark. This distribution ensures the evaluation stresses multi-hop and conditional reasoning rather than simple fact retrieval. Table 10 validates this design by tracing how structural complexity materializes across the difficulty tiers. The profile exhibits a clear, monotonic progression: as difficulty increases from L1 to L4, conditional reasoning transitions from entirely absent to ubiquitous, and cross-document references become highly prevalent.

Correspondingly, the mean size of the required reference set and the citation-graph depth grow in lock-step with the difficulty levels. It is important to note that difficulty in RegOps-Bench is an intrinsic property of the reference closure rather than mere surface phrasing. The overall reference-count distribution is long-tailed: while roughly 45% of the questions resolve at a single hop, over 16% require navigating four or more hops along the citation graph, rigorously testing a system’s capacity for deep reference traversal.

8.4 Ground-Truth Reference-Set Analysis↩︎

Across the 250 questions, the ground-truth reference sets contain 933 citations with multiplicity (314 unique articles), averaging 3.73 references drawn from 1.36 documents per question. By annotation granularity, 71.6% of citations are article-level (조), 27.0% are sub-clause (항/호), and 1.4% are appendix forms (별표), scoring collapses sub-clause citations to their governing article, the atomic unit of authority. Reference sets are well-grounded in the indexed graph: 97.1% of citations resolve to an OKG node, with only 2.9% pointing outside the constructed graph. The reference distribution is sharply concentrated on the operational core of the corpus (Table 11): the single most-cited article is the Cost-Use Standards pre-approval clause (Art. 73, 70 citations), followed by the international-joint-R&D and fund-management standards, and by the two sanction-side authorities (Enforcement Decree Art. 26 and Cost-Use Standards Art. 83). These dominant anchors correspond directly to the deterministic expert rules R3 (pre-approval) and R4 (sanction exhaustion).

8.5 Validating the Axis-Decoupling Principle↩︎

The augmentation is governed by the axis-decoupling principle: substantive question type and structural difficulty are sampled independently, so a single question type can realize any difficulty level depending only on the anchor and injected facets. Table 12 reports the empirical type\(\times\)difficulty cross-tabulation, which confirms the intended spread: single-clause questions concentrate at L1, exception-paired questions at L2, while the multi-facet and general-principle types populate the L3/L4 tiers where deep reference closures are required. As a quantitative control, 69.4% of augmented items realize exactly the targeted difficulty level after generation; the difficulty re-validation step discards the divergent remainder so that the released set is internally consistent. The facet injection that drives this spread draws on six facet dimensions (transaction, actor-role, recipient-role, geography, lifecycle, and domain) and 14 four-institution parallel-group topics, providing the combinatorial breadth needed to decouple intent from structural depth.

4pt

In Table 13, we report query distribution, corpus characteristics, and end-to-end RefWalk performance per difficulty. Synthetic queries are harder and deeper than the human board, yet RefWalk performance aligns closely with the hard tier (L3/L4) and on citation grounding overall, indicating the augmentation is faithful and non-trivial.

8.6 Computational Cost↩︎

In Table 14, metrics were evaluated on the RegOps dataset (\(n = 250\)) under identical hardware configurations using a 35B backbone. At retrieval time, RefWalk incurs an overhead of +4.18 s compared to the fastest baseline (NativeRAG). This represents the marginal cost of executing the cross-encoder across three distinct views (\(q_{\text{narrow}}\), \(q_{\text{mid}}\), \(q_{\text{wide}}\)) alongside a 1-hop OKG expansion. Notably, RefWalk is \(\sim\)​1.5\(\times\) faster than PIKE-RAG in RAG latency (9.49 s vs. 14.64 s). On the indexing side, RefWalk avoids generative LLM calls during index construction, whereas other graph-based methods incur substantial pre-computation overhead.

8.7 Sensitivity Analysis of the RRM Smoothing Constant \(k\)↩︎

5pt

As detailed in Table 15, the empirical variance of the overall Recall@10 metric fluctuates within a marginal range of just 0.40 percentage points across a multi-order sweep of \(k \in [10, 200]\). This stable behavior aligns with the structural mechanics of Reciprocal Rank Max (RRM). Consequently, the smoothing constant \(k\) solely scales the numerical resolution of tie-breaks between identical rank positions within the same view, rather than altering the global relative weights across different views.

8.8 Robustness to Operational Knowledge Graph Construction Noise↩︎

5pt

As detailed in Table 16, RefWalk exhibits high resilience to OKG construction noise, with overall Recall@10 degrading by at most 0.57 percentage points even under an aggressive \(20\%\) perturbation rate. The near-identical performance drops observed between edge dropping and rewiring (\(\le\)​0.02 percentage points variance) demonstrate that the cross-encoder reranker effectively filters out injected false-positive connections, leaving missed true-positives as the sole operative failure mode. Furthermore, the multi-hop segment (L3) degrades by only 0.31 percentage points under \(20\%\) noise. This robust behavior underscores the structural role of the OKG as a candidate pool augmentation channel rather than a direct ranking signal; a corrupted edge only induces a recall failure if the corresponding gold document is simultaneously absent from the dense retrieval seed.

8.9 Prompt Template for RefWalk↩︎

8.9.0.1 Cross-Encoder Reranking.

For the reranking stage, each query-document pair \((q, d)\) is evaluated using a cross-encoder model to compute a deterministic relevance score. The model is optimized via a binary classification objective, prompting it to generate a strict yes or no tokens indicating whether the document satisfies the query. This prompt structure is uniformly applied across all three retrieval views within the RRM-fusion pipeline and is consistently exercised throughout all experiments in this study.

The unified ChatML prompt template, which embeds the structural instruction, query placeholder, and target document context, is illustrated in Figure 2.

None

None

8.9.0.2 Topic Anchoring \(\tau(q)\).

The detailed prompt template for this task is illustrated in Figure 3. The structured topic + facet extraction prompt used by the frozen LLM call \(\tau(q)\) produces the Topic Anchor topic, actor, temporal, magnitude, situational for every RegOps and HIPAA query. The {domain} placeholder is “Korean R&D funding regulations” for RegOps and “U.S. healthcare privacy regulations (HIPAA)” for HIPAA; {language} is “Korean” or “English” respectively. The output is constrained to the QueryAnalysis JSON schema.

8.9.0.3 Multi-View Tagged-Query Rendering.

Given the topic anchor produced by \(\tau(q)\), the three retrieval views \(q_{\text{narrow}}\), \(q_{\text{mid}}\), and \(q_{\text{wide}}\) are deterministically generated as shown in Table. 17. For instance, the resulting rendered \(q_{\text{mid}}\) view is structured as follows:

8.9.0.4 Schema-Constrained Generation.

This section details the system and user prompt templates employed by the RefWalk framework to realize the schema-constrained attribution mechanism described in §4.2. Rather than generating free-form prose followed by loose citation footers, the generative model is strictly constrained via a strict JSON schema. The keys of the emitted JSON object must correspond exactly to the node_id elements present within the retrieved context, and the values are restricted to arrays of granular claims bounded by paragraph-level annotations (e.g., [제O항]).

This architectural constraint structurally prevents post-hoc rationalization (i.e., “writing first, citing later”). This setup corresponds directly to the w/ schema RefWalk variant evaluated in Table 5 of the main paper. The formalized Korean variant of the system prompt and user template for the RegOps domain are presented in Figure 5.

None

None

None

8.9.0.5 NativeRAG Free-Form Generation Baseline.

To isolate and evaluate the baseline generation quality under unconstrained conditions, we implement a conventional free-form generation model paired with an appended citation-footer instruction. This setup serves as the generation standard for multiple baselines, including NativeRAG, LightRAG, HippoRAG-2, and PIKE-RAG, as well as the w/o schema ablation configuration of RefWalk. The prompt templates for this baseline are detailed in Figure 8.

None

8.10 Qualitative Examples↩︎

To illustrate the operational behavior of RefWalk (Qwen3.6-35B) across different reasoning depths, we present three qualitative case studies representing schema-bound exception surfacing (L2), exhaustive parallel-institution closure (L3), and cross-document delegation-chain traversal (L4).

In each case, the model is required to emit a structured per-rule attribution object containing governing article identifiers paired with their specific claims, followed by a synthesized final answer. Verbatim Korean originals are provided alongside polished English translations marked as [EN].

Analysis of Example 1. As illustrated in Figure 9, the ground-truth rule closure requires targeting exactly \(\{\text{Art.~28, Art.~73}\}\). RefWalk achieves perfect citation precision and high claim alignment (citation F1\(= 1.00\), claim F1 \(= 0.89\)). Crucially, rather than treating the exchange-rate clause as an unstructured text block, the attribution schema structurally binds the exception directly to its governing statutory articles, ensuring that the final counterfactual synthesis is grounded in traceable textual support.

Analysis of Example 2. As demonstrated in Figure 10, this instance evaluates a multi-institutional parallel rule structure. To construct a legally sound answer, the model must map the entire parallel web without missing the critical distinction between institution-level and project-level caps. RefWalk successfully establishes full reference closure by identifying all four parallel anchors (citation \(F_1 = 1.00\)). Rather than compressing distinct corporate entities into a single generic rule, it preserves the formal mutatis mutandis relationships as discrete, attributable statutory claims.

Analysis of Example 3. As shown in Figure 11, the legal reasoning path spans two distinct levels of authority, successfully tracing a hierarchical delegation link (Delegates_To/Specifies) from Enforcement Decree Art. 24 down to the international joint R&D provisions in Cost-Use Standards Art. 28. RefWalk successfully navigates this cross-document dependency chain without introducing hallucinated citations.

However, consistent with the precision-first behavior observed in our empirical evaluation, the model selectively recovers the primary structural anchors of the chain while missing surrounding contextual siblings (e.g., Cost-Use Standards Art. 12, 13, and 27). This behavior accounts for its lower article-level citation recall (\(0.40\)) on this specific problem instance, directly exposing the unresolved recall headroom that RegOps-Bench is tailored to isolate.

References↩︎

---