1 Introduction↩︎

Institutional question answering systems operate under constraints that differ substantially from open-domain conversational assistants [3], [4]. Queries in such environments frequently reference domain-specific entities, operational limits, eligibility rules, or conditional workflows whose relevant information is distributed across heterogeneous documents [5].

Retrieval-Augmented Generation (RAG) grounds language model outputs in external knowledge sources [1], but in entity-dense institutional corpora, semantic proximity alone is insufficient as queries may span multiple documents through implicit relational dependencies [6].

To address these limitations, many systems introduce prompt-level orchestration or agent-based control flow [7], increasing latency and computational cost through reliance on proprietary models that prioritize cross-domain generalization at the expense of domain-specific terminology precision [8].

This work adopts an alternative perspective: structural ambiguity should be resolved prior to generation, and the language model should be restricted to synthesizing evidence rather than performing implicit structural reasoning [9]. We therefore introduce GRACE-RAG, a retrieval-governed RAG architecture that externalizes entity normalization, relationship modeling, and semantic boundary alignment into an offline structure-manufacturing pipeline. During online inference, hybrid retrieval operates over dual embedding surfaces, content chunks and relationship summaries, allowing relational hypotheses to be ranked and validated before generation [7], [10].

The contributions of this paper are threefold:

• We introduce GRACE-RAG, a retrieval-governed RAG architecture that decouples structural reasoning from generation through offline knowledge construction and bounded hybrid retrieval.

• We present a dual-surface retrieval mechanism in which relationship summaries are embedded and indexed independently, enabling relational validation and graph-guided expansion without uncontrolled traversal.

• We empirically demonstrate that governed retrieval with canonical evidence structuring delivers consistent structural quality gains, enabling a practical shift to self-hosted lightweight models with measurably reduced computational cost and operational footprint.

2 Related Work↩︎

2.1 Vector-Based Retrieval-Augmented Generation↩︎

Vector-based Retrieval-Augmented Generation (RAG) has become a dominant paradigm for grounding language model outputs in external knowledge sources [1]. This approach is effective when relevant information is localized and semantically aligned with user queries.

However, in entity-dense institutional corpora, semantic proximity alone may be insufficient to capture relevance [5]. Information necessary to answer a single query can be distributed across multiple documents, linked implicitly through relational dependencies, or expressed using heterogeneous surface forms. Under these conditions, retrieval often yields fragmented context, increasing reliance on inference-time reasoning within the language model [7].

2.2 Graph-Based Retrieval-Augmented Generation↩︎

Graph-based RAG systems introduce explicit relational structure through knowledge graphs constructed from extracted entities and relationships [3]. Graph augmentation can improve multi-hop reasoning and enhance contextual completeness in domains where relational structure is central [11].

Despite these advantages, practical implementations often face challenges related to entity fragmentation, noisy extraction, and shallow traversal depth [12]. When entity surface forms are not normalized, structurally equivalent concepts may be represented as distinct nodes, reducing graph connectivity and recall. Additionally, graph-derived context is frequently appended to vector-retrieved text rather than integrated within a unified ranking framework [13]. As a result, structural signals may remain underutilized during retrieval-time decision making.

2.3 Hybrid Retrieval Architectures↩︎

Hybrid retrieval architectures combine vector similarity search with graph-based expansion in an effort to balance semantic relevance and relational awareness [10]. In many existing systems, contexts obtained from vector search and graph traversal are merged or concatenated prior to generation [14]. While this approach can increase coverage, integration typically occurs at the context level rather than at the retrieval-ranking level. The language model is therefore responsible for reconciling heterogeneous evidence during generation [9].

Moreover, hybrid designs often provide limited mechanisms for entity canonicalization, controlled traversal, or bounded inference complexity [13]. Without explicit structural constraints, improvements in answer quality may depend heavily on prompt design or model capacity rather than retrieval architecture itself.

2.4 Model-Centric vs Architecture-Centric Paradigms↩︎

Recent advances in large language models have led to a model-centric view of RAG effectiveness, where performance gains are frequently attributed to increased model scale, extended context windows, or complex prompt orchestration [1]. Multi-stage routing and agent-based pipelines further expand inference-time complexity in pursuit of improved coverage [7]. While such approaches can yield empirical gains, they also increase latency, operational cost, and dependence on proprietary systems [8].

3 Methodology↩︎

3.1 Governing Principles↩︎

The GRACE-RAG architecture is designed around three guiding principles:

1. Structural ambiguity is resolved offline: Entity equivalence, relational dependencies, and semantic boundaries are externalized into retrieval artifacts constructed prior to inference, rather than implicitly resolved at generation time [12].

2. Retrieval governs and bounds generation: Relevance decisions are determined through structured retrieval with a fixed sequence of operations [10], restricting the language model to evidence synthesis over curated context, minimizing latent reasoning and reducing sensitivity to model scale.

3. Closed-domain precision is a retrieval property, not a model property: Institutional terminology fidelity is encoded within retrieval artifacts rather than delegated to generative model capacity, enabling deployment on lightweight self-hosted models with reduced computational and operational footprint [8], [10].

3.2 Offline Structure Manufacturing↩︎

The offline pipeline, shown in Fig. 1, transforms raw institutional documents into structured retrieval artifacts. All computationally intensive and domain-specific reasoning is resolved during this stage to minimize online complexity.

3.2.1 Corpus Characteristics↩︎

The experimental evaluation was conducted on a large institutional document corpus exhibiting high entity density, with frequent references to operational parameters, actors, and conditional constraints. The corpus configuration is summarized in Table 1.

3.2.2 LLM-Assisted Semantic Chunking↩︎

Document segmentation is treated as a semantic alignment problem rather than a fixed-window tokenization task [2]. An LLM-assisted preprocessing stage restructures documents into context-preserving segments aligned with conceptual boundaries such as definitions, constraints, procedural steps, and eligibility conditions [2]. This reduces semantic fragmentation and improves retrieval precision without inflating context size [9].

3.2.3 Relationship Extraction and Typing↩︎

Each chunk is processed to extract entities and typed relationships using an LLM-based information extraction stage [15]. Relationships capture dependencies such as eligibility constraints, procedural sequencing, transactional limits, and policy applicability.

Extracted relationships are categorized into coarse semantic buckets (e.g., transactional, compliance-related, policy-driven). These categories act as retrieval priors rather than rigid ontological constraints, enabling intent-aligned filtering during online inference.

For instance, the sentence “UPI Lite transactions are capped at Rs.500 per transaction” yields the triple: (UPI Lite, per_txn_limit_500, Transaction Limit) under the limits category.

3.2.4 Construction of Dual Retrieval Indices↩︎

The offline pipeline constructs two independent embedding indices: one over semantically aligned document chunks and another over extracted relationship summaries [15]. Each identified relationship is condensed into a single-sentence semantic representation and embedded separately from its originating chunk [10]. This design enables relational hypotheses to be scored and ranked directly through vector similarity without requiring implicit structural reasoning at generation time [14].

3.2.5 Entity Canonicalization and Graph Densification↩︎

Raw entity extraction produces surface-form variation and structural fragmentation. To address this, entities are grouped via embedding similarity and validated through LLM-assisted confirmation to determine canonical equivalence [13].

For example, security credential variants such as “UPI PIN”, “PIN”, “UPI Security PIN”, and “4-digit PIN” are consolidated into a single canonical entity representation (“UPI PIN”). Similarly, organizational references such as “Board of Directors”, “Director’s Meeting”, and “Board Decision” are normalized to a unified structural entity (“Board”). These transformations reduce redundant nodes while preserving relational semantics. Fig. 3 illustrates this structural transformation, contrasting the fragmented entity space with the canonicalized graph.

This procedure reduces entity fragmentation and increases average node degree, improving relational recall under bounded traversal. The objective is not merely normalization but topological densification of the knowledge graph.

Table 2 summarizes the qualitative impact of this process.

3.2.6 Graph Community Detection and Structural Signals↩︎

Beyond entity canonicalization, structural properties of the knowledge graph are analyzed using community detection algorithms, including Louvain and Leiden modularity optimization [3]. These algorithms identify densely connected subgraphs representing semantically cohesive clusters within the institutional domain.

For instance, in the evaluated corpus, the canonical entity UPI PIN was assigned to a high-density community comprising approximately 120 related entities (e.g., authentication workflows, transaction limits, security controls). The node exhibited a degree of approximately 18, compared to a post-deduplication graph average of \(\sim\)​6.0. Such centrality indicates that the entity participates in multiple relational contexts across documents [11], [13]. Community membership and degree centrality therefore act as structural signals during retrieval, enabling prioritization of topologically influential nodes when forming relational hypotheses.

3.3 Online Retrieval-Governed Inference↩︎

3.3.1 Query Typing and Intent Conditioning↩︎

The online pipeline (Fig. 1) processes incoming queries exclusively over artifacts generated offline. Incoming queries are first classified into a small set of intent categories (e.g., factual, procedural, constraint-oriented). This classification does not alter control flow but conditions retrieval priorities, enabling intent-aligned filtering of relational signals during hybrid retrieval.

3.3.2 Subquery Decomposition and Parallel Processing↩︎

Complex queries are decomposed into atomic subqueries to improve retrieval precision. Entity extraction and canonical mapping are applied to each subquery independently. Subqueries are processed in parallel, ensuring bounded latency and avoiding iterative agent-style loops.

3.3.3 Hybrid Retrieval Per Subquery↩︎

For each subquery, retrieval proceeds through two coordinated channels: semantic similarity search over chunk embeddings and graph-guided expansion over canonical entities [15]. Graph traversal produces candidate relational hypotheses, which are validated using relationship-summary embeddings prior to chunk selection [10]. Retrieved evidence from both channels is merged and globally reranked before generation. Figure 4 illustrates this bounded hybrid retrieval process.

3.3.4 Bounded Inference Complexity↩︎

The online pipeline executes a fixed sequence of retrieval and ranking operations per query [7]. No iterative reasoning loops or adaptive routing strategies are employed. By constraining inference complexity, the architecture ensures predictable latency and reduces dependence on model-scale reasoning.

4 Model capacity independence↩︎

Model capacity independence refers to the extent to which response quality remains stable across variations in parameter scale when retrieval structure is controlled. The GRACE-RAG architecture externalizes structural reasoning prior to generation, ensuring the generator operates over curated and structurally coherent evidence rather than heterogeneous, weakly aligned context.

To evaluate whether this restructuring reduces sensitivity to parameter scale, we analyze performance across ten evaluation dimensions grouped into three categories. Concise definitions of all ten metrics are provided in Appendix 9.

Structural Metrics: Correctness, Completeness, Depth, Anticipatory Coverage, Search Engine Quality, and Structure.

Presentation Metrics: Clarity and Formatting.

Hybrid Metrics: Relevance and Informativeness.

Based on this grouping, two hypotheses are formulated:

• H1 (Architecture Dominance): Architectural restructuring produces larger improvements than parameter scaling for structurally governed metrics.

• H2 (Scale Dominance in Presentation): Parameter scaling produces larger improvements than architectural restructuring for presentation-oriented metrics.

Under this framework, model capacity independence does not imply that scale is irrelevant. Rather, it asserts that once structural ambiguity is resolved at retrieval time, improvements in relational completeness and dependency integration become less sensitive to parameter magnitude.

5 Evaluation↩︎

5.1 Experimental Setup↩︎

Three model capacities were evaluated: Mistral 24B², GPT OSS 120B³, and Gemini 2.5 Flash, each under two retrieval regimes: a Baseline Architecture using vector-based retrieval with standard chunk embeddings, and the GRACE-RAG Architecture using a retrieval-governed hybrid pipeline with entity canonicalization, graph traversal, and dual embedding surfaces. This results in six total evaluation configurations under controlled conditions [16].

5.2 Evaluation Criteria↩︎

Response quality was assessed using the ten-parameter framework introduced in Section 4 and summarized in Appendix 9, employing an LLM-as-a-judge evaluation protocol [1]. Each response was evaluated across all ten dimensions on a standardized 0–10 scale.

The evaluation protocol focuses on identifying two distinct sources of performance variation: improvements attributable to retrieval architecture and improvements attributable to model capacity [17]. By maintaining identical prompts, document collections, and preprocessing pipelines across all configurations, the evaluation isolates the effect of architectural restructuring from parameter scaling. LLM-as-a-judge evaluation is adopted as the primary protocol given the absence of ground-truth answer annotations in closed-domain institutional settings, consistent with established practice in open-ended RAG evaluation [18]. Judge scores were spot-checked against human assessments on a representative sample to validate scoring consistency, as detailed in the Reproducibility Statement.

5.3 Architecture Effect (Vertical Comparison)↩︎

Under Mistral 24B, structurally governed metrics exhibit substantial gains. Completeness improves from 6.88 to 8.66, representing a 25.9% increase. Depth increases from 5.96 to 8.07 (+35.4%), while anticipatory coverage rises from 5.11 to 7.77 (+52.1%). The overall average score improves from 6.99 to 8.39, corresponding to a 20.0% increase.

Similar patterns are observed for GPT OSS 120B, where the overall score increases by 18.7% (7.15 to 8.49). Even under Gemini 2.5 Flash, which already exhibits strong baseline performance, the GRACE-RAG architecture produces a 6.0% increase in overall score (8.40 to 8.90).

These results indicate that architectural restructuring yields consistent structural gains across model capacities.

5.4 Model Scaling Effect (Horizontal Comparison)↩︎

Within each architectural regime in Table 3, scaling from Mistral 24B to GPT OSS and Gemini primarily improves presentation-oriented metrics. For example, clarity and formatting demonstrate the most pronounced gains under larger models, while structural dimensions such as completeness and depth exhibit comparatively smaller incremental improvements once retrieval structure is fixed [19].

This pattern suggests that parameter scaling predominantly enhances linguistic refinement, whereas relational completeness and constraint integration are more strongly influenced by retrieval organization [1], [5].

5.5 Interaction Effect↩︎

Cross-architecture comparison reveals a compression of performance gaps across model capacities. Notably, the GRACE-RAG architecture operating with Mistral 24B (overall score 8.39) approaches the baseline performance of Gemini 2.5 Flash (8.40). In structurally governed metrics such as depth and completeness, the proposed mid-scale configuration matches or exceeds larger baseline models.

This interaction effect indicates that disciplined retrieval design reduces sensitivity to parameter magnitude in entity-dense institutional settings. Fig. 5 illustrates the compression of performance gaps across model sizes under the two retrieval architectures.

6 Discussion↩︎

6.1 Architecture as a Structural Multiplier↩︎

The empirical results indicate that retrieval structuring functions as a structural multiplier rather than a marginal optimization. By resolving entity equivalence, encoding typed relationships, and integrating relational validation prior to generation, the GRACE-RAG architecture increases the density and coherence of evidence presented to the language model. This suggests that relational signal quality, rather than parameter magnitude alone, governs performance in entity-dense institutional domains [5]. A qualitative response comparison illustrating these structural differences is provided in Appendix 8.

6.2 Parameter Scale versus Retrieval Discipline↩︎

The comparative analysis separates two largely orthogonal effects. Parameter scaling primarily enhances linguistic fluency and surface-level organization, as reflected in clarity and formatting metrics [1]. Retrieval discipline, in contrast, governs relational integration, constraint coverage, and evidence alignment [9]. Once structural ambiguity is reduced at retrieval time, incremental gains from additional parameters in structurally governed dimensions diminish.

6.3 Performance Gap Compression Across Model Capacities↩︎

The GRACE-RAG architecture operating with a mid-scale model approaches or exceeds the structural performance of larger models under baseline retrieval. This compression of performance gaps demonstrates that disciplined retrieval reduces sensitivity to parameter count in structurally governed tasks. Consequently, improvements traditionally attributed to scale may instead reflect unresolved structural ambiguity in weaker retrieval pipelines [7], [20]. As shown in Fig. 5, the GRACE-RAG quality curve shifts upward uniformly while inter-model variance narrows.

6.4 Implications for Retrieval-Augmented System Design↩︎

These findings support an architecture-centric perspective on RAG system development. In practical terms, this reduces dependence on frontier-scale proprietary systems for structurally coherent responses [10]. Architectural refinement therefore provides a controllable and reproducible mechanism for improving relational quality without proportionally increasing model capacity [9].

7 Conclusion↩︎

This work introduced GRACE-RAG, a retrieval-governed, graph-augmented RAG architecture designed to externalize structural reasoning from the generative stage to the retrieval layer. Through offline structural reasoning and bounded hybrid retrieval, GRACE-RAG restructures how evidence is aggregated prior to generation.

Empirical evaluation across three model capacities: Mistral 24B, GPT OSS 120B (MoE), and Gemini 2.5 Flash, demonstrates consistent improvements in structurally governed response metrics. As demonstrated in Table 3, completeness, depth, and anticipatory coverage increase substantially under the GRACE-RAG pipeline, with overall response quality improving by up to 20% under mid-scale models. These improvements persist across model capacities, indicating that relational coherence and constraint coverage are primarily determined by retrieval organization rather than by parameter scale alone [19]. In knowledge-intensive domains, retrieval discipline enables self-hosted lightweight models to match or exceed proprietary systems on structural quality, preserving domain jargon precision while reducing computational and latency footprint [7].

Ethics Statement↩︎

All experiments were conducted on anonymized, internally curated document corpora. No personally identifiable information (PII) was used at any stage of this work. No confidential or proprietary information is disclosed through the results, examples, or findings presented in this paper.

Reproducibility Statement↩︎

The offline and online pipeline components of GRACE-RAG are described in sufficient procedural detail in Sections 3.2 and 3.3 to permit independent reimplementation. Exact model identifiers are provided as footnotes in Section 5.1. Embedding models, reranking strategy, graph community detection libraries, evaluation protocol, and judge-consistency checks are documented in Appendix 10. The experimental corpus comprises approximately 300 institutional documents and will be released upon acceptance. Partial code release covering the offline pipeline is planned upon acceptance.

8 Illustrative response comparison↩︎

The following examples illustrate qualitative differences between baseline retrieval and the GRACE-RAG architecture across query types.

8.0.0.1 Example 1

User Query: Hello, I am considering linking my RuPay Credit Card to UPI. What are the advantages?

8.0.0.2 Baseline Response (Vector-Based Retrieval)

Linking your RuPay Credit Card to UPI offers several advantages:

1. Seamless Payments: Make credit-based merchant payments using UPI QR codes without needing a physical card.

2. Secure Transactions: Transactions are authenticated via UPI PIN, ensuring security.

3. AutoPay Setup: You can set up AutoPay for bill payments, making recurring payments convenient.

4. Dispute Resolution: Access dispute resolution via UPIHelp for any issues that arise.

5. No Physical Card Needed: Perform transactions directly through your UPI app (like BHIM, PhonePe, Google Pay, or Paytm).

8.0.0.3 GRACE-RAG Response

Linking a RuPay Credit Card to UPI allows customers to make secure and seamless credit-based merchant payments using UPI QR codes, without needing a physical card.

This feature enables users to link their RuPay Credit Card to UPI apps (like BHIM, PhonePe, Google Pay, etc.) using their registered mobile number. Once linked, users can make payments by scanning merchant QR codes and authenticating with a UPI PIN.

The primary advantage is the convenience of making contactless payments at UPI-enabled merchants. Users can also set up AutoPay for recurring bill payments, ensuring timely payments without manual intervention.

Notable benefits:

• Seamless linking via UPI app using the registered mobile number

• QR-based merchant payments authenticated via UPI PIN

• AutoPay setup for recurring bill payments

• Access to dispute resolution via UPIHelp

9 Evaluation metric definitions↩︎

The ten evaluation parameters used throughout the study are defined as follows:

10 Reproducibility Details↩︎

Models and Infrastructure
Experiments were conducted using Mistral-Small-3.1-24B-Instruct-2503, gpt-oss-120b, and Gemini 2.5 Flash, with exact model identifiers provided as footnotes in Section 5.1. Chunk and relationship summary indices were constructed using sentence-transformers embeddings. Global reranking was performed using a cross-encoder reranker. Graph community detection was implemented using the Louvain and Leiden algorithms via NetworkX.

Chunking and Retrieval Hyperparameters

Evaluation Protocol
Response quality was assessed using an LLM-as-a-judge protocol with Mistral-Small-3.1-24B-Instruct-2503 as the judge model. All ten evaluation metrics were scored simultaneously within a single structured prompt on a standardized 0–10 scale. The evaluation set comprised more than 200 queries spanning factual, procedural, and constraint-oriented intent categories. Judge scores were spot-checked against human assessments on a representative 30-query sample; mean absolute deviation between judge and human scores was within 0.4 points across all metrics, validating scoring consistency.

Corpus and Code
The experimental corpus comprises approximately 300 institutional documents and will be fully released upon acceptance. Partial code release covering the offline structure manufacturing pipeline and dual-index construction is planned upon acceptance. Components dependent on proprietary infrastructure will be documented with sufficient specification to permit reimplementation.

References↩︎

---

1. Preprint↩︎

2. Mistral-Small-3.1-24B-Instruct-2503↩︎

3. gpt-oss-120b: an open-weight Mixture-of-Experts model with 120B total parameters and approximately 5B active parameters per forward pass, released by OpenAI↩︎