1 Introduction↩︎

News retrieval aims to identify a comprehensive set of relevant articles for downstream ranking, recommendation, and content analysis in modern news systems [1]. However, user queries in this domain are often abstract or underspecified, particularly when addressing emerging events, breaking stories, or evolving topics, making effective semantic query expansion a critical prerequisite for high-recall retrieval [2]–[4]. Moreover, news corpora exhibit pronounced temporal dynamics: new articles are continuously published while previously relevant articles quickly become outdated, leading to frequent and abrupt shifts in relevance distributions [5], [6]. These characteristics demand retrieval mechanisms that are both semantically expressive and temporally adaptive.

Traditional news retrieval methods address semantic sparsity and temporal drift through query expansion [2] and freshness-aware ranking [7]. Techniques such as pseudo-relevance feedback and language-model-based expansion enrich under-specified queries to improve recall [3], while complementary approaches leverage temporal signals to prioritize recent or temporally aligned content [5], [6]. However, these methods typically perform expansion in an ad hoc, per-query manner and lack persistent semantic structures that evolve over time. This limits their ability to balance effectiveness and rapid adaptation to shifting relevance in real-world settings.

Recently, retrieval-augmented generation (RAG) has become a standard paradigm for grounding language models in external knowledge, improving semantic coverage in knowledge-intensive and time-sensitive information access settings [8]–[11]. Agentic extensions to classical RAG introduce planning, iterative query reformulation, and self-refinement, fostering more expressive and adaptive semantic expansion [12]–[15]. While effective for complex reasoning, many agentic RAG approaches operate via short-horizon, per-query inference loops, incurring significant latency and token costs through redundant, on-the-fly expansions. This tight coupling of reasoning and retrieval hinders reuse across queries or time, limiting their viability in time-sensitive, efficient news settings [16], [17].

A fundamental limitation of existing agentic RAG systems is that semantic expansion is treated as a transient byproduct of per-query reasoning rather than a persistent asset. This design fails to capitalize on two critical opportunities: reuse and relational modeling. Because expansions are discarded after each request, the system suffers from redundant computation and cannot accumulate semantic knowledge over time. Consequently, it fails to model how subtopics relate to one another or how a query’s facets evolve. This is particularly detrimental in news retrieval, where high-level intent (e.g., "AI in consumer electronics") remains stable, but specific subtopics and surface realizations shift daily. Current methods tend to overfit recently salient terms (e.g., "AI chips") while neglecting semantically coherent but less prominent subtopics like "on-device inference" or "privacy-preserving personalization." Over time, this structural mismatch between transient expansions and persistent intent causes systematic recall loss for relevant but underexplored content.

Motivated by this observation, we argue that semantic expansion must be materialized as a persistent, structured representation that captures the relationships between topics, rather than serving as a transient artifact. While prior graph-based methods improve matching efficiency [18], [19], they fail to address how such structures can be constructed via agentic reasoning or adapted to temporal dynamics. To bridge this gap, we propose DynaTree, a framework that decouples expansion from real-time retrieval to enable reuse. In the first stage, we amortize agentic RAG to build a reusable retrieval tree that maps the query’s semantic space. In the second stage, a lightweight daily subtree selection mechanism adapts to evolving news distributions without repeated agent execution or model retraining. This design achieves high recall with low latency, making DynaTree ideal for real-time retrieval over continuously updated corpora. Finally, our method has been deployed in the production-level news retrieval system Syft and evaluated on real user traffic through online A/B testing. Over a ten-day period (Jan. 28–Feb. 6, 2026), the dynamically adapted variant with daily subtree selection consistently improves survival rate from \(0.32\)–\(0.53\) to \(0.59\)–\(0.73\), yielding up to \(\sim\!1.5\times\) gains over a fixed offline-selected subtree and outperforming all existing production recallers. Figure 1 provides an overview of the deployed product pipeline.

Our main contributions are summarized as follows:

• We propose DynaTree, the first agentic framework to decouple semantic expansion from online retrieval. By materializing topic relationships into a reusable retrieval tree and enabling adaptation via lightweight subtree selection, it effectively reconciles the trade-off between recall, adaptability, and latency in real-time news retrieval.

• We introduce the Syft News dataset, a large-scale benchmark designed to evaluate high-recall retrieval under realistic temporal distribution shifts. Extensive experiments on this and public BEIR benchmarks demonstrate that DynaTree consistently outperforms strong RAG and agentic baselines.

• We report on the successful deployment of DynaTree within the Syft news platform’s production pipeline. Operating on live traffic under strict latency constraints, the system delivered measurable gains in retrieval quality, validating its practical effectiveness.

2 Related Work↩︎

2.0.0.1 Retrieval-Augmented Generation and Agentic Retrieval

Retrieval-Augmented Generation (RAG) grounds language model outputs in externally retrieved evidence, improving factuality and coverage for knowledge-intensive tasks [8]. Subsequent work has systematized RAG design choices, including retriever–generator coupling, context selection, and efficiency trade-offs  [20], while recent surveys on LLMs for information retrieval (LLM4IR) highlight the role of large language models as query rewriters, rerankers, and reasoning modules within retrieval pipelines [21]. A key emerging direction is agentic retrieval, where models plan, invoke tools, and iteratively refine retrieval through reasoning steps. Representative frameworks such as ReAct and Reflexion demonstrate how step-wise tool use and self-reflection can improve retrieval-informed generation  [12], [13], while complementary work emphasizes query rewriting as a structured interface between LLM reasoning and retrieval, showing that rewritten queries can better align retrieval with downstream generation objectives [22]. More recent agentic RAG systems incorporate self-evaluation and adaptive retrieval control to improve robustness across queries [14], [23]. Despite these advances, most existing approaches tightly couple semantic expansion, retrieval, and reasoning within per-query or short-horizon inference loops, making semantic expansions transient and computationally expensive to repeat in continuously updated environments. In contrast, our framework materializes semantic expansion as a persistent retrieval tree and decouples offline semantic exploration from daily lightweight optimization, enabling amortized agentic reasoning and structural reuse for time-sensitive news retrieval.

2.0.0.2 Semantic Expansion and Time-Sensitive Information Retrieval

Semantic expansion has been a central technique for improving recall on abstract or underspecified queries. Classic relevance feedback and probabilistic relevance models provide principled ways to enrich query representations using pseudo-relevant documents [3]. Subsequent surveys summarize pseudo-relevance feedback and knowledge-based expansion as core paradigms in information retrieval  [2], [24]. In parallel, time-sensitive retrieval introduces additional challenges, as relevance distributions shift with the continuous arrival of new documents and the rapid obsolescence of older information. Prior temporal IR work has studied recency-aware ranking and temporal signals for web and news search  [25], [26], as well as broader formulations of temporal information retrieval and benchmark tasks  [27], [28]. Classic stream-based settings, such as knowledge-base acceleration and temporal summarization, study timely updates or meaningful deltas from incoming document streams [28], [29]. They are relevant to time-sensitive retrieval, but are often entity-centric or update-centric. More recently, structured and graph-based retrieval methods have been proposed to better capture semantic relationships among documents or concepts, including graph-augmented RAG and hierarchical or tree-organized retrieval schemes [18], [19], [30]. While such approaches improve global semantic coverage or multi-hop interaction, their structures are typically constructed from static corpora or document relations and are not explicitly designed to be reused or adapted under strong temporal dynamics. Recent LLM-based methods further address temporal gaps by conditioning generation on freshly retrieved documents or incorporating time awareness into model behavior, but adaptation is still performed at the document or query level without maintaining a persistent semantic structure across time [7], [11]. In contrast, our approach explicitly separates stable semantic coverage from daily temporal adaptation: the former is materialized as a reusable agentic retrieval tree, while the latter is implemented through lightweight subtree selection over a time-localized proxy. This design enables high recall in dynamic news environments at substantially reduced cost.

3 Methodology↩︎

3.1 Problem Formulation and Framework↩︎

We study the problem of semantic expansion for effective news retrieval through an explicit retrieval tree abstraction. Given a user query \(q\) and a large news corpus \(\mathcal{D}=\{d_1,\dots,d_N\}\), our goal is to systematically explore the semantic space induced by \(q\), materialize this exploration as a reusable and structured representation, and leverage it to improve retrieval quality under realistic ranking cutoffs. Most existing query expansion and agentic RAG approaches rely on flat or short-horizon expansions, which tightly couple semantic exploration with per-query retrieval decisions and hinder structural reuse under evolving news distributions. In contrast, we model semantic exploration as a tree-structured process, where different branches capture distinct subtopics, semantic interpretations, and refinement trajectories of the original query. By explicitly organizing semantic expansion into a structured retrieval tree, our abstraction enables systematic comparison, pruning, and reuse of alternative retrieval behaviors, and provides a well-defined foundation for adaptive retrieval under dynamic news environments.

3.1.0.1 Retrieval Tree and Subtree-Induced Retrieval.

Formally, a retrieval tree \(\mathcal{T}\) is rooted at the original query \(q\). Each node represents a semantic expansion, and edges encode refinement or specialization relations between expansions. A root-to-leaf path \[p = (v_0, v_1, \dots, v_L), \quad v_0 = q\] defines a coherent semantic trajectory that progressively refines the original query. Rather than treating individual expansion terms independently, we regard each path as an atomic semantic unit that captures a complete, structured interpretation of the query, preserving both subtopic context and refinement order.

While the full tree \(\mathcal{T}\) provides broad semantic coverage, using all paths simultaneously may introduce redundancy or amplify semantically misaligned expansions. We therefore consider a feasible set of candidate subtrees \(\mathbb{T}(\mathcal{T})\), where each subtree \(\mathcal{T}' \subseteq \mathcal{T}\) is a connected subgraph containing the root. Each subtree induces a distinct retrieval behavior through the collection of semantic paths it retains. We denote by \(\mathcal{R}(q, \mathcal{T}')\) the ranked list of documents returned by the retrieval-and-ranking procedure induced by \(\mathcal{T}'\).

Given a relevance set \(\mathcal{D}^{+}(q) \subseteq \mathcal{D}\), objective is to select the subtree that maximizes retrieval effectiveness. We instantiate this using Recall to emphasize semantic coverage at retrieval stage: \[\mathcal{T}^{\star} = \arg\max_{\mathcal{T}' \in \mathbb{T}(\mathcal{T})} \;\mathrm{Recall}\bigl( \mathcal{R}(q, \mathcal{T}'),\; \mathcal{D}^{+}(q) \bigr) \label{eq:objective}\tag{1}\]

We use Recall as the selection objective to emphasize semantic coverage at the retrieval stage, and report ranking-oriented metrics such as NDCG@10 in evaluation. This formulation explicitly decouples semantic exploration from retrieval optimization: the retrieval tree is constructed once to comprehensively capture the structured semantic space of a query (Stage I), while retrieval behavior is adaptively optimized over time via lightweight subtree selection (Stage II).

3.1.0.2 Path-Based Encoding and Scoring.

Each subtree \(\mathcal{T}'\) induces a set of root-to-leaf paths \(\mathcal{P}(\mathcal{T}')\), which we interpret as an ensemble of complementary semantic trajectories covering different subtopics and refinement perspectives of the query. For each path \(p \in \mathcal{P}(\mathcal{T}')\), we construct a unified textual representation \(x_p\) by composing the textual descriptions along the refinement trajectory, and encode it holistically using a text-embedding model: \[\mathbf{e}_p = \mathrm{Enc}(x_p) \in \mathbb{R}^{d} \label{eq:path95emb95stage3}\tag{2}\]

Given a document \(d\) with embedding \(\mathbf{e}_d\), its relevance score under subtree \(\mathcal{T}'\) is computed by aggregating semantic similarity across all induced paths: \[s(d \mid \mathcal{T}') = \frac{1}{|\mathcal{P}(\mathcal{T}')|} \sum_{p \in \mathcal{P}(\mathcal{T}')} \cos\!\left( \mathbf{e}_d,\; \mathbf{e}_p \right) \label{eq:path95ensemble95score}\tag{3}\] This path-ensemble formulation favors documents that receive consistent semantic support from multiple structured trajectories, thereby promoting comprehensive semantic coverage in recall-oriented news retrieval settings.

3.1.0.3 Discussion: Why a tree structure?

DynaTree adopts a tree because Stage I expansion is inherently path-conditioned and recursive: each node is generated along a specific root-to-node semantic trajectory, making each path an ordered expansion unit. This structure also makes Stage II connected-subtree selection tractable under a fixed online budget. In contrast, general graphs introduce path explosion and unstable structure selection. Thus, the tree is not only a representation choice, but also a computational constraint for efficient latency-sensitive adaptation.

3.2 Agentic Retrieval Tree Construction↩︎

Stage I constructs a retrieval tree \(\mathcal{T}\) that systematically explores semantic space of a query for effective news retrieval, while explicitly controlling redundancy, semantic drift, and unproductive expansion. Rather than treating retrieval-augmented generation as a monolithic, per-query operation, we formulate tree construction as a role-specialized agentic process that decomposes semantic exploration into planning, retrieval, evidence normalization, and structural self-correction. This decomposition enables fine-grained control and reasoning over different aspects of semantic expansion within an agentic RAG framework, resulting in a structured, interpretable, and reusable representation that captures diverse query interpretations and their underlying subtopic structure.

3.2.0.1 Recursive Expansion as a Controlled Operator.

Let \(p_\ell = (v_0,\dots,v_\ell)\) denote a root-to-node path in the retrieval tree. Each expansion step applies a controlled expansion operator that maps the current path to a set of semantically distinct child nodes: \[\{v_{\ell+1}^{i}\} = \mathcal{F}_{\mathrm{agent}} \bigl(q,\; p_\ell,\; \mathcal{D}\bigr) \label{eq:agentic95expand}\tag{4}\] where \(\mathcal{F}_{\mathrm{agent}}\) denotes a coordinated interaction among agents, subject to explicit structural constraints such as max tree depth, branching factor, and semantic diversity. This abstraction enables high-level control over growth and coverage of the retrieval tree, while remaining agnostic to internal realization of individual agents.

3.2.0.2 Planning Agent: Path-aware Semantic Planning.

The Planning Agent proposes a retrieval query jointly conditioned on the original user query \(q\) and the semantic trajectory encoded by the current path \(p_\ell\). This process is modeled as a stochastic policy: \[x_\ell \sim \pi_{\mathrm{plan}}(q, p_\ell)\] where \(x_\ell\) denotes the planned retrieval query at depth \(\ell\).

By conditioning on the entire root-to-node path rather than the current node alone, the Planning Agent explicitly incorporates expansion history and accumulated semantic context. This path-aware design encourages semantic progression along the retrieval tree, discouraging shallow paraphrasing and redundant reformulations. Consequently, deeper nodes tend to explore increasingly specific, complementary, or orthogonal semantic facets of the original query, leading to broader and more structured coverage of the underlying subtopic space.

3.2.0.3 Retrieval Agent: Adaptive Retrieval Routing.

Given a planned query \(x_\ell\), the Retrieval Agent adaptively selects an appropriate retrieval strategy: \[r_\ell \sim \pi_{\mathrm{ret}}(x_\ell)\] where \(r_\ell\) may correspond to lexical retrieval (e.g., BM25), dense retrieval, or a hybrid configuration. Conditioned on the selected strategy, the agent retrieves a candidate evidence set: \[\mathcal{D}_\ell = \mathrm{Retrieve}(x_\ell, r_\ell, \mathcal{D})\]

The selection policy accounts for both the semantic characteristics of the planned query (e.g., lexical specificity versus abstraction) and contextual signals accumulated along the expansion path. By routing different branches of the retrieval tree to complementary retrieval signals, the framework avoids early commitment to a single retriever and enables robust evidence discovery across heterogeneous semantic facets.

3.2.0.4 Augmented Agent: Budget-aware Evidence Normalization.

Raw retrieval outputs are often noisy, redundant, and may substantially exceed the available context budget. The Augmented Agent performs budget-aware evidence normalization, transforming heterogeneous retrieval results into a compact, structured, and information-dense evidence set suitable for downstream expansion. This process may involve lightweight re-ranking, extractive or abstractive compression, and explicit redundancy reduction, all subject to a predefined budget constraint. To ensure robustness, abstractive compression gracefully falls back to extractive normalization when confidence is insufficient. All retained evidence preserves source identifiers, maintaining full traceability to the original documents.

Formally, given retrieved documents \(\mathcal{D}_\ell\) and a context budget \(B\), the normalization operator is defined as \[\mathcal{E}_\ell = \mathcal{G}_{\mathrm{aug}} \bigl( q,\, \mathcal{D}_\ell,\, B \bigr), \quad \text{s.t. } \sum_{e \in \mathcal{E}_\ell} \mathrm{len}(e) \le B \label{eq:augmented95normalization}\tag{5}\] where \(\mathcal{E}_\ell\) denotes the normalized evidence set. The operator jointly optimizes relevance to \(q\), information density, and redundancy control, while strictly enforcing budget feasibility.

3.2.0.5 Child Node Generation.

Conditioned on the normalized evidence \(\mathcal{E}_\ell\), the system generates multiple child nodes, each corresponding to a distinct semantic refinement grounded in retrieved evidence. Each child node is associated with a concise textual description and is immediately incorporated into the retrieval tree, enabling further controlled expansion and downstream evaluation.

3.2.0.6 Reflection Agent: Structural Self-correction.

Recursive expansion without feedback may lead to semantic drift, over-specialization, or structurally unproductive branches. To mitigate these issues, we introduce a Reflection Agent that performs retrieval-grounded structural self-correction during retrieval tree construction. After a batch of child nodes is expanded, the agent evaluates each newly induced root-to-node path \(p_{\ell+1}^i = (v_0,\dots,v_\ell,v_{\ell+1}^i)\) by issuing a lightweight retrieval using \(p_{\ell+1}^i\) as query context, and analyzes the resulting evidence distribution \(\mathcal{D}_{\ell+1}^i\) in terms of relevance concentration, semantic coherence, and contribution to overall coverage. Based on this retrieval feedback, the agent selects a localized structural action \(a_{\ell+1}^i \in \{\mathrm{\small Add}(v_{\ell+1}^i),\mathrm{\small Prune}(v_{\ell+1}^i),\mathrm{\small Revise}(v_{\ell+1}^i)\}\) to adjust the tree structure. The Reflection Agent operates purely at the structural level, suppressing redundant or low-yield branches and improving balance and coverage. Formally, the reflection is defined as: \[\begin{align} a_{\ell+1}^i &\sim \mathrm{Reflect}\bigl(\mathcal{T},\; p_{\ell+1}^i,\; \mathcal{D}_{\ell+1}^i\bigr) \\ \mathcal{T} &\leftarrow \Phi\bigl(\mathcal{T},\; v_{\ell+1}^i,\; a_{\ell+1}^i\bigr) \end{align} \label{eq:reflection95update}\tag{6}\]

where \(\mathrm{Reflect}(\cdot)\) denotes a retrieval-grounded structural policy and \(\Phi(\cdot)\) applies the corresponding localized transformation.

3.2.0.7 Discussion.

Stage I formulates retrieval tree construction as a controlled, role-specialized agentic process, enabling systematic semantic exploration while explicitly managing redundancy, semantic drift, and structural imbalance. The resulting retrieval tree is constructed once per query topic and serves as a persistent, reusable representation of the query’s semantic space, allowing Stage II to adapt retrieval behavior to evolving news distributions without re-invoking expensive agentic reasoning.

3.3 Dynamic Adaptive Subtree Selection↩︎

Stage I constructs a retrieval tree that captures diverse semantic interpretations of a query. However, indiscriminately applying the full tree is neither necessary nor optimal under evolving news distributions. As surface realizations, topical emphasis, and relevance patterns shift over time, different substructures of the tree may exhibit substantially different retrieval effectiveness. To address this mismatch, we introduce a dynamic adaptive subtree selection mechanism that adapts retrieval behavior by selecting an appropriate subtree, without re-running expensive semantic exploration.

As formalized in Section 3.1, the ideal objective is to maximize recall-oriented news retrieval performance over the full corpus. Directly optimizing Eq. (1 ) is not only computationally infeasible, but also misaligned with the structural nature of the problem: the goal of Stage II is not to estimate absolute recall, but to reliably rank alternative semantic structures under stable and bounded cost. Instead, Stage II focuses on efficiently approximating this objective by enabling reliable relative comparison among candidate subtrees under stable and bounded computational cost.

3.3.0.1 Evaluation Proxy Construction.

We first instantiate the retrieval function \(\mathcal{R}(q,\mathcal{T})\) using the full retrieval tree to obtain a global ranking. Evaluation is then restricted to a compact candidate pool \[\mathcal{C} = \mathrm{Top}_{M}\bigl(\mathcal{R}(q,\mathcal{T})\bigr)\] which concentrates documents most strongly supported by the current semantic structure while keeping downstream computation tractable. Each document in \(\mathcal{C}\) is represented by a dense embedding, enabling efficient similarity-based operations.

To avoid overemphasizing dominant semantic modes, we further construct a diversity-aware evaluation proxy. Specifically, we perform coarse-grained clustering over document embeddings in \(\mathcal{C}\) and allocate evaluation quotas proportionally across clusters. Let \(\{\mathcal{C}_1, \dots, \mathcal{C}_K\}\) denote the resulting partition. Given a target proxy size \(N_{\mathrm{eval}}\), cluster-level quotas are defined as \[n_k = \left\lfloor N_{\mathrm{eval}} \cdot \frac{|\mathcal{C}_k|}{|\mathcal{C}|} \right\rfloor, \qquad k = 1,\dots,K \label{eq:cluster95quota}\tag{7}\] Within each cluster \(\mathcal{C}_k\), documents are selected by global rank, yielding \(\mathcal{P}_k = \mathrm{Top}_{n_k}(\mathcal{C}_k)\). The final evaluation proxy \(\mathcal{P}_{\mathrm{eval}}\) is initialized by aggregating \(\{\mathcal{P}_1, \dots, \mathcal{P}_K\}\). The remaining slots, if any, are filled according to the global ranking until the proxy reaches the target size \(N_{\mathrm{eval}}\). When clustering quality degrades, the procedure gracefully falls back to stratified rank-based selection.

Subtree selection is intentionally evaluated on a compact proxy rather than the full corpus, yet this approximation remains well aligned with the ideal objective. The candidate pool is derived from full-tree retrieval, which already emphasizes high-coverage regions, while diversity-aware sampling preserves representation across multiple semantic facets. Crucially, subtree selection is inherently relative: as long as proxy-based estimates preserve the ordering among candidate subtrees, the selected structure remains stable, even if absolute recall values are imperfect.

3.3.0.2 Subtree Evaluation and Selection.

As formulated in Section 3.1, the ideal objective is to select a subtree that maximizes Recall with respect to the true relevance set \(\mathcal{D}^{+}(q)\), which is infeasible to obtain at scale. We therefore approximate this objective using a compact yet semantically diverse evaluation proxy \(\mathcal{P}_{\mathrm{eval}}\), where documents are weakly annotated by an LLM as relevant or non-relevant to \(q\), yielding a weakly labeled relevant subset \(\mathcal{P}_{\mathrm{eval}}^{+}(q)\), where \(|\mathcal{P}_{\mathrm{eval}}^{+}(q)| = \alpha\). For each candidate subtree \(\mathcal{T}' \in \mathbb{T}(\mathcal{T})\), we instantiate the induced path-ensemble retrieval function \(\mathcal{R}(q,\mathcal{T}')\) and estimate its Recall@\(\alpha\) on \(\mathcal{P}_{\mathrm{eval}}\). The selected subtree is thus obtained by \[\mathcal{T}^\star = \arg\max_{\mathcal{T}' \in \mathbb{T}(\mathcal{T})} \mathrm{Recall@}\alpha\bigl( \mathcal{R}(q, \mathcal{T}')|_{\mathcal{P}_{\mathrm{eval}}}, \mathcal{P}_{\mathrm{eval}}^+(q) \bigr)\]

which serves as a practical approximation to the ideal objective in Eq. (1 ).

3.3.0.3 Discussion.

Dynamic adaptive subtree selection enables rapid adaptation to evolving news distributions by reconfiguring semantic structure instead of regenerating semantic expansions. By selecting reusable subtrees, the framework amortizes expensive exploration in Stage I and achieves efficient, robust adaptation in Stage II, maintaining high coverage, low latency, and strong interpretability for large-scale news retrieval. Stage II uses LLM-based weak labels only for relative subtree comparison, so label noise mainly affects selection stability rather than directly biasing retrieval. To reduce proxy bias, we build the proxy via embedding-based clustering and quota-based sampling, encouraging coverage of diverse semantic modes beyond dominant full-tree retrieval directions. This design makes agentic retrieval viable in production, where semantic exploration is costly, news distributions evolve continuously, and retrieval policies must adapt under strict latency constraints.

4 Experiments↩︎

3pt

In this section, we evaluate the proposed agentic semantic expansion framework under realistic industrial constraints, using both offline controlled experiments and online platform evaluations. Our evaluation focuses on assessing the robustness and effectiveness of structured agentic retrieval across complementary retrieval metrics and deployment settings. Our experimental design is structured around the following research questions:

• RQ1: Does the agentic retrieval tree consistently improve retrieval effectiveness over strong RAG and agentic baselines across both Syft news data and public benchmarks?

• RQ2: How do different design choices within the agentic framework contribute to overall effectiveness, including individual agentic components and best-subtree selection compared to using the full retrieval tree?

• RQ3: How sensitive is the proposed framework to key design parameters, such as retrieval tree depth and size of the evaluation proxy used for dynamic subtree selection?

• RQ4: Does the proposed method lead to measurable improvements in real-world user experience on our online news retrieval platform?

4.1 Experimental Setup↩︎

4.1.0.1 Datasets.

Our primary evaluation corpus is a proprietary news dataset collected by our project team (Syft), covering news stories produced on June 11–15, June 29, and June 30, 2025 across diverse real-world topics (See Appendix 7 for details). We define 50 query themes for offline evaluation, for which relevance judgments are annotated against the news corpus of each corresponding day, and for online evaluation, we further introduce 30 complex themes and 20 long-tail themes to reflect more challenging real-world queries. To assess generalization beyond the internal domain, we also evaluate our approach on 5 publicly available datasets from the BEIR benchmark [31], which provides standardized retrieval tasks across diverse domains.

4.1.0.2 Evaluation Metrics.

Offline retrieval performance is evaluated following BEIR protocol  [31], using Recall@100 and NDCG@10. For datasets with more than 100 relevant documents per query (e.g., Syft news and Trec-Covid in BEIR), we report capped Recall@100 as defined in BEIR. Recall@100 measures high-recall retrieval effectiveness, while NDCG@10 captures ranking quality with position bias [32]. Online performance is evaluated using a platform-provided survival rate, which reflects whether retrieved items are retained after downstream filtering and quality control, a standard proxy for content utility in industrial retrieval systems.

4.1.0.3 Baselines.

We compare our method against a diverse and representative set of strong baselines spanning multiple paradigms of LLM-based retrieval and semantic expansion. For offline evaluation, we include agentic and reasoning-based methods (ReAct [12], Reflexion [13], FreshLLMs [11], and Self-RAG [14]), a representative LLM-based query expansion approach (HyDE [33]), as well as structure- and knowledge-aware retrieval methods, including RAPTOR [30], GraphRAG [18], and the recent KG\(^2\)RAG [19] framework, which incorporates explicit knowledge graph structures into RAG. To better relate DynaTree to classical stream-based retrieval, we evaluate KBA-style lexical and nugget-based baselines on the Syft news benchmark, including BM25, expanded BM25, nugget-based BM25, and nugget coverage. These baselines are intended as diagnostic comparisons for topic-level news retrieval. For online evaluation, we further compare against five production-grade recallers deployed within the same product line, including relation-based recall, decompositional recall, direct semantic recall, collaborative proxy recall, and web search recall. (See Appendix 8 for details).

4.1.0.4 Other Settings.

Unless otherwise specified, all agentic components use Gemini-2.5-Flash [34]. Documents and semantic expansions are encoded with text-embedding-3-large, and cosine similarity is used for matching. All experiments enforce a fixed semantic expansion budget, including constraints on retrieval tree depth and total node count. To isolate the effect of dynamic adaptive subtree selection, the retrieval tree is constructed once and reused across experiments. For the Syft news benchmark, the tree is built on the June 29, 2025 snapshot and reused for all subsequent dates; Stage II performs subtree selection using a fixed proxy set of size \(N_{\mathrm{eval}} = 200\) under a unified evaluation protocol, with a three-level retrieval tree. In contrast, BEIR benchmarks do not involve subtree selection: Stage II is designed to adapt retrieval behavior under evolving news distributions, whereas BEIR provides a static offline corpus. We therefore directly use the full retrieval tree, constructed on a restricted corpus of the top 10,000 query-relevant documents selected via dense retrieval, to ensure a stable and reproducible evaluation setting. All BEIR datasets also use a three-level retrieval tree, except SciDocs and FiQA, which adopt a two-level tree due to their larger query sets. We report absolute construction cost and scaling analysis in Appendix 9.

4.2 Offline Evaluation↩︎

4.2.1 Main Results (RQ1)↩︎

Table 1 reports the main offline results on the Syft news benchmark and the BEIR datasets. On the Syft news corpus, our method consistently achieves the strongest performance in both Recall@100 and NDCG@10 across all seven evaluation dates, demonstrating superior early-stage ranking quality and robust recall under realistic retrieval cutoffs. Notably, the improvements remain stable across temporally distinct news snapshots, indicating that the reusable agentic retrieval tree effectively adapts to evolving relevance distributions without repeated semantic exploration. On the BEIR benchmarks, our approach also attains the best overall performance on average, outperforming strong query-expansion and agentic RAG baselines across heterogeneous domains and query types. These results suggest that the proposed retrieval tree representation generalizes well beyond the news domain without task-specific tuning, while Stage-II subtree selection further improves adaptation in dynamic news settings.

4.2.1.1 Alignment with KBA-style Stream Retrieval.

DynaTree is not designed for the original KBA objective of entity-centric delta detection, but it shares with KBA-style evaluations the need to retrieve relevant information from continuously evolving document streams. The key difference is that DynaTree targets topic-level, latency-constrained news retrieval for downstream ranking rather than knowledge-base updates. To examine this connection, we introduce KBA-style lexical and nugget-based baselines on the Syft news benchmark, including BM25, expanded BM25, nugget-BM25, and direct nugget coverage. As shown in Table 2, DynaTree achieves 0.475 Recall@100 and 0.757 NDCG@10 on the seven-day average, substantially outperforming the strongest KBA-style baseline, which achieves 0.158 Recall@100 and 0.389 NDCG@10. These results suggest that flat lexical expansion or isolated nugget matching is insufficient for topic-level semantic coverage in dynamic news streams, while DynaTree benefits from structured semantic expansion and adaptive subtree selection.

4.2.2 Ablation Study (RQ2)↩︎

4.2.2.1 Impact of Agentic Components.

Table 3 reports both single-module ablations and Shapley-based attributions, with all results averaged over seven evaluation days on the Syft news benchmark using the full tree. Single ablation reflects the effect of removing a module only in the full configuration, while Shapley values measure average marginal contributions over all component subsets and thus capture interaction effects. Empirically, removing any module degrades overall performance (e.g., w/o P lowers Recall@100 from 37.50 to 36.82, and w/o F lowers NDCG@10 from 67.82 to 67.05), confirming that all agents are beneficial. Shapley analysis shows that Planning contributes most to Recall (+0.253), whereas Reflection contributes most to NDCG (+0.566), while Retrieval and Augment exhibit smaller or mixed effects, consistent with strong interdependencies. Detailed attribution protocol and results of \(2^4\) configs are provided in Appendix 10.

4pt

4.2.2.2 Effect of Stage-II Subtree Selection.

Figure 3 compares retrieval performance between using the full retrieval tree and the dynamically selected best subtree across seven days of news data. Stage-II subtree selection consistently improves both Recall@100 and NDCG@10 on all evaluated dates. The gains are stable over time, indicating that selecting a compact, adaptive subtree is more effective than indiscriminately using the full tree. These results demonstrate that Stage-II successfully adapts retrieval behavior to temporal variations in news distributions, achieving higher recall and better ranking quality.

4.2.2.3 Uniform vs. Weighted Aggregation.

Across seven days of news data, we compare two retrieval uses of the dynamically selected best subtree: uniform averaging versus softmax weighting of article–path similarities. Uniform averaging encourages balanced coverage across selected semantic paths, while Stage II already provides implicit structure-level weighting through subtree selection. Uniform aggregation slightly outperforms softmax weighting over seven daily snapshots (0.475/0.757 vs. 0.464/0.752 in Recall@100/NDCG@10). Full results are reported in Appendix 11.

4.2.3 Sensitivity Analysis (RQ3)↩︎

4.2.3.1 Effect of Retrieval Tree Depth.

Figure 4 examines the effect of retrieval tree depth by comparing Level-2 and Level-3 best subtrees across seven evaluation days on Syft news. Across all days, Level-3 consistently achieves higher Recall@100 and NDCG@10 than Level-2, with more stable gains under distributional shifts. This suggests that deeper retrieval trees provide a richer semantic structure, which becomes effective when adaptive subtree selection isolates high-utility branches while suppressing redundant or noisy expansions.

4.2.3.2 Evaluation Proxy Size Analysis.

We analyze how the size of the evaluation proxy used for subtree selection affects Stage II performance on the June 29 news snapshot. As the proxy size increases, both Recall@100 and NDCG@10 improve steadily, indicating more reliable relative comparisons among candidate subtrees. Very small proxies yield unstable estimates, while performance gains diminish beyond approximately 150 samples. With 200 proxy examples, Stage II achieves Recall@100 of 0.360 and NDCG@10 of 0.844, closely approaching the upper-bound performance (using full data as proxy). We also observe strong agreement between LLM-based weak labels and human annotations on sampled proxy examples, suggesting that the proxy provides a reliable signal for relative subtree comparison. Overall, these results demonstrate that Stage II is data-efficient and remains robust under realistic news distributions, even when guided by a compact evaluation proxy (Figure 5).

4.3 Online Deployment and Evaluation (RQ4)↩︎

4.3.0.1 Online Setting.

We evaluate DynaTree in Syft, a production-level AI-powered news platform developed by Orion Arm AI that delivers personalized, topic-centric news channels to a global user base. Syft aggregates news from diverse international sources, deduplicates overlapping reports, and produces concise summaries, organizing content around persistent user-defined interest channels (e.g., technology policy, financial markets, or regional affairs) rather than a generic feed. From a retrieval perspective, Syft operates a large-scale online pipeline where multiple production-grade recallers jointly generate candidates, including relation-based recall (RelRec), decompositional recall (Decomp), direct semantic recall (SemRec), collaborative proxy recall (Proxy), and web search recall (WSRec). Candidates from all recallers are merged and processed by a shared downstream filtering and ranking pipeline under identical online constraints. The channel-centric and continuously evolving nature of Syft induces highly diverse and dynamic retrieval demands, making it a realistic and high-stakes environment for evaluating online news retrieval quality, where recall effectiveness directly impacts user experience.

4.3.0.2 A/B Testing.

To assess the effectiveness of online daily subtree adaptation, we conduct an A/B test in the deployed DynaTree system. The full agentic retrieval tree is constructed once using the January 27, 2026 news corpus, covering 100 representative topic channels, and is kept fixed throughout the experiment, ensuring that both variants share an identical semantic structure and differ only in subtree selection. User traffic for each channel is evenly split: Port A uses a static strategy, where query expansions are generated from a best-performing subtree identified offline and kept unchanged, while Port B applies a dynamic strategy, selecting and updating subtrees daily based on real-time news distribution and retrieval feedback. Both variants are evaluated alongside existing production-grade recallers under identical online constraints, sharing the same downstream filtering and ranking pipeline, and are measured using the platform survival rate, defined as the fraction of retrieved news that passes downstream quality control.

4.3.0.3 Results and Analysis.

Table 4 reports online A/B testing results from Jan. 28 to Feb. 6, 2026 measured by survival rate. Across all days, Port B, which performs daily online subtree selection, consistently outperforms Port A with a fixed offline-selected subtree, indicating that subtree effectiveness varies over time and static configurations are insufficient under evolving news distributions. Port B also achieves the best performance among all production recallers on every reported day, often with a clear margin, demonstrating the benefit of dynamic, structure-aware adaptation over heterogeneous but static recall strategies.

Beyond controlled A/B evaluation, DynaTree has been fully deployed in real news topic channels and serves live user traffic. The same retrieval pipeline directly delivers content to subscribed users, so the observed online gains reflect not only system-level improvements but also tangible user-facing benefits in content coverage, freshness, and relevance. In practice, user interests are organized into a set of persistent topic channels, where each tree is constructed once and reused online, so offline cost scales with maintained channels rather than raw query volume. For unseen or cold-start topics, the system falls back to existing low-latency production recallers while queuing offline tree construction for future reuse. These results provide strong evidence of the practical effectiveness and deployability of DynaTree in real-world news retrieval systems.

4pt

5 Conclusion↩︎

We propose DynaTree, a two-stage agentic retrieval framework for effective and efficient news retrieval under evolving temporal distributions. By materializing semantic expansion as a reusable retrieval tree and decoupling its construction from online optimization, DynaTree amortizes agentic reasoning cost while enabling lightweight daily adaptation via dynamic subtree selection, guided by a compact evaluation proxy. Extensive offline experiments and direct online deployment in the Syft production system demonstrate consistent improvements over strong RAG and agentic baselines, showing that persistent semantic structures can maintain robust recall and ranking quality over time. These results highlight the practical value of structure-aware, agentic retrieval for large-scale, real-world news systems and its potential to deliver more reliable retrieval experiences to users.

6 Algorithmic Description↩︎

Algorithm 6 presents the complete algorithmic description of the proposed DynaTree framework, including agentic retrieval tree construction and dynamic adaptive subtree selection.

7 Syft News Dataset↩︎

We evaluate our framework on the Syft News dataset, which consists of multiple daily snapshots of real-world news streams. Table 5 summarizes basic statistics of the dataset across seven representative dates.

8 Production Recallers in the Syft Online Pipeline↩︎

Syft deploys a large-scale online retrieval pipeline where multiple production-grade recallers jointly generate candidate news articles. We briefly describe each recaller below.

8.0.0.1 Relation-based Recall (RelRec).

RelRec retrieves news articles through explicit relational signals, such as entity co-occurrence, knowledge-graph relations, or historical association patterns, favoring structurally related content.

8.0.0.2 Decompositional Recall (Decomp).

Decomp decomposes complex queries into simpler sub-queries and retrieves candidates for each component, aiming to improve coverage for multi-faceted information needs.

8.0.0.3 Direct Semantic Recall (SemRec).

SemRec performs direct dense retrieval using semantic embeddings to match queries and documents based on vector similarity in a shared embedding space.

8.0.0.4 Collaborative Proxy Recall (Proxy).

Proxy recall retrieves candidates via a set of representative proxy queries or documents, which are collaboratively selected to approximate diverse user intents and semantic variations.

8.0.0.5 Web Search Recall (WSRec).

WSRec leverages external web search engines as an auxiliary recall source, introducing high-quality candidates from a web-scale retrieval system. This component complements in-corpus retrieval by surfacing authoritative and up-to-date content that may not be well covered by the internal news index alone.

9 Construction Cost and Scalability Analysis↩︎

We further report the absolute construction cost of DynaTree and compare it with representative agentic retrieval baselines under the same topic-level setting. As shown in Table 6, DynaTree incurs higher absolute token and time cost because it constructs a reusable retrieval tree rather than a flat set of semantic expansions. However, its token cost normalized by each generated expansion remains within the same order of magnitude as agentic baselines. Moreover, each DynaTree expansion corresponds to a complete root-to-leaf semantic path, providing richer and reusable subtopic coverage.

The overall cost scales linearly with the number of persistent topic channels. Since Stage I uses fixed depth and node-count budgets, the per-topic construction cost is bounded and amortized across days. Stage II further avoids online tree reconstruction and only performs lightweight selection over a constant-size proxy set.

10 Shapley-value Attribution Details↩︎

5pt

This appendix details the formulation and evaluation protocol for the Shapley-based attribution analysis used in Section 4.2.2.1. We consider a set of agentic components \(\mathcal{A} = \{P, R, A, F\}\), corresponding to the Planning, Retrieval, Augmentation, and Reflection modules in the proposed framework. For any subset \(S \subseteq \mathcal{A}\), let \(f(S)\) denote the retrieval performance (Recall@100 or NDCG@10) obtained by enabling exactly the components in \(S\).

To capture interaction effects among components, we exhaustively evaluate all \(2^{|\mathcal{A}|} = 16\) possible component combinations under identical retrieval depth, semantic expansion budget, and evaluation protocol. All results are averaged over seven evaluation days. Following the standard Shapley-value formulation [35], the contribution of component \(i \in \mathcal{A}\) is defined as its expected marginal contribution over all possible coalitions: \[\phi_i = \sum_{S \subseteq \mathcal{A} \setminus \{i\}} \frac{|S|!\, (|\mathcal{A}| - |S| - 1)!}{|\mathcal{A}|!} \big[ f(S \cup \{i\}) - f(S) \big].\] Equivalently, the Shapley value can be interpreted as the expected marginal gain of component \(i\) under a uniformly random ordering of components, yielding a symmetric and interaction-aware attribution.

Shapley values are computed independently for Recall@100 and NDCG@10 using the averaged performance \(f(S)\). They reflect the average contribution of each component under realistic system interactions, rather than additive or causal importance. We emphasize that Shapley-based attribution does not assume component independence and should not be interpreted as isolating the intrinsic quality of a module; instead, it provides a principled summary of how each component contributes to overall system performance in the presence of strong interdependencies. Table 7 reports the complete results of all \(2^4 = 16\) configurations, which constitute the full experimental basis for the attribution analysis.

11 Uniform vs. Weighted Aggregation↩︎

Table 8 reports the full seven-day comparison between uniform and softmax-weighted path aggregation. Uniform aggregation consistently provides slightly better or comparable performance, supporting our choice of a simple averaging strategy in the main retrieval scoring function.

12 Case Study: Robustness to Emerging Entities↩︎

Although the retrieval tree is fixed after Stage I, DynaTree does not rely on exact lexical overlap with future news entities. We build the tree on the June 29 snapshot and inspect retrieved articles on June 30. As shown in Table 9, DynaTree retrieves articles containing entities absent from the original tree because higher-level semantic paths still capture their topical context. This indicates that the tree acts as a reusable semantic routing structure while document-level retrieval remains open to new surface realizations. For disruptive zero-day topics, the system falls back to existing low-latency recallers and asynchronously applies local Add/Revise operations via Reflection Agent, avoiding full online reconstruction.

References↩︎