1 Introduction↩︎

Large Language Model (LLM) search agents (or search agents for short) [1], [2] have shown strong promise in addressing complex information needs that may require reasoning, query decomposition, and/or information synthesis from multiple sources. These agents benefit from multiple interactions with a retrieval model to obtain required information for performing their knowledge-intensive tasks. In case of unstructured or semi-structured text corpora, these interactions are in the form of keyword or natural language queries. These approaches rely on decades of research in developing retrieval models, from lexical matching [3]–[5] to semantic matching based on dense representation [6], [7] or sparse representations [8], [9]. These models operate on pre-computed representations of documents to construct an index for the corpus. Relevance scores are also computed per document.¹

This paper explores a fresh perspective to this problem in which information seeking on unstructured data can be done at any granularity. In other words, instead of document-level representations, indexing, and relevance scoring, a piece of text of any size can be retrieved for each query. This enables us to perform more “surgical” information retrieval as opposed to being restricted with pre-determined text chunks and representations. To achieve this, we envision search agents that treat the corpus as an environment. Under this view, the agent can issue executable (surgical) search operations over the corpus, inspect intermediate results, refine constraints, and compose evidence across multiple steps. This shifts retrieval from a black-box ranking procedure to an explicit sequence of controllable corpus operations. Such an interface is especially appealing for many knowledge-intensive reasoning tasks in which answering a question may require exact entity matching, lexical filtering, symbolic pattern search, or following bridge entities across documents. This perspective is closely related to recent progress in code agents, where executable search tools such as grep and ripgrep provide a simple yet effective interface for locating relevant context in code repositories [10]. Inspired by this form of tool-mediated search, we ask whether a similar interaction pattern can be extended beyond code repositories to open-domain question answering over large textual corpora containing millions of unstructured documents.

Contemporary to our work, [11] and [12] independently propose agents to bypass pre-computed retrieval indices and searches a raw corpus through Unix-style shell commands, such as keyword matching or executable text-processing programs. These works demonstrate that direct corpus access can serve as an effective interface for exact matching, multi-step evidence discovery, and compositional question answering. These methods are primarily built around prompting large proprietary models with strong code-generation capabilities to orchestrate search at inference time. For example, [12] rely on closed-weight agents such as Claude,² making the resulting system computationally expensive and operationally inefficient, often requiring substantial time, sometimes even one hour or more, to complete a single query. In contrast, we are interested in methods that are feasible in the real world, thus focusing on training compact models and efficient operation executions at large scale. In order to be consistent with these contemporary work, we also refer to this category of approaches as Direct Corpus Interaction (DCI).

To achieve our goals of effective, efficient, and practical direct corpus interactions, we introduce GrepSeek, an optimized DCI search agent that trains a compact LLM to search, filter, and compose evidence over large text corpora through executable shell commands. This shifts DCI from an inference-time prompting strategy with large proprietary models (as in [12]) to a learned capability of a smaller agent. Training such an agent is challenging: naively applying reinforcement learning (RL) often produces degenerate behavior, such as overly broad commands, excessive context retrieval, or unstable search behavior. To stabilize learning, we first create a cold-start dataset that demonstrates successful DCI behavior. For each training question, an answer-aware Tutor is given the ground-truth answer and constructs a backward chain of shell commands whose execution retrieves corpus documents supporting the answer. This backward construction is particularly useful for complex and multi-hop questions, as it lets the Tutor identify supporting evidence one hop at a time while maintaining an explicit chain from the final answer back to the original question. We then convert the verified backward chain into a forward, causally valid trajectory using an answer-blind Planner. The Planner generates reasoning traces and commands from the agent’s observable history, simulating how the agent would solve the task at inference time. The Tutor then aligns these steps with the verified commands and evidence from the backward chain. This produces trajectories that remain causally grounded in the information observed so far, while allowing the Tutor to guide the Planner toward the verified search path. Finally, we refine the initialized policy using Group Relative Policy Optimization (GRPO) [13], allowing the agent to further improve its task-oriented search behavior through direct interaction with the corpus.

For a direct corpus interaction agent to be practical in the real world, retrieval latency must remain manageable even when operating over corpora containing millions of documents. However, executing standard shell pipelines sequentially over large multi-gigabyte text collections introduces substantial I/O and processing bottlenecks, making naive execution prohibitively slow for interactive agents. To address this, we develop a semantics-preserving sharded-parallel execution engine that dynamically distributes compatible shell pipelines across parallel corpus shards. This substantially reduces retrieval latency while preserving byte-exact equivalence with standard sequential execution.

To evaluate GrepSeek, we conduct experiments across seven knowledge-intensive question answering benchmarks spanning both single- and multi-hop questions. The single-hop benchmarks include Natural Questions (NQ) [14], TriviaQA [15], and PopQA [16]. The multi-hop benchmarks include HotpotQA [17], 2WikiMultihopQA [18], MuSiQue [19], and Bamboogle [20], all of which require iterative evidence aggregation and compositional reasoning across multiple documents. Our experiments show that GrepSeek substantially outperforms standard index-based RAG systems, untrained agentic frameworks, and even search agents optimized with RL to retrieve using dense and sparse retrievers. In particular, GrepSeek achieves the best token-level \(F_1\) performance on four out of seven benchmarks—NQ, HotpotQA, 2WikiMultihopQA, and MuSiQue—with statistically significant improvements on several datasets.

The gains are especially pronounced on multi-hop reasoning tasks, where traditional retrieval systems frequently suffer from semantic conflation and entity ambiguity introduced by retrievers. In contrast, by explicitly executing exact string-matching shell pipelines (e.g., rg -F), GrepSeek preserves fine-grained lexical distinctions and can isolate rare symbolic patterns, exact entity names, and intermediate bridge entities required for compositional reasoning. Although our approach exhibits minor degradation on datasets with substantial surface-form variation or semantically broad phrasing, GrepSeek ultimately achieves the strongest overall performance, establishing DCI as a highly competitive and practical alternative to search agents with index-based retrieval. To make DCI agents practical at scale, our semantics-preserving sharded-parallel execution engine accelerates shell-based retrieval by up to \(7.6\times\), reducing average search latency from \(5.39\) seconds under standard sequential execution to \(0.71\) seconds with sharded-parallel execution. Therefore, this brings down the average overall end-to-end latency of GrepSeek (including reasoning, multi-turn information seeking, and final answer generation) on all datasets to approximately \(8.6\) seconds per query on a single NVIDIA A100 GPU with 80GB VRAM, 32 CPU cores, and 32GB of system RAM. To support future research on direct corpus interaction agents, we release our codebase, training data, and model checkpoints.³

2 Optimizing Direct Corpus Interaction Search Agents↩︎

Contemporary to our work, and independently, [12] introduce Direct Corpus Interaction (DCI), where an agent bypasses pre-computed retrieval indices and searches a raw corpus through Unix-style shell commands, such as keyword matching and executable text-processing programs. However, their approach treats DCI primarily as an inference-time prompting strategy, relying on large proprietary models such as Claude with strong code-generation capabilities to orchestrate search. This makes the resulting system computationally expensive and operationally inefficient, often requiring substantial time—up to an hour—to answer a single query. In contrast, our work studies the challenges of optimizing smaller search agents to learn DCI as a trained capability, including unstable optimization on large corpora, overly broad command usage, and excessive context retrieval, enabling compact models to interact with the corpus and solve tasks through learned search behavior.

2.0.0.1 DCI Search Agent:

Figure 1 (Right) provides an overview of the DCI agent–corpus interaction, and Figure 2 illustrates a representative trajectory. DCI search agent \(\pi_{\theta}\) operates within the ReAct framework [21]. Given a question \(q\) and the system prompt shown in Figure 6 in Appendix 7, the agent interacts directly with a corpus \(\mathcal{C}\),⁴ where each line corresponds to a document. The interaction proceeds for at most \(T\) steps, producing a trajectory \(\tau = \{(t_i, a_i, o_i)\}_{i=1}^{T}\), where \(t_i\) denotes the reasoning trace, \(a_i\) the action, and \(o_i\) the resulting observation. At step \(i\), conditioned on the question \(q\) and the previous actions and observations \(\tau_{<i}\), the policy \(\pi_{\theta}\) generates a reasoning trace and an action: \((t_i, a_i) \sim \pi_{\theta}(\cdot \mid q, \tau_{<i})\). Reasoning traces are generated within <think> XML tags. Actions corresponding to tool invocations are emitted using the Hermes-style <tool_call> format, and the resulting tool outputs are returned to the model within <tool_response> tags. When the agent decides to terminate, it produces the final answer \(\hat{y}_q\) within <answer> tags. The action \(a_i\) is either a corpus interaction command⁵ (i.e., a shell command) or a termination that outputs the answer. Following each command, an execution engine runs the command over the corpus file \(\mathcal{C}\) and returns an observation \(o_i\), which is appended to the trajectory and used for subsequent reasoning and action generation. The remainder of this section describes the training and efficient tool execution.

2.1 Training DCI Search Agent↩︎

We observe that directly optimizing the agent to interact with corpus using RL leads to unstable behavior; the agent struggles to produce effective commands and frequently retrieves excessively large corpus segments, which increases context length and destabilizes optimization.⁶ To address this, we adopt a two-stage training. First, we automatically construct a cold-start dataset to improve the agent’s initial tool-use behavior in interaction with the corpus and impose behavioral constraints on interactions. The model is first supervised on this cold-start data before being optimized using RL.

2.1.1 Cold-Start Data Generation↩︎

Given a dataset \(D=\{(q_i, y_i)\}_{i=1}^{|D|}\) of question-answer pairs that require information from the corpus \(\mathcal{C}\), our data generation pipeline (Algorithm [alg:coldstart]) consists of two main phases followed by a quality filtering stage. The process relies on an answer-aware Tutor LLM (\(\mathcal{M}_{T}\)) to construct verified evidence chains, and an answer-blind Planner LLM (\(\mathcal{M}_{P}\)) to synthesize realistic forward reasoning trajectories.

2.1.1.1 Backward Phase:

The Tutor decomposes the query \(q\) and gold answer \(y\) into an ordered sequence of sub-queries (Algorithm [alg:coldstart], line [line:decompose]; prompt in Figure 8 in Appendix 7). To ensure that the agent learns genuine information-seeking behavior rather than exploiting access to the answer, we construct the retrieval trajectory in reverse (\(N \rightarrow 1\); lines [line:backward95loop]–[line:bridge]). At each backward step, the Tutor proposes a shell command \(c_i\) intended to retrieve a document \(d_i\) that entails the current target answer \(a\) (lines [line:discover] and [line:propose]). Crucially, we enforce a strict answer-leak rule during command generation (prompt in Figure 18 in Appendix 7): the proposed command must be target-masked, forbidding the use of the target entity \(a\) or any of its aliases \(F\) as retrieval terms. This is necessary because the backward process has access to future information through \(y\); without masking, it can retrieve supporting evidence by querying the answer, resulting in unrealistic retrieval behavior that does not reflect inference-time behavior.

To improve retrieval robustness, the Tutor is allowed up to \(M\) refinement in Discover procedure (lines [line:refine95loop]–[line:return95fail]). At each attempt, Tutor proposes a command, executes it, and verifies if the retrieved documents support the target answer using a verification step (line [line:exec95check]; prompt in Figure 9 in Appendix 7). This increases the likelihood of obtaining valid evidence while filtering out brittle or spurious retrieval trajectories. Once at least one valid document is identified, a bridge extraction step determines the antecedent entity in \(d_i\) that answers the preceding sub-query \(q_{i-1}\) (lines [line:if95bridge]–[line:bridge]; prompt in Figure 11 in Appendix 7). The extracted entity then becomes the target answer for the next backward hop. After completing all backward steps, it produces a multi-hop chain that connects the original query to the final answer through causally consistent intermediate shell command steps.

2.1.1.2 Forward Phase:

Upon successfully constructing a verified chain of documents and commands, the sequence is reversed into chronological order (Algorithm [alg:coldstart], line [line:reverse95init]) to simulate the information flow available to the agent during inference. Although the retrieval path is constructed backward for verification purposes, a deployed agent only observes past interactions and retrieved evidence when making decisions. Reversing the trajectory therefore ensures that training trajectories faithfully match the causal structure encountered at inference time. At each forward step, the answer-blind Planner drafts an initial reasoning trace \(\theta_d\) and action conditioned solely on the current causal history \(\mathcal{H}\) (line [line:draft]; prompt in Figure 12 in Appendix 7). Because the Planner does not have access to the verified evidence chain or future retrieval states, its proposed reasoning often lacks the precision necessary to justify the optimal shell command \(c_i\). To bridge this gap, the Tutor model performs a constrained alignment step that edits the Planner’s reasoning trace to logically motivate \(c_i\) while remaining strictly grounded in the observable interaction history (line [line:align]; prompt in Figure 15 in Appendix 7). The resulting trajectory combines the realism of forward causal reasoning with the reliability of backward-verified evidence construction.

2.1.1.3 Automatic Quality Filtering:

To ensure that the cold-start dataset provides a stable initialization for RL optimization, all assembled trajectories undergo rigorous filtering (Algorithm [alg:coldstart], lines [line:answer]–[line:judge95gate]). First, the Planner generates a final answer \(\hat{y}\) from the complete interaction history \(\mathcal{H}\) that achieves non-zero token-level overlap with the ground-truth answer \(y\) (\(\mathrm{F_1}(\hat{y}, y) > 0\)). This ensures that the constructed trajectory contains sufficient information for answer generation. Second, the formatted trajectory \(\mathcal{T}_{\mathrm{train}}\) is evaluated by the Tutor for causal and logical consistency (line [line:judge95gate]; prompt in Figure 16 in Appendix 7). The judge enforces strict temporal boundaries, discarding trajectories whose reasoning or retrieval commands implicitly reveal entities or facts not yet observable in the agent’s current history. This is necessary because the backward construction process has access to future information via the gold answer and verified evidence chain, and without explicit verification, subtle forms of future-state leakage may persist even under explicit answer masking. Examples of the generated data using our pipeline are shown in Appendix 10.

2.1.2 Optimization of DCI Search Agent↩︎

2.1.2.1 SFT on Synthetic Trajectories:

After constructing the cold-start data, we first perform supervised fine-tuning on them. Each training example consists of the full interaction sequence, including reasoning traces, tool invocations, tool responses, and the final answer. The objective of this stage is to initialize the agent with stable retrieval and reasoning behavior before RL. In particular, SFT teaches the agent to produce concise and causally grounded search commands and avoid pathological retrieval behavior such as excessively broad corpus scans.

2.1.2.2 Reinforcement Learning with GRPO:

Following the SFT stage, we further optimize the policy using GRPO [13].⁷ For each query \(q\), the policy \(\pi_\theta\) samples a group of \(n=5\) trajectories \(\tau^{(1)}, \dots, \tau^{(n)} \sim \pi_\theta(\cdot \mid q)\), where each trajectory consists of interleaved reasoning traces, tool invocations, tool responses, and a final answer prediction. Each sampled trajectory \(\tau^{(i)}\) receives an answer reward \(R_{\mathrm{ans}}(\tau^{(i)})\) based on token-level F\(_1\) [22] overlap between the predicted answer \(\hat{y}^{(i)}\) and the gold answer set \(\mathcal{Y}\). To enforce adherence to the required interaction protocol, we additionally define a binary format indicator \(\phi(\tau^{(i)}) \in \{0,1\}\) that verifies whether the trajectory satisfies the expected structural constraints, including properly formed <think>, <tool_call>, <tool_response>, and <answer> blocks. The final trajectory reward is therefore \(R(\tau^{(i)}) = \phi(\tau^{(i)})\, R_{\mathrm{ans}}(\tau^{(i)})\), so that only structurally valid trajectories receive non-zero learning signal. Details of the reward function are provided in Appendix 7.3. GRPO computes a relative advantage by normalizing rewards within each group: \[A^{(i)} = \frac{ R(\tau^{(i)}) - \mathrm{mean}(\{R(\tau^{(j)})\}_{j=1}^{n}) }{ \mathrm{std}(\{R(\tau^{(j)})\}_{j=1}^{n}) + \epsilon },\] which encourages trajectories that outperform other samples generated for the same query while reducing sensitivity to reward scale. The reward formulation primarily incentivizes accurate answer generation while implicitly favoring trajectories that yield successful evidence retrieval and corpus interaction behavior. Initializing RL from the SFT-trained policy improves optimization stability, as the policy already exhibits structured retrieval behavior and causally consistent reasoning prior to RL.

2.2 Efficient Corpus Interaction↩︎

Unlike RAG that retrieve from a pre-computed index, the DCI agent performs retrieval by executing shell commands over the corpus, which may contain millions of documents.⁸ The agent interacts with the corpus through Unix tools, including rg, grep, awk, sed, cut, sort, uniq, wc, head, and tail. Because a trajectory may involve multiple corpus-wide scans, efficient command execution is critical for inference throughput. A key design requirement of an efficient execution engine is that all optimizations remain semantics preserving: every command must produce output identical to execution over the original corpus. To make this practical, we employ a collection of semantics-preserving optimizations that substantially reduce retrieval latency using shell commands without altering the observations for the agent. The detailed implementation is provided in Appendix 7.2.

2.2.0.1 Sharded-Parallel Corpus Search:

To accelerate corpus interaction, we execute compatible shell pipelines in parallel across \(S\) line-aligned corpus shards while preserving byte-exact equivalence with sequential execution (Algorithm [alg:sharded-search] in Appendix 7.2). Given a shell pipeline \(c\) consisting of \(m\) stages connected via the pipe operator |, the engine first decomposes the pipeline into its constituent commands \((s_1, \dots, s_m)\) and dynamically classifies its reduction semantics to determine whether the pipeline can be safely parallelized or must fall back to sequential execution. The classification process conservatively guarantees correctness. If the initial command is not a valid search operator (\(s_1 \notin \{\texttt{rg}, \texttt{grep}\}\)) or if any stage depends on global or cross-line state (\(\exists s_j \;\text{s.t.}\; \mathrm{\small Unsafe}(s_j)\)), the pipeline is executed sequentially over the original corpus. Otherwise, the engine identifies pipelines composed entirely of shard-independent stateless transformations (e.g., cut, tr, and line-wise sed), which can be evaluated independently on each shard. For valid pipelines, execution proceeds independently across the \(S\) shards, producing partial outputs \(\{R_1,\dots,R_S\}\).

The final output is reconstructed using a strategy-specific reduction rule determined by the terminal stage in the pipeline, following the first applicable case: (1) purely stateless pipelines are merged through deterministic shard-order concatenation (\(\biguplus_i R_i\)); (2) purely stateless pipelines ending in head -n apply local top-\(N\) truncation on each shard to reduce memory usage, then concatenate the shard outputs, followed by a final top-\(N\) truncation; (3) purely stateless pipelines ending in count operations such as wc -l aggregate shard counts through scalar summation (\(\sum_i \mathrm{Int}(R_i)\)); (4) pipelines involving sort, optionally followed by uniq, and terminated by head -n, are merged using a deterministic \(k\)-way merge procedure [23] before the final top-\(N\) selection; and (5) any other pipeline is conservatively executed sequentially over the original corpus.

The engine supports arbitrary piped shell commands, allowing the agent to compose complex multi-stage retrieval programs during inference. By restricting shard-parallel execution only to pipelines whose outputs can be reconstructed exactly from shard-local computations, the system substantially improves retrieval throughput while remaining behaviorally identical to sequential execution.⁹

2.2.0.2 Persistent Search Daemon:

To further reduce latency, we keep the corpus in memory and execute retrieval commands through a persistent search daemon shared across the rollout. The daemon maintains long-lived search workers that avoid repeated process startup and corpus loading across successive tool calls, which is important because a single trajectory may involve many retrieval operations. Commands are executed using memory-mapped search primitives, and in practice most generated queries correspond to simple fixed-string filtering operations implemented with rg. As a result, retrieval performance is primarily limited by memory bandwidth and data access patterns rather than by the computation performed by the search operators themselves.¹⁰

3 Experiments↩︎

3.1 Experimental Setup↩︎

3.1.0.1 Datasets & Evaluation:

Following prior work [2], we evaluate on seven benchmark datasets: three single-hop datasets—NaturalQuestions (NQ) [14], TriviaQA [15], and PopQA [16]—and four multi-hop datasets—HotpotQA [17], 2WikiMultihopQA (2Wiki) [18], MuSiQue [19], and Bamboogle [20].¹¹ Unless otherwise specified, we report results on the official test splits; otherwise, we use the development sets when test labels are not available. For training, we use only the training sets of NQ and HotpotQA, and evaluate generalization on the remaining datasets as out-of-distribution test sets. Dataset statistics are reported in Table 2 in Appendix 6. We use the 2018 Wikipedia dump [7] of 21M documents as the corpus.¹² For evaluation, we use token-level F\(_1\) as the primary metric, as it captures partial correctness under surface-form variation, and report exact match (EM) in the appendix for completeness [22].

3.1.0.2 Training & Inference Settings:

We use Qwen3.5-9B¹³ [24] as the LLM. To train GrepSeek in SFT stage, we construct a 10k-sample cold-start SFT dataset with a balanced mixture of HotpotQA and NQ. As the Tutor and Planner, we use Qwen3.5-27B¹⁴, with a maximum of \(M=5\) refinements. The model is trained for one epoch on this dataset. Following the SFT, the policy is optimized using GRPO [13] for 200 steps with a group size of \(n=5\) on the full HotpotQA and NQ datasets. During inference, the agent uses nucleus sampling [25] with temperature \(0.6\), a maximum of \(T=6\) turns, and a context length of 16,384 tokens to support multi-turn corpus interaction. A complete list of hyperparameters and system configurations for the SFT, GRPO, and inference phases is provided in Tables 3, 4, and 5 in Appendix 7.4.

3.1.0.3 Baselines:

Following prior work [2], we compare against a range of baselines, including (1) a direct LLM inference setting without external search, and (2) retrieval-augmented methods: RAG [26], IRCoT [27], Search-O1 [1], rejection sampling with a search engine and Search-R1 [2] (GRPO-optimized).¹⁵ All baselines use the same backbone LLM and are trained (when applicable) and evaluated under the same settings as our method for a fair comparison (Appendix 7.4). We use three retrievers to retrieve top-3 documents from the corpus: BM25 [4] as a sparse lexical baseline, E5 [31] as a dense embedding (110M parameters),¹⁶ and a large-scale Qwen3 embedding model [32] (4B parameters).¹⁷ This allows us to systematically assess performance across both traditional and strong neural retrievers. Dense retrievers are implemented using FAISS¹⁸ [33] with HNSW index [34] (\(M=32\), \(\text{efConstruction}=128\), \(\text{efSearch}=128\)) for fast retrieval over the vector database.

3.2 Main Findings↩︎

3.2.0.1 Comparison of Performance with Baselines:

We compare GrepSeek against a range of retrieval-augmented agentic search baselines, with results reported in Table [tab:f1-result]. Overall, GrepSeek substantially outperforms non-agentic approaches (Direct and standard RAG), untrained agentic methods (IRCoT and Search-O1), and trained agentic baselines (Rejection Sampling), regardless of the underlying sparse or dense retriever. Among baselines, Search-R1 is the strongest competitor due to its reinforcement learning optimization. Nevertheless, GrepSeek achieves the best performance on 4 out of the 7 benchmarks (NQ, HotpotQA, 2Wiki, and MuSiQue), with statistically significant improvements on 3 (\(^\uparrow\)). Notably, the largest gains are observed on multi-hop reasoning benchmarks, where GrepSeek in most cases outperforms dense retrieval baselines. This suggests that direct corpus interaction is particularly effective for iterative evidence aggregation and maintaining strict entity precision across reasoning steps—for instance, correctly distinguishing a specific subsidiary from a parent company (Example [ex:hotpot-rockstar] in Appendix 9), or avoiding cascading name-collision errors common to dense retrievers (Example [ex:hotpot-faithlutheran] in Appendix 9). While performance decreases slightly on TriviaQA and Bamboogle, the observed differences are not statistically significant. The only statistically significant drop of our method compared to the baseline occurs on the PopQA dataset (\(^\downarrow\)).

These performance trade-offs are closely tied to the retrieval behavior of GrepSeek. Since the agent directly interacts with the raw text corpus through shell-based retrieval (e.g., rg), its search is primarily driven by explicit lexical constraints and iterative filtering operations. This surgical strategy is highly effective for compositional reasoning and queries containing strong textual anchors, such as rare chemical formulas (Example [ex:hotpot-formula] in Appendix 9), distinctive phrasing (Example [ex:hotpot-joggers] in Appendix 9), and exact full-name matches (Example [ex:popqa-arndt] in Appendix 9), which frequently confound semantic embeddings. However, datasets with limited lexical overlap or intentionally ambiguous phrasing present a greater challenge. For example, PopQA focuses on long-tail entities where our agent’s reliance on exact string matching makes it brittle to surface-form variations and diacritics (e.g., missing an entity entirely due to an unexpected accent mark, as seen in Example [ex:popqa-vaillant] in Appendix 9). Furthermore, because rg lacks semantic relevance ranking, the agent can struggle when target keywords are heavily overloaded, occasionally burying the most authoritative document in favor of chronologically earlier matches (Example [ex:bamboogle-citibank] in Appendix 9). In such settings, dense retrievers hold a distinct advantage by mapping lexical variations to shared embedding spaces. Despite these limitations on the semantic tail, GrepSeek achieves the strongest overall micro-average score (\(0.5691\)), significantly outperforming the best dense retrieval baseline (\(p < 0.05\)). These results indicate that while direct corpus interaction may occasionally falter on heavily semantic queries, it provides a highly precise, scalable, and effective alternative to dense retrieval systems for general-purpose open-domain question answering and complex multi-hop reasoning.

3.2.0.2 Comparison of Latency with Best-Performing Baselines:

Based on the results in Table [tab:f1-result], the Search-R1 variants using E5 and Qwen3 embeddings emerge as the strongest dense retrieval baselines. To analyze the efficiency trade-offs of GrepSeek relative to these systems, we compare inference latency, runtime retrieval memory footprint, and offline indexing cost. The results are shown in Figure 3 (a–c). All efficiency experiments are conducted on a machine with 32 CPU cores (on a set of 50 examples from each dataset, total of 350 examples), while dense retriever indexing is performed using a single NVIDIA A100 GPU. As shown in Figure 3a, GrepSeek has a higher end-to-end inference latency per query (\(8.67\) s) compared to E5 (\(4.77\) s) and Qwen3-4B (\(6.07\) s), primarily due to longer reasoning trajectories and increased LLM decoding time (\(7.86\) s). However, the optimized execution engine keeps the actual retrieval execution cost low, requiring only \(0.81\) s for tool interaction. Despite this latency overhead, GrepSeek provides substantial efficiency advantages in memory and preprocessing cost. As shown in Figure 3b, GrepSeek requires only \(14\) GB of host memory, corresponding directly to the raw corpus size, whereas dense retrieval systems require substantially larger memory footprints to store embeddings and indexing structures (\(70\) GB for E5 and \(221\) GB for Qwen3-4B). Moreover, Figure 3c shows that GrepSeek completely eliminates offline embedding precomputation, requiring only approximately \(1\) minute of setup time, compared to \(3.2\) and \(62.4\) A100-hours for E5 and Qwen3-4B, respectively.

We additionally study how the optimized execution engine scales with increasing shard parallelism. Figure 3d reports command execution latency as the number of corpus shards increases from \(S \in \{1,2,4,8,16,32\}\). The results show near-linear speedups at smaller shard counts, reducing latency from \(5.39\) s at a single shard to \(1.22\) s at \(8\) shards. Increasing the shard count further continues to improve performance, reaching \(0.71\) s at \(32\) shards, although gains gradually plateau at larger values of \(S\). This is expected, as execution becomes bottlenecked by hardware constraints such as memory bandwidth saturation, process scheduling overhead, and the cost of merging shard-local outputs.

3.2.0.3 Ablations of GrepSeek:

To study the contribution of each training stage, we perform an ablation analysis by comparing GrepSeek against variants without SFT or without RL optimization. Results are reported in Table 1 for F\(_1\) and Table 6 in Appendix 8 for EM.¹⁹ The results show that GrepSeek significantly outperforms both ablated variants across all datasets, demonstrating that both the synthetic cold-start SFT stage and the subsequent RL optimization are critical for strong retrieval and reasoning performance. In particular, removing RL substantially degrades multi-hop reasoning performance, while removing SFT leads to severe instability and the largest overall performance drop, highlighting the importance of structured trajectory initialization before reinforcement learning.

To further study the effect of cold-start data scale on downstream RL performance, we evaluate policies initialized with varying amounts of trajectories: 0 (base model), 2.5k, 5k, and 10k. Each initialization is subsequently optimized using the same GRPO configuration. The resulting token-level F\(_1\) scores are shown in Figure 4, while the corresponding EM results are provided in Figure 17 in Appendix 8. The results show that even a relatively small supervised initialization of 2.5k trajectories substantially improves performance over the untuned base model across all benchmarks, highlighting the importance of inducing command-generation for retrieval behavior prior to RL optimization. Increasing the dataset size to 5k and 10k trajectories further improves performance, although gains become progressively smaller, with the micro-average beginning to plateau beyond 5k examples.

3.2.0.4 Training Dynamics:

Figure 5 shows the training dynamics over 200 GRPO steps. As shown in Figure 5a, GrepSeek achieves higher average rewards throughout training compared to all Search-R1 baseline variants using dense or sparse retrievers (E5, BM25, and Qwen3-Emb-4B). This improvement, however, comes with increased computational cost. Figure 5b shows that GrepSeek generates longer sequences, due to both extended reasoning traces and the inclusion of raw retrieved corpus context, resulting in lower inference throughput as explained earlier. Interestingly, the retrieval behavior of GrepSeek evolves differently from retrieval-based baselines. As shown in Figure 5c, baselines tend to increase the number of retrieval queries during training before eventually stabilizing. In contrast, GrepSeek gradually reduces the number of executed search commands over time. We observe that the agent initially relies on multiple independent retrieval operations, but progressively learns to compose more expressive multi-stage shell pipelines by chaining operators through piping, allowing more information to be extracted per command invocation.

3.3 Analysis↩︎

3.3.0.1 Retrieval Behavior:

R0.53

2.5pt max width=

Because the DCI agent interacts with the corpus through shell rather than retrievers, its retrieval strategy is inherently interpretable. An analysis of the generated commands reveals a highly structured and selective policy. Across all evaluation benchmarks, the agent consistently limits output verbosity by using | head in all invocations, and relies on exact-string matching (-F in almost all cases), avoiding unintended regular-expression generalization. In addition, a large fraction of commands (approximately 70%) employ cascaded filtering (e.g., rg … | rg …) to iteratively narrow the search space. The agent also adapts its search effort to task difficulty, issuing more retrieval commands on multi-hop datasets (2.6–3.4 on HotpotQA, MuSiQue, and 2WikiMultihopQA) than on single-hop datasets (2.0–2.4 on NQ, TriviaQA, and PopQA).

To disentangle behaviors induced by cold-start SFT from those learned during RL, we track trajectory statistics across training steps (Table [tab:cmd-evo]). We observe that low-level syntactic properties of the generated pipelines—such as pipe depth, use of fixed-string matching, truncation patterns, and cascaded filtering—remain largely stable throughout RL training, indicating that these structural retrieval “primitives” are established during SFT. In contrast, RL primarily shapes higher-level search behavior. As training progresses, the agent reduces the number of commands per trajectory (from \(3.06\) to \(2.56\)), increases the amount of context extracted per query (e.g., head -n growing from approximately 5 to 10 lines), and allocates substantially more tokens to reasoning (\(4{,}251 \rightarrow 6{,}409\)). Overall, RL refines an already structured retrieval interface by improving efficiency and encouraging more reasoning, while preserving the underlying interaction patterns established during SFT.

3.3.0.2 Case Studies:

To qualitatively analyze the operational characteristics of GrepSeek, we compare its trajectories against the strongest dense retrieval baseline (Search-R1 with Qwen3-Emb-4B) across our benchmark suite (see detailed transcripts in Appendix 9). The empirical traces show that direct execution of shell pipelines over raw text enables a high degree of lexical precision. In particular, GrepSeek can isolate rare symbolic patterns such as chemical formulas (Example [ex:hotpot-formula]) and exact entity names (Example [ex:popqa-arndt]) in a single rg -F query, whereas dense retrievers may merge closely related entities due to embedding-level smoothing. The agent also performs effective multi-hop evidence linking via explicit keyword composition (Example [ex:hotpot-joggers]) and reliably resolves entity collisions, such as distinguishing subsidiaries from parent organizations (Examples [ex:hotpot-rockstar] and [ex:hotpot-faithlutheran]). At the same time, the analysis highlights inherent limitations of lexical search. Because standard Unix tools do not perform semantic ranking and instead return matches in file order, relevant evidence can sometimes be preceded by irrelevant or less informative passages (Example [ex:bamboogle-citibank]). Moreover, reliance on exact string matching introduces brittleness to surface-form variation: small spelling differences or missing diacritics (Example [ex:popqa-vaillant]) can prevent retrieval of relevant content, forcing reliance on downstream reasoning or parametric knowledge. In such cases, dense retrieval methods can be more robust due to their ability to generalize across lexical variations in embedding space.

4 Related Work↩︎

4.0.0.1 Retrieval-Augmented Agentic Search:

Knowledge-intensive question answering is challenging for LLMs because many questions require facts that may be missing, outdated, or unreliable in the model’s parametric memory [16]. Access to external knowledge is central to improving factuality and coverage. Retrieval-Augmented Generation (RAG) [26] addresses this need by retrieving relevant evidence from an external corpus and conditioning generation on the retrieved context. More broadly, this can be viewed as an instance of retrieval-enhanced machine learning [35]. In this paradigm, the retrieval module determines what information is exposed to the model, typically through an index-based interface such as sparse lexical retrieval with BM25 [4] or dense retrieval with embedding models such as E5 [31] and Qwen3-Embedding [32]. This retrieval step gives the model access to non-parametric knowledge and has become a standard approach for knowledge-intensive tasks.

However, a single retrieval step is often insufficient for complex questions. Many information needs require intermediate reasoning: the system may need to identify an entity, use that entity to form a follow-up query, retrieve additional evidence, and then compose information across documents. This motivates retrieval-augmented reasoning methods, where retrieval is not merely a preprocessing step but part of a multi-turn reasoning process. Such multi-turn behavior is important for both unstructured corpora and structured settings, where systems may need to inspect intermediate results, refine constraints, and issue follow-up operations over databases or tables, as in STARQA [36]. IRCoT [27] interleaves chain-of-thought reasoning with retrieval, while more recent search-agent systems extend this idea to longer-horizon tool-use trajectories, including Search-R1 [2] and Search-O1 [1].

Recent agentic search methods differ in which parts of the pipeline are optimized. Some systems treat both the LLM and the retriever or search engine as black boxes, relying on prompting or inference-time orchestration to decide when and how to search, as in Search-O1 [1]. Other methods train the language model to issue better search queries and reason over retrieved evidence while keeping the underlying retriever fixed, as in Search-R1 [2]. A third line of work jointly optimizes the reasoning policy and the retrieval or ranking component, as in CoSearch [37]. These approaches primarily differ in how they improve reasoning, planning, or retrieval over a conventional search interface. Our work studies a complementary direction: instead of training an agent to better use a fixed retriever, we train a compact open-weight model to interact directly with the corpus through deterministic shell-based search operations.

Question answering is widely used to evaluate retrieval-augmented and agentic search systems because it directly tests whether a system can retrieve sufficient evidence and synthesize the correct answer. Multi-hop QA benchmarks are especially relevant because they require multi-turn search behavior: a model must often retrieve one piece of evidence, use it to identify a new information need, and then combine evidence across steps. Following prior search-agent work such as Search-R1 [2], we evaluate on both single-hop and multi-hop QA benchmarks. We acknowledge that broader deep-search benchmarks, such as BrowseComp [38] and Total Recall QA [39], also evaluate important aspects of long-horizon information seeking. We leave evaluation on these broader deep-research settings to future work.

4.0.0.2 Direct Interaction with Corpus:

Direct Corpus Interaction (DCI) provides a different way to connect language models with external information. Instead of relying on a retriever to rank passages, the agent issues explicit operations over the raw corpus and controls how evidence is matched, filtered, and composed. Prior work has studied direct textual search in code and repository settings [10], [40], as well as recent DCI-style agents for open-domain retrieval over large-scale corpora [11], [12], [41]. This direction is also related to systems work on efficient string and regular-expression search over large or compressed text collections, including Succinct and Swift [42], [43]. We view these systems primarily as efficiency-oriented predecessors rather than agentic-search baselines: they improve the execution substrate for direct search, while our focus is on whether an LLM can learn when and how to use direct corpus operations as part of multi-step reasoning.

5 Conclusion & Future Work↩︎

We introduced GrepSeek, a Direct Corpus Interaction (DCI) search agent that bypasses traditional pre-computed search indexes by operating directly over raw text corpora using standard Unix shell commands. Through a two-stage training pipeline—consisting of synthetically generated cold-start SFT followed by RL with GRPO—we demonstrated that search agents can learn to execute highly effective, interpretable, and lexically precise retrieval programs. GrepSeek achieves strong performance on challenging multi-hop reasoning benchmarks by precisely isolating symbolic patterns and enforcing strict entity-level constraints, succeeding in scenarios where dense embedding-based models often fail due to semantic conflation. In addition, our optimized sharded-parallel execution engine substantially reduces runtime memory requirements and eliminates the expensive offline indexing stage required by dense retrieval systems. Despite these advantages, our analysis also highlighted the limitations of purely lexical retrieval, including sensitivity to surface-form variation (e.g., diacritics) and the absence of semantic relevance ranking.

Future work will explore several directions to address these issues. First, we plan to investigate hybrid retrieval architectures that combine direct corpus interaction with index-based retrieval models. Second, we aim to enhance the expressiveness and robustness of the shell-based interface by incorporating richer matching primitives, including fuzzy matching and more advanced regular-expression operators. Finally, we will focus on improving inference efficiency by reducing decoding overhead from long reasoning traces, through techniques such as more compact trajectory generation and improved context management, enabling more efficient deployment in high-throughput settings. We further plan to expand our evaluation to long-form question answering, adhoc document retrieval, and retrieval from unseen corpora.

Acknowledgments↩︎

This work was supported in part by the Center for Intelligent Information Retrieval, in part by the Office of Naval Research contract #N000142412612, in part by the National Science Foundation grant #2402873 and #2402874, and with support from Google.org. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the sponsors.

6 Datasets↩︎

Following prior work [2], we evaluate our method on a comprehensive suite of seven knowledge-intensive benchmark datasets. These datasets are carefully selected to evaluate both single-step fact retrieval and complex, multi-step reasoning. To standardize the formatting and evaluation protocol, all datasets are obtained from the FlashRAG repository.²⁰

6.1 Evaluation Benchmarks↩︎

The evaluation suite is divided into single-hop and multi-hop datasets to isolate the agent’s ability to perform targeted retrieval versus iterative corpus exploration.

6.1.0.1 Single-Hop Datasets

These tasks require retrieving a single, highly relevant fact or document to answer the user’s query:

• Natural Questions (NQ) [14]: A dataset of real user queries issued to Google Search. We use the open-domain split, which requires systems to retrieve relevant Wikipedia passages to answer questions formulated by users without prior knowledge of the target.

• TriviaQA [15]: A collection of complex trivia questions authored by trivia enthusiasts. While the questions often contain compositional linguistic structures, the answers can typically be derived from a single retrieved document.

• PopQA [16]: An entity-centric QA dataset specifically designed to probe long-tail knowledge. The dataset is constructed from Wikidata triples and focuses on rare entities where parametric knowledge in LLMs typically fails, strictly necessitating accurate external retrieval.

6.1.0.2 Multi-Hop Datasets

These tasks require the agent to execute multiple interdependent search queries, gathering partial information to inform subsequent retrieval steps:

• HotpotQA [17]: A dataset requiring reasoning across at least two distinct Wikipedia articles. Questions are specifically designed to require information from multiple sources to synthesize and produce a correct the final answer.

• 2WikiMultihopQA (2Wiki) [18]: Constructed using Wikidata properties to generate compositional questions. This dataset introduces explicit logical structures to the multi-hop reasoning process, requiring the agent to follow strict reasoning chains across multiple documents.

• MuSiQue [19]: A rigorous multi-hop QA dataset designed to minimize “shortcut” reasoning. The questions are composed by chaining multiple single-hop questions, heavily filtered to ensure that models cannot guess the answer through lexical overlap or single-document retrieval.

• Bamboogle [20]: A smaller but highly challenging dataset consisting of questions manually authored to defeat standard search engines. It requires deep, multi-step evidence gathering that cannot be resolved using surface-level web snippets or simple entity linking.

None

We use the 2018 Wikipedia dump [7] of 21M documents as the corpus.²¹

6.2 Data Splits and Training Protocol↩︎

To evaluate the generalization capabilities of our approach, we employ a strict split between in-distribution training datasets and out-of-distribution evaluation datasets. For the training phase, the agent is trained exclusively on a combined dataset consisting of the training splits of NQ (\(79,168\) examples) and HotpotQA (\(90,447\) examples). This provides the agent with exposure to both fundamental single-hop retrieval dynamics and complex multi-hop reasoning strategies, totaling \(169,615\) training examples. During inference, we evaluate the system across all seven datasets to test both held-out in-domain performance and generalization to unseen datasets. We utilize the official test splits for datasets where they are publicly available. The final evaluation encompasses \(51,713\) total queries across the seven benchmarks. Full dataset sizes and split statistics are detailed in Table 2.

6.3 Evaluation Metrics↩︎

To assess the performance of the agent across all benchmark datasets, we evaluate the generated responses using standard metrics for open-domain question answering: Exact Match (EM) and token-level F\(_1\) score [22].

• Exact Match (EM): This measures the percentage of predictions that match the gold answer exactly. It serves as a measure of final answer correctness. Prior to comparison, both the predicted and ground truth answers undergo a standard normalization procedure, which includes lowercasing, punctuation removal, and the stripping of definite and indefinite articles (e.g., “a”, “an”, “the”).

• F\(_1\) Score: To provide a more granular measure of partial correctness and answer overlap, we compute the token-level F\(_1\) score. This calculates the harmonic mean of precision and recall over the individual tokens present in the predicted and reference answers, applying the same text normalization steps as EM. For examples where the dataset provides multiple reference answers for a single query, we compute the score against all references and report the maximum F\(_1\) score.

In the main body of the paper, we report F\(_1\), while EM is additionally provided in the appendix.

None

7 GrepSeek’s Implementation Details↩︎

This section provides implementation details and settings used for GrepSeek.

7.1 Prompts↩︎

The main system prompt for the DCI agent is shown in Figure 6. The prompts described in this section collectively define the instructional framework used to generate synthetic cold-start training trajectories. All agent roles share a common system prompt that specifies the corpus structure, interaction format, and permissible shell tools (Figure 7).

None

In Phase A, the pipeline employs an Answer-Aware Tutor to decompose multi-hop questions into ordered single-hop sub-queries (prompt in Figure 8). Guided by system instructions that enforce the anti-leak constraint (prompt in Figure 18), the Tutor iteratively constructs a backward retrieval trajectory through an initial command proposal stage (prompt in Figure 19) followed by targeted refinement steps (prompt in Figure 10). Each retrieved document is validated using a per-hop entailment judge (Figure 9), after which a dedicated bridge extraction prompt identifies the intermediate entity required to connect the current retrieval step to the preceding sub-query (prompt in Figure 11).

None

None

After constructing a verified retrieval path, Phase B transitions to an Answer-Blind Planner operating under a separate forward-acting system prompt (prompt in Figure 12). At each step, the Planner generates an initial history-conditioned reasoning trace and retrieval action proposal (prompt in Figure 13). A Tutor-edit stage then refines the Planner’s reasoning to align it with the verified retrieval command while remaining strictly grounded in the observable interaction history (prompt in Figure 15). This process produces trajectories that preserve realistic forward causal reasoning while avoiding future-state leakage introduced by backward evidence construction.

None

None

None

None

None

Finally, in Phase C, the Planner generates a final answer conditioned exclusively on the accumulated interaction trajectory (prompt in Figure 14). The completed trajectory is then evaluated by a Trajectory Coherence Judge (prompt in Figure 16), which enforces strict temporal consistency constraints and rejects trajectories that implicitly reveal target entities, retrieval terms, or unobserved facts before they become available within the agent’s causal history.

None

7.2 Efficient Corpus Interaction↩︎

This appendix provides a comprehensive technical overview of the system-level optimizations employed by the DCI agent’s command-execution engine. The engine is designed under a strict correctness-first principle: any pipeline whose parallel execution cannot be guaranteed to be byte-identical to sequential execution is safely executed via a single-file fallback mechanism.

7.2.1 Corpus Sharding and Parallel Fan-Out↩︎

The primary bottleneck in evaluating shell pipelines over large-scale corpora (e.g., a 14 GB JSONL file) is the inherently sequential execution model of standard Unix search tools. To address this, the engine performs a one-time, idempotent, line-aligned sharding of the corpus into \(S\) disjoint partitions.

• Line-Aligned Sharding: The corpus is partitioned along line boundaries (e.g., via split -d -n l/S), ensuring that each JSON record remains intact within a single shard. By construction, concatenating all shards in order reconstructs the original corpus without byte-level modification.

• Thread-Level Fan-Out: At inference time, shell pipelines are executed concurrently across the \(S\) shards using a thread pool. Since each shard is processed via independent subprocess.run calls (which in turn invoke tools such as rg), threading avoids Python-level process management overhead while allowing all shard executions to proceed in parallel. This reduces end-to-end latency for full-corpus scans approximately proportional to the number of shards, up to the I/O and memory bandwidth limits of the system that is hosting the optimized engine.

7.2.2 Pipeline Classification and Merge Strategies↩︎

To guarantee exact behavioral equivalence with sequential corpus execution, the engine employs a conservative pipeline parser that dynamically classifies each shell pipeline and routes it to one of five deterministic execution strategies. If a pipeline begins with an unsupported primitive or contains stateful operations that violate shard independence—such as line-indexing flags (e.g., -n), count-based modes (e.g., -c), contextual windowing (e.g., -A, -B, -C), or in-place transformations (e.g., sed -i)—it is immediately executed via the single-file fallback path. For supported pipelines, the engine aggregates the per-shard partial outputs \(\{R_1, \dots, R_N\}\) using the following semantics:

CONCAT:

Applied to fully stateless pipelines (e.g., rg, grep, cut, tr, sed). The engine executes the pipeline on each shard independently and concatenates the outputs \(R_i\) in shard order.

HEAD:

Applied to pipelines terminating in head -n K. The engine applies the truncation locally per shard to bound memory usage to \(K \times N\) lines. The bounded outputs are concatenated in shard order, and a final global head -n K is applied.

COUNT:

Applied to pipelines terminating in counting operations (e.g., wc -l). The engine extracts the scalar count from each shard and computes the global sum.

SORTHEAD:

Applied to top-\(K\) retrieval pipelines containing sort, optionally uniq, and terminating in head -n K. The engine applies sort | head -n K to each shard. The resulting sorted streams are merged using a deterministic \(k\)-way merge (sort -m), followed by an optional global uniq and a final global head -n K.

SEQUENTIAL:

Applied to any unrecognized, unparseable, or globally stateful pipeline. The pipeline is executed sequentially against the unified single-file corpus to guarantee correctness.

7.2.3 I/O and System-Level Optimizations↩︎

Because the engine predominantly evaluates fixed-string filtering operations, retrieval latency is largely determined by memory bandwidth and I/O access patterns. To maximize throughput, the system implements a tiered I/O optimization stack.

7.2.3.1 RAM-Resident Corpus Placement:

When system memory permits, the corpus and all shards are staged in a RAM-backed filesystem (e.g., /dev/shm). This ensures that all reads are served directly from main memory, avoiding filesystem and disk latency. In addition, the engine proactively warms the page cache at startup to eliminate cold-start penalties on the first retrieval query.

7.2.3.2 Deterministic Execution Flags:

The engine injects a set of deterministic performance flags into supported Unix tools. Memory-mapped I/O (e.g., --mmap for rg and grep) reduces system-call overhead, while --no-config disables user-level configuration to ensure reproducibility. The environment is also fixed to LC_ALL=C, enabling bytewise matching and avoiding locale-dependent overhead without affecting literal search semantics.

7.2.4 Persistent Daemon Architecture and Telemetry↩︎

To eliminate the recurring costs of Python wrapper initialization and process startup across multiple tool calls within a single trajectory, the execution engine is deployed as a long-running persistent daemon. The evaluation loop communicates with the persistent ShardedSearchEngine via a length-prefixed JSON protocol over a Unix socket, amortizing per-call overhead and reducing latency by approximately 1–3 milliseconds per invocation. Finally, the engine preserves standard Unix failure semantics (e.g., returning exit code 0 if any shard produces a match) and deduplicates standard error streams to avoid redundant global error logging. Each tool invocation additionally records fine-grained telemetry—including the parsed command, selected merge strategy, shard configuration, and fallback decision—enabling detailed analysis of fast-path utilization during inference.

7.3 Reward Function↩︎

7.3.0.1 Correctness reward:

Let \(\hat{y}\) denote the agent’s final answer, extracted from the last \(\texttt{<answer>} \cdots \texttt{</answer>}\) block in the trajectory, and let \(\mathcal{Y}=\{y_1,\dots,y_m\}\) be the set of gold reference answers. We compute a token-level \(F_1\) score between \(\hat{y}\) and the reference set. Following prior work [22], each prediction and reference answer is normalized by lowercasing, removing punctuation, dropping articles (“a”, “an”, “the”), and tokenizing on whitespace. For a normalized prediction \(\hat{y}\) and a reference answer \(y \in \mathcal{Y}\), let \(P\) and \(G\) denote their respective token multisets. We define the overlap as the multiset intersection \(o = \sum_t \min(P(t), G(t))\), where \(P(t)\) and \(G(t)\) denote token counts. If \(|P|=0\) or \(|G|=0\), we set \(F_1(\hat{y}, y)=0\). Otherwise, precision, recall, and \(F_1\) are defined as: \[\mathrm{p} = \frac{o}{|P|}, \quad \mathrm{r} = \frac{o}{|G|}, \quad F_1(\hat{y}, y) = \frac{2\,\mathrm{p}\cdot\mathrm{r}}{\mathrm{p}+\mathrm{r}}.\] Since multiple surface forms may be valid, the final answer reward \(R_{\mathrm{ans}} \in [0,1]\) is defined as the maximum score over all reference answers: \[R_{\mathrm{ans}} = \max_{y \in \mathcal{Y}} F_1(\hat{y}, y).\] This provides a dense learning signal that assigns partial credit to partially correct answers, rather than relying on a sparse binary exact-match reward.

7.3.0.2 Format reward:

We define a binary format indicator \(\phi \in \{0,1\}\) to evaluate the structural validity of a trajectory. A rollout is valid (\(\phi=1\)) if and only if it satisfies three strict formatting criteria: (1) all special tags (e.g., <think>, <tool_call>, <tool_response>, and <answer>) are properly balanced and non-overlapping; (2) the trajectory follows the prescribed state-transition structure (reasoning \(\rightarrow\) tool invocation \(\rightarrow\) environment response \(\rightarrow\) final answer), with no text generated outside the designated blocks; and (3) the trajectory terminates within a closing </answer> tag.

This constraint mirrors the strictly formatted interaction protocol enforced during cold-start supervised fine-tuning (SFT), where all trajectories are structurally valid by construction. Consequently, \(\phi\) acts as a gate that isolates and penalizes formatting violations introduced by the policy during reinforcement learning exploration phase.

7.3.0.3 Combined reward:

The final reward during reinforcement learning optimization is defined as: \[R = \phi \cdot R_{\mathrm{ans}} = \phi \cdot \max_{y \in \mathcal{Y}} F_1(\hat{y}, y).\] Thus, only structurally valid trajectories receive a non-zero learning signal, preventing the policy from exploiting the reward function through malformed or out-of-format outputs.

7.4 Experimental Settings & Hyperparameters↩︎

7.4.0.1 SFT Training Phase:

Table 3 summarizes the configuration for both synthetic cold-start data generation and the subsequent Supervised Fine-Tuning (SFT) stage. For data generation, we use the 27B variant of Qwen3.5 as both the answer-aware Tutor (\(\mathcal{M}_T\)) and answer-blind Planner (\(\mathcal{M}_P\)) to construct a 10,000-trajectory dataset. During the Tutor’s backward discovery phase, we apply a dynamic temperature schedule \(0.4 + 0.1 \times \text{iter}\) (up to \(M=5\) refinement steps) with Nucleus Sampling [25] to encourage broader exploration when initial retrieval attempts fail. In contrast, the Planner’s forward assembly and the final coherence judge use fixed temperatures of \(0.7\) and \(0.6\), respectively, with top-\(p=1.0\) across all stages.

For SFT, the 9B Qwen3.5 policy model (\(\pi_\theta\)) is trained for one epoch using AdamW [44] with a peak learning rate of \(5 \times 10^{-6}\) and a global batch size of 32. We use a linear warmup over the first 5% of training steps followed by a constant learning rate schedule. The maximum sequence length is set to 16,384 tokens to accommodate long interaction trajectories, including tool calls and retrieved context. Training is performed in the verl²² framework using FSDP [45] with bfloat16 precision and Ulysses sequence parallelism (degree 4). The experiments are conducted on 4 Nvidia A100 (80GB) GPUs on a machine with 1024GB memory.

7.4.0.2 GRPO Training Phase:

Table 4 lists the configuration parameters employed during the reinforcement learning phase using Group Relative Policy Optimization (GRPO) inside the verl framework. The agent policy (\(\pi_\theta\)) is initialized from the checkpoint optimized during the Supervised Fine-Tuning (SFT) stage and is further trained for a total of 200 global steps. For each input query, the policy samples a group of \(n=5\) independent trajectories to compute relative advantages. Trajectory rewards are determined by the continuous token-level \(F_1\) score, multiplied by a binary formatting gate to strictly penalize structural violations. Computed rewards are centered around the group mean and normalized by their standard deviation, providing a comparative learning signal that optimizes the policy without requiring an explicit critic network. We use a symmetric PPO clip ratio of 0.2 and disable the explicit KL divergence penalty to the reference policy, running exactly 1 proximal epoch per training step to minimize policy degradation.

During rollout generation via the vLLM engine,²³ sampling parameters are configured with a temperature of 1.0 and a top-\(p\) of 1.0 to encourage diverse tool-use exploration while preserving syntax consistency. To support deep, multi-turn interactions over the corpus, the trajectory budget is capped at a maximum sequence length of 16,384 tokens and a maximum of 6 assistant turns. Policy updates are calculated using the AdamW optimizer with a peak learning rate of \(5 \times 10^{-6}\) and a constant schedule following a 5% linear warmup. The optimization runs across a global batch size of 256 questions per step, evaluated in mini-batches of 32 and micro-batches of 1 per GPU. Distributed processing is managed in bfloat16 mixed precision utilizing a Ulysses sequence parallel size of 2 for both the actor and reference models. The experiments are conducted on a machine with 4 NVIDIA A100 (80GB) GPUs and 1024 GB of system memory, taking approximately 4 days to complete.

7.4.0.3 Inference Setting:

Table 5 outlines the core configuration parameters and architectural settings used during the inference phase of our finalized DCI agent. At evaluation time, the agent policy leverages a decoding temperature restricted to \(0.6\) alongside a top-\(p\) of \(1.0\) to ensure highly stable, syntactic consistency during multi-turn tool interaction. To prevent context space saturation from excessively broad document retrieval, the maximum length for any individual shell tool output is explicitly bounded via the tool_max_tokens parameter to \(2,048\) tokens. In strict alignment with the environment constraints used during reinforcement learning, the maximum depth of a single interaction trajectory is capped at \(6\) assistant turns, providing a consistent structural framework for the model’s sequential reasoning. To maximize throughput and safely accommodate long-context interaction histories, the inference infrastructure relies on a maximum sequence length of \(16,384\) tokens. The model is served using a specialized high-concurrency configuration capable of handling a maximum of \(256\) sequences simultaneously, utilizing a high GPU memory allocation threshold of \(0.90\). The physical compute layer is provisioned across a hardware infrastructure consisting of \(2\times\) NVIDIA A100 GPUs, orchestrated with a tensor parallel size of \(2\) to optimize distributed memory access and minimize generation latency during dense search rollouts.

None

None

8 Additional Results↩︎

In this section, we present supplementary results evaluated using the stricter Exact Match (EM) metric. While the main text reports token-level \(F_1\) as the primary evaluation measure, EM is included here to provide a more stringent assessment of answer correctness. Importantly, all key conclusions derived from the \(F_1\) analysis remain consistent under EM, indicating that the observed improvements reflect genuine gains rather than partial-match artifacts. Table [tab:em-result] reports the performance of all evaluated methods under EM. Consistent with the token-level \(F_1\) results, GrepSeek maintains a strong performance advantage, achieving the highest overall micro-average EM score of \(0.4948\), representing a statistically significant improvement over the best dense retrieval baseline (\(p < 0.05\)).

At the dataset level, GrepSeek achieves the best EM scores on four of the seven benchmarks (NQ, HotpotQA, 2Wiki, and MuSiQue), with statistically significant gains on NQ, HotpotQA, and 2Wiki (\(^\uparrow\)). This trend strongly corroborates our main findings: direct corpus interaction is particularly effective in scenarios requiring precise multi-hop reasoning, iterative evidence aggregation, and exact lexical matching. Conversely, EM also highlights the inherent limitations of purely lexical filtering. Because GrepSeek relies on exact string-level matching, it is sensitive to surface-form variation and semantic paraphrasing. This limitation is most evident in the statistically significant performance drop on PopQA (\(^\downarrow\)), as well as lower performance on TriviaQA and Bamboogle compared to the strongest Search-R1 configurations using the Qwen3-4B dense retriever. Despite these localized trade-offs on semantically broad or long-tail queries, the aggregate EM results confirm that GrepSeek remains a highly precise and competitive alternative to index-based retrieval systems.

The EM ablation results (Table 6) further support the importance of each training stage. The full GrepSeek model consistently outperforms both ablated variants across all datasets (\(^\uparrow\)). Removing RL optimization (w/o GRPO) reduces the micro-average EM from \(0.4948\) to \(0.3569\), highlighting the role of policy optimization in improving sequential tool-use decisions. Removing the SFT initialization (w/o SFT) leads to a more severe degradation, collapsing EM to \(0.2836\), which reflects instability when RL is applied without structured trajectory bootstrapping. Similarly, Figure 17 shows the scaling behavior of the cold-start SFT stage under EM across different dataset sizes (\(0\), \(2.5\)k, \(5\)k, and \(10\)k trajectories). The trends closely match those observed in \(F_1\): even a small initialization of \(2.5\)k trajectories yields a substantial improvement over the base model, while further scaling provides diminishing but consistent gains, with performance gradually plateauing beyond \(5\)k examples.

9 Case Studies↩︎

In this section, we present qualitative case studies comparing the reasoning and retrieval trajectories of our DCI agent (GrepSeek) against the strongest dense retrieval baseline, Search-R1 with the Qwen3-Emb-4B retriever. These examples illustrate the different behaviors, strengths, and failure modes of direct corpus interaction through shell commands versus embedding-based semantic retrieval. For each example, we provide the original question, the ground-truth answer, and the generated reasoning traces, retrieval commands, and retrieved observations produced by both systems. The examples highlight two key phenomena observed throughout evaluation:

• Lexical Precision and Multi-Hop Evidence Isolation: GrepSeek performs particularly well in settings that require exact lexical matching and iterative evidence filtering. Using shell operators such as rg and grep, the agent can isolate rare symbolic strings, progressively refine intermediate retrieval results, and compose multi-stage retrieval pipelines that accurately bridge evidence across documents (see Examples [ex:hotpot-formula], [ex:hotpot-rockstar], [ex:hotpot-faithlutheran], and [ex:hotpot-joggers]). This behavior is especially beneficial for multi-hop reasoning, entity disambiguation, and cases where small lexical details determine correctness. In contrast, dense retrievers may smooth over these distinctions due to embedding-level semantic compression, sometimes leading to incorrect generalization or entity confusion.

• Surface-Form Sensitivity and Ranking Limitations: At the same time, direct corpus interaction inherits the limitations of lexical retrieval. Since shell-based search lacks a learned semantic ranking mechanism, relevant documents may appear later in the retrieval stream despite containing the correct evidence (see Example [ex:bamboogle-citibank]). Moreover, strict surface-form matching can make the agent sensitive to lexical variations such as spelling differences or omitted diacritics, causing failures in cases where dense retrievers naturally generalize through semantic similarity (see Example [ex:popqa-vaillant]).

9.0.0.1 Discussion of Qualitative Examples:

Here we discuss and explain the provided case studies:

• Symbolic and Rare-Token Matching (Example [ex:hotpot-formula]): This example demonstrates GrepSeek’s ability to locate highly specific strings like chemical formulas that dense retrievers often struggle to represent in vector space. While the dense retriever returns semantically “chemistry-adjacent” documents, it fails to identify the exact formula. Our agent uses rg -F for an exact match, proving the utility of direct symbolic interaction for technical queries.

• Entity Precision and Disambiguation (Examples [ex:hotpot-rockstar] and [ex:hotpot-faithlutheran]): These cases illustrate how lexical precision prevents the “semantic collapse” often seen in dense models. In Example [ex:hotpot-rockstar], our agent distinguishes between a subsidiary studio and its parent brand. In Example [ex:hotpot-faithlutheran], the agent uses the full, unique school name to isolate a single location, whereas the dense retriever retrieves a different school with a similar name, causing the reasoning to cascade into a geographical error.

• Multi-Hop Evidence Composition (Example [ex:hotpot-joggers]): Here, we highlight the agent’s ability to filter retrieval results iteratively. By searching for a specific band and then filtering for “singer,” the agent isolates the relevant person. It then effectively parses a “highest [noun]” construction, showing that DCI can handle complex, multi-step queries by treating the corpus as a structured database.

• Temporal and Stale Information (Example [ex:bamboogle-skytree]): This highlights the importance of precise record-keeping. Our agent retrieves the document explicitly referencing the “current” record holder, while the dense retriever is misled by a document containing stale information about the previous record holder (CN Tower).

• Limitations of Ranking and Surface Forms (Examples [ex:bamboogle-citibank], [ex:popqa-arndt], and [ex:popqa-vaillant]): These examples expose the failure modes of DCI. In Example [ex:bamboogle-citibank], the lack of ranking forces the agent to rely on file order, burying the correct answer. In Examples [ex:popqa-arndt] and [ex:popqa-vaillant], we see that if the agent does not guess the exact surface form (e.g., handling diacritics or name variants), it fails where semantic models would intuitively generalize, underscoring the trade-off between lexical control and semantic robustness.

10 Examples of Generated Synthetic Trajectories↩︎

This appendix presents a collection of synthetic trajectories generated through our data construction pipeline for supervising and training the DCI agent. These examples illustrate how the model learns to maintain coherent multi-turn reasoning while effectively using shell-based search operations. In particular, SFT Example [ex:sft-castles] shows the agent resolving geographical intersections, while SFT Examples [ex:sft-pacemaker] and [ex:sft-actress] demonstrate multi-hop compositional reasoning and cross-domain bridging, respectively. Each trajectory illustrates the step-by-step evolution of reasoning as the agent generates shell commands to retrieve supporting evidence.

References↩︎

---

1. Document refers to any retrievable entry, regardless of how the given text is chunked. These chunks should be identified and fixed prior to indexing and are the same for all queries.↩︎

2. Available at: https://www.anthropic.com/claude/sonnet↩︎

3. Available at: https://github.com/alirezasalemi7/grepseek↩︎

4. The corpus does not need to be stored as a single physical file. We expose it to the agent as a single logical file for simplicity, allowing the model to reason about one unified corpus interface. Internally, the execution engine can map the same command to the underlying collection of files and execute it accordingly.↩︎

5. The action space consists of Unix tools, such as rg, grep, find, sed, awk, head, tail, cat, ls, wc, sort, cut, uniq, and tr. In practice, the agent primarily relies on rg and head.↩︎

6. We observed that this approach frequently resulted in both VRAM and host RAM out-of-memory failures, even on systems provisioned with up to 1024 GB of RAM, which makes the training procedure unstable.↩︎

7. Our method is compatible with standard reinforcement learning algorithms; we adopt GRPO due to its favorable memory efficiency and stability for long-horizon tool-use trajectories.↩︎

8. In this paper for our experiments, we use a Wikipedia corpus of \(21\)M passages (approximately 14GB).↩︎

9. In practice, the vast majority of pipelines generated by the agent are compatible with shard-parallel execution, with non-parallel or globally stateful commands occurring only rarely.↩︎

10. These optimizations affect only execution efficiency and are not required for correctness. The system can also operate directly on disk-resident corpora with identical outputs.↩︎

11. All datasets are obtained from https://hf.co/datasets/RUC-NLPIR/FlashRAG_datasets.↩︎

12. Available at: https://hf.co/datasets/PeterJinGo/wiki-18-corpus, \(\sim14GB\) text.↩︎

13. Available at: https://hf.co/Qwen/Qwen3.5-9B↩︎

14. Available at: https://hf.co/Qwen/Qwen3.5-27B↩︎

15. While many recent agentic frameworks focus on improving reasoning[28], [29] or multi-agent orchestration [30], our primary goal is to isolate the effect of the retrieval mechanism itself. Therefore, we focus on methods that differ mainly in how evidence is retrieved—namely, traditional sparse and dense retrievers versus corpus direct interaction. Improvements proposed by other orchestration-based methods are largely orthogonal and could in principle be integrated with either paradigm.↩︎

16. Available at: https://hf.co/intfloat/e5-base-v2↩︎

17. Available at: https://hf.co/Qwen/Qwen3-Embedding-4B↩︎

18. Available at: https://github.com/facebookresearch/faiss↩︎

19. As discussed earlier, directly optimizing the base model without SFT initialization was highly unstable. For the w/o SFT setting, we therefore report results from the final checkpoint before training collapse.↩︎

20. Available at: https://hf.co/datasets/RUC-NLPIR/FlashRAG_datasets↩︎

21. Available at: https://hf.co/datasets/PeterJinGo/wiki-18-corpus↩︎

22. Available at: https://github.com/verl-project/verl↩︎

23. Available at: https://vllm.ai/↩︎