1 Introduction↩︎

Large language models (LLMs) combined with retrieval-augmented generation (RAG) systems [1] have recently achieved strong results on established benchmarks. A major challenge for LLMs lies in their training data [2]. These models are rarely exposed to highly specific and hard-to-access documents, such as historical sources which, when digitized, are often stored in digital libraries. This limitation significantly impacts their ability to process and reason about specialized historical corpora, where domain expertise and contextual understanding are paramount. By grounding generation in retrieved documents, RAG reduces—though does not eliminate—hallucination and, in some cases, enables good performance without task-specific fine-tuning. As RAG architectures diversify [3], [4], reported benchmark scores have continued to rise [5], [6], but these gains mask important limitations when models are applied to complex, domain-specific tasks.

First, hallucination and retrieval errors remain a problem in long-context and multi-document scenarios: when answers require reasoning over lengthy documents or the synthesis of information from several heterogeneous sources, errors compound, and factual consistency degrades. Even for RAG, it has been shown how failure can increase in long context [7].

Second, standard benchmarks remain poorly aligned with domain-specific challenges. They usually focus on multiple-choice and short-answer questions, short passages, or single-document question answering [8], [9]. Recent initiatives have attempted to address this limitation by proposing more targeted benchmarks, such as Humanity’s Last Exam [10]. However, many of these datasets are still relatively general in scope and focus primarily on factual questions.

These shortcomings become particularly evident in the case of historical corpora, where existing benchmarks fail to capture the kinds of reasoning that domain experts require such as temporal reasoning, cross-source linking, and the handling of highly specific, hard-to-access documents.To illustrate this, we focus on a case study of the French Third Republic (1870–1940), drawing on agenda-setting theory [11] to situate parliamentary debates within their media ecosystem. This period, marked by an autonomous Parliament [12] and the rise of the “grande presse d’information¹” [13], provides an ideal context for studying how complex, cross-source questions emerge. Fully digitized corpora of parliamentary debates² exemplify these challenges: historians often need to connect evidence across heterogeneous documents, trace arguments over time, and aggregate sparse mentions dispersed across records. To better capture how issues circulated and gained prominence, we complement this corpus with digitized newspapers from the same period. By linking parliamentary proceedings with contemporary press coverage, we aim to reconstruct the dynamic interplay between institutional discourse and public framing, and to support the generation of questions that reflect the full complexity of historical agenda-setting processes. The issue lies not only in the thematic scope of the questions, but also in the types of reasoning that are relevant to historians. Those requirements affect every component of a RAG pipeline: indexing and chunking methodologies for long texts, retrieval architectures and similarity metrics, the exploitation of metadata and temporal signals, and answer-generation approaches that must support complex argumentative or interpretive responses rather than only fact verification or multiple-choice answers.

In this paper, we present a method for generating, analyzing, and validating high-quality questions that reflect historians’ research problems. Drawing on historical parliamentary debates (sources that are thematically rich, extensive, and span several decades) and cross-referencing them with contemporaneous press archives, we demonstrate how to incorporate temporal dynamics and source sparsity into question generation, and how clustering and metadata can help identify promising multi-hop candidates. We further analyze the structure of both the corpora and the questions to identify natural seeds for multi-hop question generation. We evaluate RAG systems at both the retrieval and generation stages on a carefully selected subcorpus of French Third Republic parliamentary proceedings.

Our work makes four main contributions. First, we propose HistoriQA-ThirdRepublic³ a dataset historically aligned, requiring reasoning across diverse sources and formats, and designed to challenge RAG systems. It comprises 897 single-hop and 885 multi-hop questions, all automatically generated through controlled prompting and validated by a historian for factual and contextual coherence. Second, we introduce a method for generating domain-specific multi-hop questions from heterogeneous corpora, combining embedding-based similarity, temporal constraints, and expert-guided prompt engineering. We further evaluate the resulting questions by analyzing their evolution in semantic space and conducting a preliminary classification of their types. Third, we introduce an evaluation protocol that leverages historian judgments to assess both question and answer quality. Finally, we conduct an automatic evaluation of RAG retrieval and generation performance and investigate the alignment between LLM-based judgments and human annotations.

Taken together, these contributions aim to bridge the gap between off-the-shelf RAG benchmarks and the specific needs of historical scholarship, enabling more meaningful evaluation and targeted development of retrieval and generation methods for complex, domain-specific corpora in the humanities and social sciences.

2 Related work↩︎

2.0.0.1 Multi-Document Reasoning Datasets

Multi-document reasoning have been developed to challenge information retrieval and question answering systems. Resources such as HotpotQA [14] and 2WikiMultihopQA [15] provide deeper insights into the question-generation process. In particular, HotpotQA introduces a detailed taxonomy of question categories, covering various question types and forms of multi-hop reasoning. Building on this line of work, datasets like MoreHopQA [16] propose even more complex multi-hop questions that require additional layers of reasoning, including commonsense, arithmetic, and symbolic inference.

2.0.0.2 Automatic Question Generation for RAG Evaluation

Research on automatic question generation has shown promising results for improving the evaluation of RAG systems in specific context, particularly by enabling the systematic assessment of a model’s reasoning capabilities over long and structured documents. By combining embeddings, metadata, keywords, and clustering techniques, it is possible to generate domain-aligned questions ranging from single-hop factual queries to multi-hop, temporally grounded questions that require cross-source synthesis. Such synthetic question sets can be used to stress different parts of the RAG pipeline and to create more realistic evaluation scenarios for domain specialists [17]–[19].

2.0.0.3 Retriever Performance in Long-Context and Historical Documents

Recent studies have examined the ability of retrievers to reason over long-context documents in a historical context with particular attention to the comparison of chunking strategies and the evaluation of different retriever architectures [20]. Earlier work often relied on fixed-size chunks [21], determined by the chosen segmentation strategy, without accounting for the documents’ intrinsic structure. As a result, some chunks may naturally be longer than others, and enforcing uniform segmentation can be problematic.

3 Corpus Description↩︎

The parliamentary debates of the French Third Republic are available as plain text files resulting from OCR, provided through the digital library of the Bibliothèque nationale de France. For this study, we focus on the 1887 debates in the Chamber of deputies, which provides continuous coverage of parliamentary sessions while remaining small enough for detailed analysis⁴. In 1887, the publicity of parliamentary debates was also firmly established, allowing the press to report extensively on parliamentary proceedings. That same period was marked by international tensions (most notably with Germany), political scandals reaching the highest levels of the state, and a surge of anti-parliamentarism crystallized in the “Boulangist crisis”, which posed a major challenge to the republican regime⁵. These events found in the press both a powerful amplifier and, in some cases, an active ally, as certain newspapers openly supported Boulanger’s followers. These crisis illustrate how parliamentary debates and press coverage became deeply intertwined, shaping public perception and fueling political polarization. This context provides a particularly rich case for studying the interplay between Parliament and media, as it captures both institutional discourse and its mediation in the public sphere.

We complement the corpus of parliamentary proceedings with two contemporary newspapers, Le Gaulois and L’Intransigeant, using plain-text transcripts released by the Bibliothèque nationale de France⁶. These daily newspapers represent two opposing positions on the political spectrum: on the one hand, monarchist and conservative right-wing views; on the other, socialism, although L’Intransigeant would gradually shift toward populism. This multimodal perspective is designed to test whether RAG can connect parliamentary discourse with press coverage, thus requiring reasoning across heterogeneous sources.

While this temporal focus inevitably narrows the scope of the corpus, it remains substantial enough to enable close collaboration with historians, who can provide qualitative feedback on generated questions and RAG outputs.

Table 1 reports descriptive statistics for the three corpora used in our study. Each corpus consists of documents or chunks. For the parliamentary debates, segmentation relied on regular expressions identifying session boundaries and speaker turns, while enforcing a minimum chunk size of 10,000 characters. Newspaper articles were subsequently extracted via dedicated regex patterns applied to the relevant portions of the reports. The debates dataset (Les Débats) comprises more than 3,200 documents.

To enhance textual coherence, we filtered out non-debate sections (e.g., agendas, procedural notes, vote lists). The resulting 963 documents display wide variation in length, mirroring the thematic diversity and rhetorical dynamics of the assembly. This filtered subset thus offers a more faithful view of deliberative discourse while reducing noise for downstream analysis.

The two complementary corpora consist of press reports on parliamentary sessions published in Le Gaulois and L’Intransigeant⁷.

Although shorter than the parliamentary proceedings, these texts add interpretative and often polemical perspectives. Combined with the debates, they create a rich contrast between institutional discourse and mediated public interpretation—an essential feature for evaluating multi-hop reasoning.

4 Methodology↩︎

In this section, we present our approach to generating and evaluating historical questions from the corpus. We describe the prompting strategies, embedding-based similarity measures, and visualization techniques used to assess the relation between the generated questions and their source material.

4.1 Single-hop Question Generation↩︎

We focus on generating questions answerable from single text chunks, restricting our scope to parliamentary debates from the filtered sub-corpus (963 documents). Through iterative refinement with our team historian, we developed a prompt designed to favour (i) substantive debate content over procedural detail, (ii) the articulation of political positions and key issues, and (iii) paraphrase over surface-level text reuse. The model returned structured JSON output, yielding an empty list when no suitable question could be produced.

To assess semantic relationships, we computed TF–IDF representations and applied PCA dimensionality reduction (Figure 2). On average, the cosine distance (\(d = 1 - \text{similarity}\)) between questions and their source chunks is 1.04 for the baseline prompt and 1.26 for the optimized prompt indicating higher semantic divergence after prompt refinement. This increase suggests that the optimized prompt encourages conceptual reformulation rather than lexical copying—a trend corroborated by qualitative inspection.

4.2 Multi-hop question generation↩︎

We next address the generation of multi-hop questions that span both parliamentary debates and newspaper articles. A natural starting point is to pair text chunks that are temporally proximate. While this approach substantially reduces the number of potential combinations, it does not ensure that the retrieved documents are semantically related, and thus suitable for multi-hop reasoning within a RAG framework.

Figure 3 presents a two-dimensional projection of newspaper samples, each linked to its top-3 most similar debate chunks, highlighted in yellow. This visualization illustrates the potential for identifying candidates for multi-hop question generation: in many cases, clusters of semantically close text segments can be observed in the embedding space. The embedding model appears to capture meaningful relationships between press samples and parliamentary debates. This unsupervised visualization supports our assumption that appropriate candidates for multi-hop question generation can be detected automatically. To refine the selection, we applied a similarity threshold of \(0.7\), ensuring that only highly relevant pairs were retained. For the multi-hop question generation phase, we once again collaborated with the historian on our team to iteratively refine the prompting strategy. Our expert provided domain-specific examples that helped define the types of relationships and reasoning chains most valuable for historical interpretation.

We identified two complementary dimensions for guiding the generation process. The first concerns the type of questions. Inspired by the HotpotQA taxonomy of questions [14], early iterations produced a first set of multi-hop questions that involved, for instance, identifying an opinion expressed in the debates and examining how the press reacted to it; we refer to these as follow-up questions. We then extended this to bridge-entity questions, where a shared reference (e.g., a political figure or event) connects a debate passage with a newspaper article. Finally, we introduced comparative questions designed to contrast differing viewpoints across multiple sources. The full prompts used for these experiments are detailed in Appendix 12 and provided in the supplementary material available in the dataset repository.

The second dimension relates to the source pairing strategy. We focused on two types of document relationships:
Newspaper-to-debate questions. For each newspaper chunk, we retrieved its top-1 most similar debate passage based on cosine similarity between embeddings. This direction (press to debate) was chosen to limit the total number of generated questions while maximizing the diversity of reasoning types. A similarity threshold of \(0.7\) was again applied to filter weakly related candidates.
Newspaper-to-newspaper questions. We computed all similarity pairs between the two selected newspapers. A threshold of \(0.7\) was applied, along with a temporal constraint requiring that the paired articles be published no more than one week apart.

For the present study, we set aside multi-hop questions relying exclusively on parliamentary debates. Although such questions could provide insights into the evolution of political arguments over time, they carry the risk that multi-hop reasoning would occur across adjacent or even identical sessions, thereby limiting the diversity of perspectives. By combining debates with newspapers, we instead emphasize the interpretive gap between official parliamentary proceedings and their representation in the press. This approach introduces an additional challenge: the retrieval process must operate across distinct corpora, each characterized by different styles, formats, and lengths.

Table 2 presents samples of the generated questions. In total, we produced \(885\) multi-hop questions. Figure 4 presents a sunburst plot for the first four words of the generated questions.

5 Experiments↩︎

This section addresses three main questions: How do current models perform on a long-context historical dataset ? (RQ1) To what extent do LLM-based evaluations of retrieval and generation align with human judgments ? (RQ2) How are question types related to model performance and error patterns ? (RQ3)

5.1 Experimental Settings↩︎

Question generation experiments were conducted using the Cohere API. We employed the Command-R model for reasoning-oriented tasks and Command-A for both question generation and RAG-based answering. Dense retrieval was performed using embeddings computed with cohere-embed-v4, while retrieved passages were reranked with cohere-rerank-multilingual-v3.0. The vector database was implemented using ChromaDB.

In addition to Cohere models, we evaluated several large language models for both generation and RAG-based answering, including Llama 3.2 3B Instruct, Llama 3.3 70B Instruct, GPT-OSS 20B, and GPT-4o-mini.

The final prompts and system configuration were refined through multiple iterative cycles in close collaboration with our team historian, who is a specialist of this period. This process enabled the selection of representative few-shot examples for question generation, LLM-as-a-judge evaluation, and RAG-based answering.

5.2 Retrieval Evaluation↩︎

We evaluate the retrieval stage of the RAG pipeline by measuring how often the gold source chunks used to generate each question are successfully returned by the retriever.

Because questions were generated from known source passages, this setup provides an unambiguous oracle for retrieval performance. However, we acknowledge that in a large historical corpus, multiple documents may validly support the same answer or provide alternative perspectives. Consequently, our retrieval metrics should be interpreted as a conservative lower bound; they specifically measure the system’s ability to recover the exact evidence used to construct the question, even though other relevant documents may exist within the top-\(K\) results. To account for this strict exact-match limitation, we provide an extended analysis of “day-tolerance recall” and temporal near-misses in Appendix 11.

Table 4 reports Recall@K, Accuracy@K (i.e the proportion of questions for which all gold documents appear in the top-\(K\)), and MRR@K for the main experimental configurations.

Single-hop retrieval is highly effective: the embedding retriever alone achieves Recall@3 = 67.8, and reranking provides modest gains. This indicates that for questions relying on a single chunk, dense embeddings capture most of the necessary signal.

Multi-hop retrieval is more challenging. Cross-newspaper retrieval achieves low recall with embeddings alone (Recall@3 = 14.3). Cross-domain retrieval (newspaper–debate) performs better (embedding Recall@3 = 47.8). In general cross-domain retrieval performs way better than cross-newspaper retrieval. This indicates strong corpus-bias effect when retrieval is performed on the full database (debates + newspapers): the retriever frequently returns debate chunks even for newspaper–newspaper queries. Concretely, for \(K=3\) an average 80.6% of retrieved documents belonged to the parliamentary debates corpus; this proportion increases with \(K\) (83.6% for \(K=5\), 86.8% for \(K=10\)). Restricting retrieval to the newspaper-only sub-dataset markedly improves cross-newspaper performance, confirming that source-type priors and corpus-aware retrieval are crucial for cross-source tasks.

A non-obvious finding is that the effect of reranking is not uniformly positive. While reranking often improves Recall@10 and MRR@10, it can reduce top-\(K\) metrics and Accuracy@3 in some configurations. For example, in the newspaper–debate setting embedding-only Recall@3 = 47.8 drops to 45.5 after reranking This behavior likely stems from mismatches between reranker training objectives, domain differences, and corpus biases.

In summary, high single-hop accuracy suggests that answer synthesis, rather than retrieval, is the main bottleneck for simple questions. In contrast, low multi-hop recall and the occasional negative impact of reranking are key limiting factors for end-to-end RAG performance. A detailed breakdown of these retrieval failures—specifically highlighting how the retriever exhibits strong source-type confusion—is provided in Appendix 11.

3pt

5.3 LLM-as-a-Judge Evaluation↩︎

In this section, we evaluate the alignment between human judgments and LLM-as-a-judge evaluations. A subset of \(50\) questions, together with their RAG-generated answers, was annotated by our domain expert. Our objective is to assess how closely an LLM-as-a-judge can replicate these human evaluations.

We made multiple iterations over the questions to ensure their relevance and to check for potential hallucinations. RAG answers were carefully read against the source document and annotated as correct, incorrect, or partial. We report the alignment rate, defined as the percentage of cases in which the human and the LLM agree. We consider two judging modes:
Answer-reference mode : in which the LLM-as-a-judge is given the question, the RAG-generated answer (based on retrieved documents), and the reference answer produced during the question-generation phase. The LLM is asked to assess whether the RAG answer aligns with the reference answer.
Source-oracle mode : in which the LLM-as-a-judge is given the question, the RAG-generated answer, and the gold documents used to generate the question. The LLM is asked to determine whether the RAG answer is consistent with these source documents.

The first mode requires an intermediate step to generate a reference answer, whereas the second relies directly on the gold documents. We compare both the alignment between human and LLM judgments, and the agreement between the two LLM judging modes.

We experimented with different prompt configurations for the LLM judge, using two models: a base model (Command-A) and a reasoning-enhanced variant. For prompts, we compared (i) a standard formulation allowing partial answers and (ii) a constrained formulation with only two labels (correct or incorrect), in which partially correct human annotations were treated as incorrect. We also evaluated alignment in cases where not all gold documents were successfully retrieved.

The detailed alignment analysis is provided in Table 5. In the Source-oracle setting—where the judge has access to the original chunks and the corresponding LLM-generated answers—the alignment is perfect for instances where RAG outputs are deemed correct by human evaluators. Across the 20 questions labeled as correct by humans, the LLM judge reached full agreement. Conversely, for incorrect answers, the RAG system exhibits markedly lower alignment.

When using a high-capacity model such as Cohere Command-R, the LLM judge appears highly reliable for answers it classifies as correct. Restricting the analysis to RAG systems using such high-quality models further strengthens this observation.After filtering the dataset to retain only instances where RAG answers were validated as correct, 530 questions remain, distributed across the Follow-up (300), Comparative (121), and Bridge Entity (109) categories.

4pt

5.4 RAG Answer Evaluation↩︎

After filtering for questions where the RAG answers were correctly evaluated by the LLM-as-a-Judge, we perform a full evaluation on this validated high-confidence subset. While we recognize that restricting the analysis to instances where the judge and human expert reach consensus introduces a selection bias toward clearer examples, this methodology was chosen to ensure that the reported RAG performance reflects model reasoning capabilities rather than evaluator noise or "judge hallucinations." In this setup, we do not operate in oracle mode, allowing the retriever to potentially return incorrect documents. We use a configuration with \(K=3\) retrieved documents and no reranking, as reranking was found to negatively affect performance on the multi-hop question dataset. The RAG system is prompted to answer each question based on the retrieved documents, and the resulting answers are evaluated by the LLM-as-a-Judge. This evaluation assesses how effectively each RAG pipeline retrieves and synthesizes information from the source corpus, as reflected in the accuracy scores provided by the LLM evaluator.

Table 3 reports results for a set of models differing in architecture. gpt-oss-20b attains the highest accuracy on both single-hop (58.85%) and multi-hop (18.86%) questions. Llama-3.3-70B-Instruct (53.53% single-hop, 17.92% multi-hop) and gpt-4o-mini (52.72% single-hop, 13.77% multi-hop) follow in performance. Llama-3.3-3b-Instruct shows the lowest performance on both tasks, with 24.10% on single-hop and 3.2% on multi-hop questions.

The gap between single- and multi-hop performance underscores the persistent challenge of compositional reasoning in retrieval-augmented generation. While single-hop accuracy varies significantly (ranging from 24.10% to 58.85%), scores drop substantially on multi-hop questions for all models. This pattern suggests that retrieval quality alone is insufficient: the answering step requires analyzing multiple and potentially long chunks. Effective integration and reasoning across multiple contexts remain critical limiting factors.

6 Conclusion and Future Works↩︎

We introduced a French-language dataset of 897 single-hop and 885 multi-hop questions from French Third Republic parliamentary debates and newspapers, designed to advance RAG system evaluation for historical research. Our historian-guided methodology ensures questions reflect genuine historical inquiry.

While single-hop retrieval performs well, multi-hop reasoning and cross-source synthesis remain challenging. Our experiments used Cohere API models for their benchmark performance. Future work will compare alternative models across question generation, retrieval, reranking, and LLM evaluation; expand the dataset; and refine RAG architectures to improve cross-source reasoning and assess model robustness.

7 Limitations↩︎

We relied on Cohere API models for question generation, retrieval, and alignment evaluation, which means results may differ with alternative models. In addition, the question generation and manual validation were performed by a single historian; future work would benefit from multiple annotators to assess inter-annotator agreement. Furthermore, the corpus is limited to a single year (1887) and two newspapers, which may not capture the full diversity of the period. Finally, our LLM-as-a-judge alignment was validated on a small subset of 50 questions. Furthermore, our final RAG performance analysis is restricted to the subset where human and LLM judgments aligned; this likely overestimates absolute performance by excluding ambiguous or high-disagreement cases. Future research should investigate judge performance specifically on these "hard" cases to better define the boundaries of automated evaluation in historical RAG systems.

8 Ethical Considerations↩︎

The historical documents used in our corpus are in the public domain, sourced from the Bibliothèque nationale de France. These texts contain viewpoints and biases from their period (1887), which the dataset is designed to capture for research purposes, not to endorse. To mitigate the risks associated with historically biased or misleading interpretations, we adopted a historian-in-the-loop methodology for question validation.

9 Bibliographical References↩︎

10 Language Resource References↩︎

11 Retrieval Error Analysis↩︎

We complement the retrieval evaluation of Section 5 by analyzing error patterns along two dimensions: (1) the proximity of failed single-hop retrievals to the gold document, and (2) the relationship between multi-hop question types, source-type confusion, and retrieval performance.

11.1 Single-Hop: Proximity of Retrieved Documents↩︎

Standard recall treats retrieval as binary: either the exact gold chunk is found or it is not. However, many apparent failures retrieve a document from the same publication date as the gold chunk, indicating that the retriever captures temporal context but selects an adjacent passage. Figure 5 compares exact-match recall with day-tolerance recall, where a retrieval counts as successful if any top-\(k\) document shares the publication date of the gold document.

The gap between exact and day-tolerance recall is consistent across all \(k\) values, showing that a substantial fraction of errors are near-misses rather than complete failures. Table 6 provides a detailed breakdown: while exact-match recall plateaus at 76.4% for top-10, same-day retrieval reaches a notably higher rate, leaving only a minority of questions as complete misses where no temporally related document appears in the top-10.

11.1.0.1 Temporal and chunk distance.

Figure 6 shows the distribution of the minimum temporal distance between retrieved and gold documents. At \(k=10\), the majority of questions have a same-day document in the top results, and the remaining are equally distributed between a 1–7 day window and an 8+ day window.

For same-day retrievals that miss the exact chunk, Figure 7 reports the chunk distance distribution. The majority of near-misses fall within a long distance from the actual chunk, suggesting that the retriever model is, most of the time, either able to find the correct chunk or misses it by a lot.

11.2 Multi-Hop: Question Type and Source Confusion↩︎

Figure 8 breaks down retrieval performance by multi-hop question category (Exact Recall@10 vs.Day-Tolerance Recall@10). The gap between the two metrics varies across types, revealing that certain categories are disproportionately affected by near-miss chunk selection.

11.2.0.1 Source-type retrieval across \(k\).

Figure 9 tracks how the proportion of questions with correct source types evolves across \(k\) values. At \(k{=}2\), 37.3% of questions retrieve all expected source types, increasing to 76.6% at \(k{=}10\). Even at \(k{=}10\), 8.6% of questions return only wrong source types. The “wrong only” category decreases steadily with \(k\), but never vanishes, indicating a persistent structural mismatch for a subset of queries.

11.2.0.2 Source-type confusion by question type.

Figure 10 breaks down the source-type retrieval status by question category. Comparative questions show the highest “wrong only” rate (25.5% at \(k{=}10\)), as they require cross-newspaper retrieval which is most affected by the debate corpus bias. Bridge Entity questions never have “wrong only” errors—their debate–newspaper structure naturally aligns with the retriever’s tendency to return debate chunks. Generic (Follow-up) questions fall in between.

11.2.0.3 Source-type confusion analysis.

Figure 11 details the source-type confusion patterns at \(k{=}5\) and compare expected vs.retrieved source types. For questions requiring both newspaper and debate documents, the retriever consistently over-represents debate chunks—consistent with the corpus bias reported in Section 5 (80.6% debate chunks at \(K{=}3\)). Questions expecting exclusively cross-newspaper sources are most affected: the retriever frequently returns debate passages instead of the required newspaper pairs. This asymmetry confirms that source-aware retrieval strategies, such as forced multi-collection distribution or classification-based routing, are essential for cross-source multi-hop question answering on heterogeneous historical corpora.

12 Prompt Specifications for Question Generation↩︎

This appendix documents the system prompts used for multi-hop question generation in the HistoriQA-ThirdRepublic dataset. All prompts were written in French to match the language of the source corpora. We present below one representative prompt in full (Follow-up: Debate–Newspaper), followed by a summary of the four remaining prompts. The complete set of prompts is available in the accompanying repository.
Type: System message
Purpose: Generate multi-hop follow-up questions requiring cross-referencing between a parliamentary debate passage and a newspaper article reporting on the same debate.

User Message↩︎

The user message contains the two text excerpts to be analysed, provided as Extrait A (parliamentary debate passage) and Extrait B (newspaper article).

References↩︎