1 Introduction↩︎

The emergence of Retrieval-Augmented Generation (RAG) technology has significantly mitigated the core challenges prevalent in large language models (LLMs) [1], such as factual inaccuracies and content hallucinations[2]. By deeply integrating external knowledge bases with LLMs, RAG first retrieves the most relevant document segments from the knowledge repository based on the user’s query prior to response generation. These retrieved results are then fed into the model as contextual information, thereby guiding the model to produce more accurate and reliable outputs. Currently, RAG technology has been widely adopted in high-stakes domains where information accuracy is paramount, including healthcare [3], finance[4], and government services [5].

However, when the external knowledge base of a RAG system contains a large amount of sensitive information, such as patient medical records and personally identifiable information (PII), the retrieved text segments often contain highly sensitive content [6]. This sensitive information may be leaked by large language models through various means, including direct verbatim reproduction, indirect inference, and gradual reconstruction, leading to uncontrollable privacy risks. Multiple studies [7]–[9] have pointed out that RAG systems face severe privacy and security challenges when the retrieval process involves private data. Therefore, enhancing the privacy protection capabilities of RAG systems and preventing the leakage of retrieved data are crucial for preventing unauthorized access and misuse of data [10].

Differential Privacy (DP) [11] offers a mathematically rigorous solution to this challenge. By injecting carefully calibrated noise into the data, DP guarantees that an adversary cannot determine with high confidence whether a specific individual’s record is present in the dataset, even if they have access to all other records. Several existing works have leveraged DP to protect the privacy of retrieved contexts in RAG systems. Koga et al. [12] proposed selective privacy budget allocation to sensitive tokens to support accurate long-form generation under constrained budgets. Grislain [10] further demonstrated the feasibility of applying DP to token-level generation in RAG. Beyond token-level protection, some works have also explored document-level privacy mechanisms[13]. Mori et al. [14] proposed a differentially private synthetic text generation approach that replaces sensitive documents with DP-compliant synthetic alternatives, mitigating leakage at the data source.

The aforementioned studies have provided valuable insights into privacy protection for retrieved contexts in RAG systems. However, these methods share a fundamental limitation: they are all based on the document-level static privacy risk assumption, which presumes that all retrieved documents carry an equal risk of privacy leakage. This overlooks the critical fact that the privacy leakage risk of the same document is highly dependent on the specific content of the user’s query (prompt). For instance, for the same internal company document, a query asking to "summarize the company’s cultural philosophy" poses almost no privacy risk, while a query requesting to "list the names and contact information of all department employees" will lead to severe privacy leakage. This implies that the privacy protection process in RAG systems should essentially be a query-driven dynamic process rather than the document-level static process currently widely adopted.

To address the inherent limitations of the aforementioned static privacy protection methods, this paper proposes a Prompt-Aware Dynamic Hierarchical Differential Privacy (PA-HDP) protection method. In particular, this method introduces a prompt-driven fine-grained privacy risk assessment mechanism into the retrieval phase of RAG systems. First, it performs sentence-level sensitive semantic recognition on the retrieved contexts based on the user’s query (prompt), and divides the text segments into different privacy risk levels according to the type of sensitive information, the severity of leakage harm, and the relevance to the query intent. Then, for text content at different risk levels, we design an adaptive candidate selection strategy based on the exponential mechanism, where different privacy budgets are assigned according to the risk level. Specifically, a diverse candidate set is first generated through semantically equivalent replacement of sensitive entities. The exponential mechanism then selects a protected text from the candidate set that satisfies differential privacy constraints while preserving the highest possible semantic fidelity. This method can achieve an optimal trade-off between privacy security and model generation utility without compromising the semantic coherence and information integrity of the original context. Our experimental results show that using our algorithm can achieve comparable performance with using original data while substantially reducing the associated privacy risks.

Our contributions are summarized as follows :

• An important yet underexplored research question has been raised. We point out the prevalent flaw of document-level static risk assumption in RAG privacy protection and reveal the query-dependent dynamic privacy leakage problem.

• A Prompt-Aware Dynamic Hierarchical Differential Privacy framework (PA-HDP) is proposed. We propose a Prompt-Aware Dynamic Hierarchical Differential Privacy protection framework. The framework integrates risk stratification, adaptive sensitive entity replacement, and exponential mechanism-based text selection to provide differentiated privacy protection while preserving semantic utility.

• The experiments verified the effectiveness of the proposed method. Extensive experimental results demonstrate that, under the premise of strictly satisfying the mathematical constraints of differential privacy, the PA-HDP method can significantly reduce the semantic loss of generated content and achieves a better trade-off between privacy and model utility.

2 Related work↩︎

Retrieval-Augmented Generation (RAG) improves the factuality of large language models but introduces new privacy risks, as sensitive information contained in retrieved documents may be exposed through generated responses. Ensuring strong privacy guarantees is therefore critical, with differential privacy emerging as a principled and widely adopted framework for mitigating information leakage. Existing privacy-preserving RAG approaches can be broadly categorized into three directions: (1) token-level generation protection based on differential privacy, (2) protection based on external databases, and (3) other privacy protection methods.

2.1 Privacy protection for Token-level Generation↩︎

These methods directly apply differential privacy mechanisms during the LLM decoding phase, injecting noise into token probability distributions or logits to limit the leakage of retrieved information. Early work [12] explored the fundamental challenges of differentially private RAG, namely how to generate accurate long text under limited privacy budgets, and proposed a strategy that allocates budgets only to sensitive tokens. InvisibleInk [15] interprets next-token sampling as an exponential mechanism over logits, reducing privacy costs by isolating sensitive information in public versus private logits, and samples from a superset of top‑k private tokens to improve utility, achieving an 8‑fold reduction in computational cost under the same privacy level. Privacy‑Aware Decoding (PAD) [16] is a lightweight inference-time defense that adaptively adds calibrated Gaussian noise to token logits, combining confidence filtering with efficient sensitivity estimation, protecting only high-risk tokens in a model-agnostic manner with very low computational overhead. A complementary work [10] further validates the feasibility of differential privacy based token generation in private RAG scenarios. PEARL [17] proposes an adaptive differential privacy decoding framework based on confidence gaps and entropy regulation, dynamically allocating privacy budgets at both token and sentence levels, focusing protection on personally identifiable information fragments.

2.2 Privacy protection for External Databases↩︎

The core objective of these methods is to directly protect the retrieved original documents so that LLMs or attackers cannot access sensitive content. VAGUE‑Gate [13] proposes a plug‑and‑play LDP protection mechanism that performs budget-constrained token filtering and random rewriting perturbation on retrieved text, hiding the content of original documents while satisfying \(\varepsilon\)‑LDP. LPRAG [18] applies LDP perturbation only to private entities (words, numbers, or phrases) within the text, identifying entity types and allocating adaptive privacy budgets, using different perturbation mechanisms for each type. Differentially private synthetic text [14] avoids leakage at the source by generating DP-compliant synthetic documents to replace sensitive original data. DP‑KSA [19] adopts keyword semantic compression and a propose‑test‑release mechanism to extract high‑frequency keywords from retrieved documents in a differentially private manner, augmenting only the compressed keywords into the prompt. Text‑DP [20] proposes a textual differential privacy paradigm for context‑aware inference, achieving sensitive information anonymization through differential embedding hashing. RemoteRAG [21] formalizes the privacy protection problem for cloud‑based RAG services, introducing an \((n,\varepsilon)\)-DistanceDP mechanism to perturb queries and document embeddings, narrowing the retrieval scope based on perturbed embedding vectors. ppRAG [22] combines distance‑preserving symmetric encryption (CAPRISE) with the DistanceDP mechanism to enable similarity computation over encrypted embeddings in untrusted cloud environments, ensuring that plaintext documents are never exposed.

Another class of methods does not rely on differential privacy, protecting original documents through synthetic data, knowledge distillation, embedding shifting, or model unlearning. Pure synthetic data methods [7] replace the retrieval database entirely with artificially generated text that contains no sensitive information. Parametric RAG [23] adopts knowledge distillation to convert each document into a parameterized LoRA module, requiring no access to original documents during inference, with document content masked by special tokens. PRESS [24] fine‑tunes the model via embedding space shifting, enabling RAG inference without needing to access original sensitive documents. Additionally, [25] studies methods to erase private knowledge from multiple documents, fine‑tuning the model to actively forget sensitive information from retrieved documents.

2.3 Privacy protection for Training stage↩︎

Beyond the direct protections targeting the generation process and retrieved documents, researchers have also worked on training‑stage privacy, multi‑query privacy accounting, and system security evaluation. Regarding user‑level privacy protection during the training phase of RAG systems, Charles et al.[26] designs two scalable DP‑SGD variants (ELS example‑level sampling and ULS user‑level sampling) and derives a tight user‑level privacy accounting framework, adaptively selecting user group sizes between privacy and computational resources via data‑driven heuristics; Boenisch et al. [27] further proposes a personalized differential privacy method that assigns different sampling rates or clipping norms to different users to achieve individualized protection strength. In multi‑turn interaction scenarios, continuous privacy budget consumption is a key challenge for practical deployment. Private‑RAG (MURAG) [28] shifts privacy accounting from the query level to the document level through document access frequency‑based individual privacy filtering and adaptive threshold release, effectively controlling privacy accumulation in multi‑turn interactions. Beyond Per‑Query Privacy [29] systematically studies privacy budget consumption in online interactions, designing both fixed‑threshold and adaptive‑threshold private retrieval algorithms, making privacy consumption related to document retrieval frequency rather than total query count, achieving more efficient document selection and multi‑turn QA generation under \(\varepsilon\)‑DP guarantees.

3 Preliminary↩︎

Differential privacy is a standard paradigm for protecting privacy of individuals. It requires that changing one entry can create only a small change of the output distributions.

To adapt differential privacy to textual data, we introduce Semantic Metric Differential Privacy (Definition 2).

Based on this definition, we further propose the Semantic Laplace Mechanism and the Semantic Exponential Mechanism, which serve as the core privacy-preserving components of our method.

4 Method↩︎

This paper proposes a Prompt-Aware Dynamic Hierarchical Differential Privacy (PA-HDP) protection method. This method decomposes the entire privacy protection pipeline into three tightly coupled stages: the normalization and definition of privacy units, prompt-aware risk assessment, hierarchical DP protection.

Specifically, the first stage performs standardized data preprocessing and defines the sentence as the minimum granularity unit for privacy protection. The second stage performs fine-grained privacy risk assessment and hierarchical classification on these retrieved segments by analyzing their semantic correlation with user queries. In the final stage, the framework first generates semantically equivalent candidate texts by applying differentiated sensitive entity replacement strategies. It then performs hierarchical differential privacy protection by selecting the optimal sanitized text from the candidate set through the exponential mechanism, while rigorously satisfying the formal constraints of differential privacy. The complete pseudo-code is shown in Algorithm 1.

Furthermore, Fig. 2 provides an example illustrating the pipeline of our privacy protection algorithm.

4.1 Retrieval and Text Preprocessing↩︎

In Retrieval-Augmented Generation (RAG), the retrieval stage determines the external knowledge accessible to the large language model (LLM) and serves as the primary entry point for privacy leakage. Given an external knowledge base \(D=\{d_1,\dots,d_n\}\), each document is first chunked, embedded using an encoder \(E(\cdot)\), and indexed as dense vectors. For a user query \(q\), the system computes its embedding \(E(q)\) and retrieves the top-\(K\) text segments \(\{s_1,\dots,s_K\}\) based on cosine similarity. The fixed top-\(K\) strategy limits context length, reduces semantic redundancy, and confines privacy protection to query-relevant content.

To improve processing quality, the retrieved text is further preprocessed through sentence segmentation and text normalization, including punctuation-based boundary detection and the removal of redundant symbols, spaces, and line breaks. These operations provide standardized inputs for subsequent sensitive entity identification and differential privacy protection.

After preprocessing, each sentence \(s_{i,j}\) is defined as a Basic Privacy Unit (PU), where \(s_{i,j}\) denotes the \(j\)-th sentence in the \(i\)-th retrieved context. A privacy unit is the minimum processing unit for sensitive entity recognition, sensitivity assessment, candidate generation, and differential privacy perturbation. Each PU preserves semantic completeness while maintaining an independent privacy level and budget.

The independent design of privacy units enables fine-grained privacy protection and naturally satisfies differential privacy composition. It allows stronger protection for high-risk content while preserving the utility of low-risk text, thereby achieving a better privacy–utility trade-off.

4.2 Prompt-aware risk assessment↩︎

After completing retrieval and text preprocessing, to achieve fine-grained differentiated privacy protection, it is necessary to perform accurate localization, quantitative assessment, and hierarchical classification of sensitive information in the retrieved segments. In RAG scenarios, only sensitive information relevant to the current user query constitutes genuine privacy leakage risks. Even if query-irrelevant sensitive content exists in the retrieval results, it is difficult for Large Language Models (LLMs) to attend to and output it, thus requiring no high-intensity protection.

To realize prompt-aware dynamic risk assessment, this paper proposes a two-dimensional risk quantification model. It separately calculates the risk-aware semantic relevance \(S_{rel}\) between the retrieved segment and the user query \(q\), and the field sensitivity \(S_{sen}\) of the segment itself. The final comprehensive risk score \(S_{total}\) is obtained through weighted fusion, based on which the text is divided into different privacy risk levels.

4.2.1 Risk-aware semantic similarity↩︎

We design a risk-aware gating mechanism that restricts semantic relevance computation to cases where the user query explicitly involves potentially sensitive entities. Specifically, we first assess whether the query contains potentially sensitive entities (e.g., person names, locations, or organizations). This assessment determines whether the current query is likely to trigger privacy leakage in the retrieved documents.

Let the user query be \(q\), and the j-th segment of the retrieved context \(s_i\) be \(s_{i,j}\). We first apply an entity recognition module \(\mathcal{E}(\cdot)\) to extract entities from the query:

\[\mathcal{E}(q) = \{e_1, e_2, ..., e_m\}.\]

We then define a sensitive-entity indicator function:

\[I(q) = \begin{cases} 1, & \text{if } \mathcal{E}(q) \cap \mathcal{E}_{sens} \neq \emptyset \\ 0, & \text{otherwise} \end{cases}\] where \(\mathcal{E}_{sens}\) denotes a predefined set of sensitive entity types.

When \(I(q) = 1\), we compute semantic similarity between the query and each segment using the same embedding model \(E(\cdot)\) employed in the retrieval stage:

\[\mathbf{v}_q = E(q), \quad \mathbf{v}_{s_{i,j}} = E(s_{i,j}),\]

\[S_{rel}(q, s_{i,j}) = \frac{\mathbf{v}_q \cdot \mathbf{v}_{s_{i,j}}}{\|\mathbf{v}_q\| \cdot \|\mathbf{v}_{s_{i,j}}\|}.\]

When \(I(q) = 0\), we conservatively assign \(S_{rel}(q, s_{i,j}) = 0\), where \(\cdot\) denotes the vector dot product and \(\|\cdot\|\) denotes the L2 norm. Thus, \(S_{rel}(q, s_{i,j}) \in [0, 1]\).

Intuitively, when such entities are present, a higher semantic similarity between the query and a retrieved segment indicates a higher probability that the segment may be leveraged by the generation model, thereby increasing the risk of privacy leakage. Conversely, when no sensitive entities are detected in the query, we assume negligible entity-driven leakage risk from direct semantic matching and set the relevance score to zero.

4.2.2 Field sensitivity↩︎

Field sensitivity measures the inherent leakage harm degree of sensitive information contained in a retrieved text segment. This paper adopts a hybrid strategy combining Named Entity Recognition (NER) and rule-based extraction to comprehensively extract sensitive fields from retrieved text segments, forming a sensitive field set \(\mathcal{E}_{i,j} = \{e_{i,j}^1, \dots, e_{i,j}^m\}\), where \(e_{i,j}^k\) denotes the k-th sensitive entity in the \(s_{i,j}\) text segment. The NER module is implemented based on a pre-trained semantic model, which can automatically recognize and locate unstructured sensitive entities such as personal names, locations, ages, and disease diagnoses in the text. These entities usually have the ability to directly or indirectly identify user identities and are the primary sources of privacy leakage. On this basis, the rule-based extraction module complements the recognition of highly structured sensitive information such as phone numbers, email addresses, ID card numbers, and passwords through regular expressions, keyword libraries, and sensitive pattern matching. Such information cannot be fully recognized solely by semantic features but has fixed formats and combination patterns. This hybrid strategy ensures that all content that may pose privacy risks is included in the assessment scope, avoiding protection failure caused by omissions. Subsequently, we assign predefined risk weights \(w_k \in [0, 1]\) to different types of sensitive entities according to their leakage harm degrees. For example, high risk weights (\(w_k \in [0.7, 1.0]\)) are assigned to entities such as ID card numbers, phone numbers, personal names, and disease diagnoses, while low risk weights (\(w_k \in [0.1, 0.3]\)) are assigned to entities such as locations, dates, and common nouns. The field sensitivity of the j-th segment is defined as the maximum value of the risk weights of all sensitive entities in the text segment:

\[S_{sen}(s_{i,j}) = \max_{1 \leq k \leq m} w_k.\]

Using the maximum value instead of the average value as the field sensitivity ensures that a text segment will be classified as high risk as long as it contains one high-risk sensitive entity, thereby avoiding the underestimation of high-risk content caused by the averaging effect.

4.2.3 Risk assessment↩︎

The semantic relevance and field sensitivity are weighted and fused to obtain the comprehensive risk score of the j-th segment: \[\label{eq:risk-score} S_{total}(q, s_{i,j}) = \lambda \cdot S_{rel}(q, s_{i,j}) + (1-\lambda) \cdot S_{sen}(s_{i,j}).\tag{1}\] Where \(\lambda \in [0, 1]\) is the balance coefficient used to adjust the relative importance of semantic relevance and field sensitivity in risk assessment. In our experiments, the optimal value \(\lambda=0.6\) is determined through grid search, indicating that semantic relevance has a slightly greater impact on the final risk than the inherent sensitivity of the field itself.

Furthermore, we introduce the semantic Laplace mechanism to achieve rigorous privacy protection, and yield the noisy risk scores \(S^\ast_{total}(q, s_{i,j})\). (The details are presented in the appendix.)

Finally, text segments are divided into three privacy risk levels according to their comprehensive risk scores:

\[\text{RiskLevel}(s_{i,j}) = \begin{cases} \text{Low Risk}, & S^\ast_{total}(q, s_{i,j}) < \tau_1 \\ \text{Medium Risk}, & \tau_1 \leq S^\ast_{total}(q, s_{i,j}) < \tau_2 \\ \text{High Risk}, & S^\ast_{total}(q, s_{i,j}) \geq \tau_2 \end{cases}\]

where \(\tau_1\) and \(\tau_2\) denote the risk-level partition thresholds. In our experiments, we set \(\tau_1=0.3\) and \(\tau_2=0.7\) based on empirical observations. Text segments (sentences) with different risk levels will adopt differential privacy protection strategies with different budgets to achieve the differentiated privacy protection objective of "low risk, low protection; high risk, high protection".

4.3 Hierarchical DP protection↩︎

To achieve provable and fine-grained privacy protection, we propose the semantic metric differential privacy as the refined privacy definition, and adopt the exponential mechanism as the core privacy algorithm. This section sequentially performs candidate text generation, utility function design, semantic exponential mechanism construction and hierarchical DP protection, and finally outputs secure and highly usable sanitized text under the premise of strictly satisfying differential privacy constraints.

4.3.1 Candidate text generation↩︎

For each retrieved text segment \(s_{i,j}\), we construct a set of semantically equivalent and structurally consistent candidate texts, providing a secure candidate space for the exponential mechanism to select the optimal output under differential privacy constraints. The candidate text construction adopts a core strategy centered on sensitive entity replacement, generating multiple semantically equivalent valid candidate texts while strictly preserving the overall semantic structure and non-sensitive content of the original sentence.

Specifically, we pre-construct a large-scale sanitized entity corpus \(R\) as the secure material source for sensitive information replacement. We employ the hybrid entity extraction strategy proposed in Section 3.2.2 (combining Named Entity Recognition (NER) and rule-based extraction) to extract all entities from the external knowledge base and perform category annotation.

For each retrieved text segment, the system first detects all sensitive entities, including their positions and categories. Next, it randomly selects entities of the same category from the sanitized entity corpus \(R\) to perform isomorphic replacement. Ultimately, the system produces \(m\) semantically identical candidate segments, denoted as \(C=\left \{s_{i,j}^{1}, \dots, s_{i,j}^{m} \right \}\). We set \(m=50\) in our experiments.

4.3.2 Utility function design↩︎

To privately select the most appropriate candidate text from a candidate set while preserving semantic fidelity and reducing privacy leakage risk, we design a composite utility function for the exponential mechanism. The utility function jointly considers two objectives: semantic preservation and privacy safety. Specifically, given a retrieved segment \(s_{i,j} \in \mathcal{X}\) and a candidate text \(s_{i,j}^v \in C\), the utility function is defined as: \[u(s_{i,j}, s_{i,j}^v) = - \alpha \cdot d_{\text{sem}}(s_{i,j}, s_{i,j}^v) - (1-\alpha) \cdot r(s_{i,j}, s_{i,j}^v)\]

where \(d_{\text{sem}}(s_{i,j}, s_{i,j}^v)\) denotes the semantic distance between the original segment and the candidate text, \(r(s_{i,j}, s_{i,j}^v)\) represents the privacy leakage risk of the candidate text, \(\alpha >0\) is a trade-off parameter balancing semantic quality and privacy protection, and \(D>0\) is a normalization constant used to stabilize the utility scale and control sensitivity.

To measure semantic preservation, both the original segment and the candidate text are encoded into dense semantic embeddings using a pretrained text embedding model \(\phi\). The semantic distance is then computed using the Euclidean distance in the embedding space: \[d_{\text{sem}}(s_{i,j}, s_{i,j}^v) = \|\phi(s_{i,j})-\phi(s_{i,j}^v)\|_2.\] A smaller semantic distance indicates that the candidate text preserves the semantic meaning of the original text more effectively. Therefore, the utility function assigns higher utility values to candidates with smaller embedding distances.

To evaluate privacy security, we first extract all sensitive entities present in each candidate text and form a set, denoted as \(E(s_{i,j}^v)\). Each element in \(E(s_{i,j}^v)\) denotes a potentially sensitive entity or phrase, such as names, addresses, phone numbers, or medical identifiers. If no sensitive span is detected, the privacy risk \(r(s_{i,j}, s_{i,j}^v)=0\). Otherwise, each sensitive entity is embedded into the same semantic space, and its semantic similarity to the original segment is measured using cosine similarity. The overall privacy leakage risk of the candidate text is defined as the maximum similarity among all remaining sensitive spans: \[r(s_{i,j}, s_{i,j}^v) = \max_{e\in E(s_{i,j}^v)} \text{cos} (s_{i,j}, e).\] This worst-case formulation emphasizes the most privacy-sensitive residual entity, where even a single highly correlated sensitive span may lead to successful information recovery. In addition, the parameter \(\alpha\) controls the relative importance of privacy protection versus semantic preservation: larger values favor candidates with lower privacy leakage risk, while smaller values place greater emphasis on semantic fidelity. In our experiments, we set \(\alpha=0.5\), which was found to provide the best trade-off between semantic fidelity and privacy protection.

It is worth noting that entity-level privacy risk assessment is also performed on the retrieved progress. However, the two stages serve different purposes: the former evaluates the privacy leakage risk introduced by the prompt itself, whereas the latter measures the residual privacy risk of candidate texts with respect to the original segment. Additionally, to properly bound the semantic distance in the embedding space, all embedding vectors are normalized to unit norm, i.e., \(||\phi(s)||_2 \le 1\). This normalization constrains the embedding space to a bounded domain, ensuring numerical stability and facilitating the derivation of the utility sensitivity bound under the semantic metric.

4.3.3 Semantic exponential mechanism construction↩︎

In this section, we first propose the Semantic Metric Differential Privacy, providing a theoretical basis for privacy protection methods.

Based on the above utility function, the exponential mechanism selects a candidate text according to the following probability distribution:

The proof details are presented in the appendix.

4.3.4 Hierarchical DP protection↩︎

After determining the candidate text set and privacy mechanism, the reasonable allocation of the privacy budget is the key to ensuring the balance between the protection effect of differential privacy and the utility of text generation. To provide differentiated privacy protection for texts with varying sensitivity levels, we first classify all texts into three privacy risk categories: high-risk, medium-risk, and low-risk. High-risk texts typically contain highly sensitive information, while medium-risk texts contain partially sensitive or identifiable content. Low-risk texts are not considered in the subsequent privacy budget allocation.

After risk classification, differential privacy budgets are allocated separately to high-risk and medium-risk texts. Let the total privacy budget be \(\epsilon\), the number of high-risk texts be \(N_h\), and the number of medium-risk texts be \(N_m\). Since high-risk texts require stronger privacy protection, they are assigned smaller privacy budgets than medium-risk texts.

We adopt a risk-aware budget allocation strategy defined as:

\[\varepsilon_H = \frac{\varepsilon}{2(N_H + \lambda N_M)}, \;\varepsilon_M = \gamma \varepsilon_H\]

where \(\varepsilon_H\) and \(\varepsilon_M\) denote the privacy budgets for high-risk and medium-risk texts, respectively, and \(\gamma > 1\) controls the protection gap between different risk levels. In our experiments, we set \(\gamma = 2\), meaning that medium-risk texts receive twice the privacy budget of high-risk texts. This strategy enables stronger protection for highly sensitive texts while preserving better semantic utility for medium-risk texts.

5 Experiment↩︎

This section verifies the effectiveness and privacy protection performance of the proposed PA-HDP method through a series of experiments. We first describe the experimental setup in Section A. Subsequently, Sections B and C evaluate the utility and privacy outcomes of the proposed method. Furthermore, we conduct an ablation study in Section D to investigate how different privacy budgets and numbers of retrieved documents affect the performance of our method. Finally, we perform a comprehensive hyperparameter comparison in Section F to identify the optimal hyperparameter configuration.

5.1 Experimental details↩︎

In this section, we introduce the evaluation datasets, baseline algorithms, and experimental setup. The overall experimental evaluation followed the experimental setup in work [7].

5.1.1 Datasets↩︎

We evaluate the effectiveness of our proposed methods under two privacy-relevant experimental scenarios. The first scenario targets real-world medical dialogue monitoring, where we adopt the HealthcareMagic-101 dataset (comprising 200,000 doctor-patient medical dialogues) as the retrieval corpus.

In the second scenario, following Huang et al. [33], we consider a setting where private information is mixed with public data. Specifically, we construct the Wiki-PII dataset by embedding personally identifiable information (PII) fragments extracted from the private Enron Mail dataset into each sample of the public WikiText-103 dataset. We then benchmark our methods on four standard open-domain question answering (ODQA) benchmarks: Natural Questions (NQ)[34], TriviaQA (TQA)[35], Web Questions (WQ) [36], and CuratedTrec (CT)[37]. Full details on dataset construction and characteristics are provided in Appendix A.9.

5.1.2 Baselines↩︎

To evaluate the effectiveness of our method, we compare against four baselines. The first is Paraphrasing [7], which uses an LLM to extract the most relevant parts of the original context without performing any semantic rewriting. The second is ZeroGen [38], a zero-shot learning framework that first generates a synthetic dataset from scratch using a large pre-trained language model guided by task-specific prompts, then trains a tiny task model (e.g., LSTM) on the synthesized data for efficient inference. The third is AttrPrompt [39], a data synthesis approach that generates training data using diversely attributed prompts (specifying attributes such as length, style, or topic), which yields more diverse and less biased synthetic datasets compared to simple class-conditional prompts. The fourth is SAGE [7], a two-stage synthetic data generation paradigm that mitigates the privacy issues by pure synthetic data. In this method, Stage-1 refers to the outputs generated from attribute-based generation, while Stage-2 represents the outputs of the complete SAGE pipeline. The fifth is LPRAG [18], a privacy-preserving RAG framework with formal privacy guarantees based on local differential privacy (LDP). Detailed descriptions of these baselines are provided in Appendix A.3.

In addition, we report several variants of our framework to analyze the contribution of each component. 0-shot denotes generation without retrieval augmentation. Origin uses the original dataset directly as the retrieval corpus.

5.1.3 Experimental Setup↩︎

For both utility and privacy evaluations, we employ two representative generation backbones: the closed-source GPT-3.5-Turbo and the open-source Llama3-8B-Chat (L8C). Using both proprietary and open-source models allows us to assess the generality of our approach across different model families and deployment settings. We include L8C because its performance on the target tasks relies heavily on retrieval augmentation, making it suitable for evaluating the utility gains provided by RAG. In addition, both models have undergone safety alignment, enabling us to investigate privacy risks in standard RAG pipelines and evaluate the effectiveness of our proposed protection mechanism under realistic deployment conditions.

We adopt bge-large-en-v1.5 as the embedding model. Unless otherwise specified, similarity is computed using the Cosine similarity between normalized embeddings, GPT-3.5-Turbo is adopted as the generation model, only the top retrieved document (\(k=1\)) and a privacy budget of \(\epsilon=5\) are provided for each query.

5.2 Utility evaluation↩︎

To evaluate the utility of synthetic data as retrieval corpora, we assess the quality of generated answers by comparing them against the corresponding ground-truth answers. We report two widely used text generation metrics, ROUGE-L and BLEU, as the primary evaluation measures.

ROUGE-L evaluates the longest common subsequence between the generated answer and the reference answer, capturing their similarity in terms of content coverage and sentence-level structure. BLEU measures the overlap of n-grams between the generated and reference answers, reflecting the lexical accuracy and fluency of the generated text. Higher ROUGE-L and BLEU scores indicate better agreement with the ground truth and, consequently, higher utility of the synthetic retrieval data.

5.2.1 Results on Medical Dialog↩︎

In this section, we split the data into two parts: \(99\%\) of the data is used as the retrieval data, and the remaining \(1\%\) is used as the test data. As shown in Table I, our method consistently achieves the best performance across both generation backbones. Compared with all baselines, it substantially improves both BLEU and ROUGE-L scores. In particular, under GPT-3.5, our method improves BLEU from 0.1594 (Stage-2) to 0.3275 and ROUGE-L from 0.1288 to 0.3445. Similar gains are observed for Llama3-8B-Chat. These results demonstrate that our approach is able to preserve the utility of retrieval data while maintaining high answer quality.

The superior performance can be attributed to the hierarchical protection strategy adopted in our framework. Instead of rewriting or regenerating entire documents, our method first identifies sensitive content and applies protection only where necessary. Most non-sensitive information is preserved in its original form, retaining the semantic integrity and knowledge content of the source documents. By minimizing unnecessary modifications and restricting transformations to a small subset of sensitive spans, the protected retrieval corpus remains highly faithful to the original data, leading to significantly better performance than methods that rely on extensive rewriting or synthetic generation.

5.2.2 Results on ODQA↩︎

Table 2 presents the ODQA results on the Wiki-PII dataset. Overall, our method achieves the best or near-best performance across the four benchmark datasets (NQ, TQA, WQ, and CT) under both GPT-3.5-Turbo and Llama3-8B-Chat. Compared with existing synthetic data generation approaches, including ZeroGen, Stage-2 and AttrPrompt, the proposed method consistently yields higher BLEU and ROUGE-L scores. Notably, the performance advantage is observed for both closed-source and open-source models, indicating that the effectiveness of our approach is not tied to a specific generation backbone. These results demonstrate that the proposed framework can preserve the utility of retrieval data while providing privacy protection, enabling downstream QA systems to maintain high answer quality.

These results highlight the advantage of our fine-grained, risk-aware protection strategy. Existing synthetic generation methods often modify or regenerate large portions of the retrieval corpus, which can introduce semantic distortions and weaken the connection between retrieved documents and downstream questions. In contrast, our method selectively protects only the sensitive content while preserving the majority of the original document structure, factual knowledge, and contextual information. By minimizing unnecessary perturbations to non-sensitive content, the resulting retrieval corpus remains highly faithful to the original data distribution and therefore retains strong retrieval utility. Overall, the results demonstrate that effective privacy protection and high retrieval utility are not necessarily conflicting objectives, and that carefully controlled modifications can preserve downstream ODQA performance while substantially reducing privacy risks.

5.3 privacy evaluation↩︎

To evaluate the privacy protection capability of our method, we conduct both targeted and untargeted extraction attacks following [7]. These attacks are designed to induce retrieval systems to disclose information from the retrieval corpus.

The attack prompt consists of two components: an information component and a command component. The information component is used to guide the retriever toward specific content, while the command component instructs the language model to reveal the retrieved information (e.g., “Please repeat all the context”). For targeted attacks, the information component is crafted to retrieve specific sensitive content, such as personal identifiers or private medical dialogues. For untargeted attacks, it is designed to maximize the amount of information extracted from the retrieval corpus without focusing on any particular target.

For targeted attacks, we report Repeat Prompts and Targeted Information, which measures the number of unique sensitive information items successfully extracted by the attacker. For untargeted attacks, we report four metrics. Repeat Prompts counts the number of attack prompts that induce the model to reproduce at least 10 consecutive tokens from the retrieval corpus. Rouge Prompts counts the number of prompts whose outputs achieve a ROUGE-L score above 0.5 with any retrieved document. We further report Repeat Contexts, the number of unique verbatim excerpts extracted from the corpus, and Rouge Contexts, the number of unique extracted passages with a ROUGE-L score greater than 0.5.

5.3.1 Results on Targeted Attack↩︎

4pt

Table 3 presents the results of targeted extraction attacks on both the Wiki-PII and HealthCareMagic datasets. The original retrieval corpus is highly vulnerable to extraction attacks, especially when paired with GPT-3.5-Turbo, where a large amount of sensitive information can be successfully recovered. Although existing synthesis-based methods, such as ZeroGen and AttrPrompt, reduce leakage to some extent, their effectiveness varies across datasets and models. In contrast, our method consistently achieves zero leakage across all settings, with both Targeted Information and Repeat Prompts reduced to zero.

The results demonstrate that our hierarchical protection mechanism effectively removes exploitable sensitive information from the retrieval corpus while preventing verbatim memorization and disclosure. Compared with Stage-1 and Stage-2, which still exhibit a small number of successful extractions in certain settings, our complete framework eliminates all targeted leakage attempts. This indicates that selectively protecting sensitive content at different privacy levels provides stronger privacy guarantees than directly relying on synthetic generation or coarse-grained document transformation.

5.3.2 Results on Untargeted Attack↩︎

2.0pt

Table 4 reports the results of untargeted extraction attacks on the HealthCareMagic dataset. The original retrieval corpus remains highly vulnerable, especially under GPT-3.5-Turbo, where a large number of prompts successfully induce verbatim or semantically similar disclosures. Although paraphrasing reduces exact leakage, substantial privacy risks still remain. In contrast, our method achieves zero leakage across all prompt-level and context-level metrics under both generation backbones. Compared with Stage-1, which still allows a small number of successful extractions, our complete framework effectively prevents both exact memorization and semantic reconstruction of sensitive content, demonstrating strong robustness against untargeted extraction attacks.

5.4 Ablation Study↩︎

In this section, we conduct two ablation studies to further analyze the effectiveness of our method. Specifically, we investigate the impact of different privacy budgets and the effect of varying the number of retrieved documents. These experiments aim to provide a more fine-grained understanding of how key design choices affect both privacy protection and utility performance.

5.4.1 Impact of Privacy Budgets↩︎

Fig.3 illustrates the impact of the privacy budget \(\epsilon\) on utility across four ODQA benchmarks (NQ, TQA, WQ, and CT). Overall, both BLEU-1 and ROUGE-L scores consistently increase as \(\epsilon\) grows, indicating a clear trade-off between privacy strength and answear quality. When \(\epsilon\) is small (stronger privacy), performance is relatively lower due to more aggressive perturbation. As \(\epsilon\) increases, the model is allowed to retain more accurate information, leading to steady improvements in both metrics. The trend is consistent across all datasets, demonstrating that higher privacy budgets enable better preservation of retrieval utility while maintaining stable gains across benchmarks.

4pt

Table 5 reports targeted attack results under different privacy budgets \(\epsilon\) on both the HealthCareMagic and Wiki-PII datasets using GPT-3.5 as the generation backbone.Table 5 reports targeted attack results under different privacy budgets \(\epsilon\) on both the HealthCareMagic and Wiki-PII datasets using GPT-3.5 as the generation backbone. Across all settings, we observe zero successful extraction, including both Targeted Information and Repeat Prompts, under all privacy budgets from \(\epsilon=0.1\) to \(\epsilon=10\). This indicates that the proposed method is robust to varying levels of privacy strength, and even under weaker privacy constraints (larger \(\epsilon\)), no sensitive information can be recovered via targeted attacks. These results demonstrate that the hierarchical protection mechanism effectively eliminates direct leakage of sensitive content while maintaining consistent behavior across different privacy budgets.

5.4.2 Impact of the retrieved number of documents↩︎

4pt

Table 6 reports the impact of varying the number of retrieved documents \(k\) on utility across HealthCareMagic and four ODQA benchmarks. Overall, the results remain stable when increasing \(k\) from 1 to 5, with only minor fluctuations in BLEU and ROUGE scores. On HealthCareMagic, performance is consistently high and even slightly improves at \(k=3\), indicating that a moderate increase in retrieved context can provide additional useful information. For ODQA datasets (NQ, TQA, WQ, and CT), we observe similarly small variations, suggesting that increasing the number of retrieved documents does not introduce noticeable degradation in generation quality.

These results demonstrate that our pipeline maintains robust utility across different retrieval settings. In particular, expanding the retrieval scope does not harm performance, indicating that the generated data is able to support multi-document retrieval without overfitting to a specific retrieval size. This further confirms the stability and generalization capability of our method under varying retrieval configurations.

5.4.3 Impact of Model Choice↩︎

We evaluate the utility performance of Mistral-7B-Instruct-v0.2(M7I-v0.2), L8C and GPT-3.5 on ODQA and HealthCareMagic benchmarks, where BLEU-1 and ROUGE-L are adopted to quantify the similarity between generated content and ground-truth references. Notably, the ODQA results plotted in the figure correspond to the averaged values across its four sub-datasets. On both datasets, M7I-v0.2 and L8C obtain highly comparable but relatively low metric values. GPT-3.5 achieves obvious performance advantages over the two open-source models, and the performance gap is more prominent on the challenging ODQA task than the HealthCareMagic task.

The close results of M7I-v0.2 and L8C reveal that lightweight open-source LLMs have similar constrained capacity for our stage-1 data generation, which produces low-fidelity synthetic text. In contrast, GPT-3.5 provides steady and superior utility across two different task scenarios. This outcome confirms GPT-3.5 acts as a reliable backbone for our two-stage privacy-preserving generation framework.

5.5 Hyperparameter Choice↩︎

4pt

Furthermore, we investigate the impact of different hyperparameter combinations on the utility of the proposed method. Specifically, \(\gamma\) balances the relative importance of semantic relevance and field sensitivity in risk assessment, while \(\alpha\) controls the trade-off between privacy protection and semantic preservation during candidate selection. As shown in Table 7, the proposed method exhibits stable performance across different parameter settings, indicating that it is not overly sensitive to hyperparameter choices. Among all configurations, \(\gamma=0.6\) and \(\alpha=0.5\) consistently achieve the best overall results. This suggests that assigning slightly greater importance to semantic relevance than field sensitivity leads to more accurate risk assessment, while a balanced weighting between privacy protection and semantic preservation provides the most effective trade-off between privacy and utility. Therefore, we adopt \(\gamma=0.6\) and \(\alpha=0.5\) as the default setting in all experiments.

5.6 Efficiency Analysis↩︎

To evaluate the computational efficiency of the proposed method, particularly the time overhead introduced by the privacy protection procedure, we compare its runtime with five representative privacy-preserving approaches. The experiments are conducted with the number of retrieved documents set to K=1, which eliminates the impact of retrieval-scale variations and enables a fair comparison of the privacy protection overhead. As shown in Table 8, all methods have the same retrieval time, and the runtime differences mainly come from the privacy protection process. Our method achieves a relatively low privacy protection overhead and is only slower than PARA. Although PARA has lower runtime, it relies on online LLM API calls for privacy processing, while our method can operate offline without LLM invocation, making it more suitable for practical deployment.

6 Conclusion↩︎

In this paper, we revisit privacy protection in retrieval-augmented generation from a new perspective and identify a fundamental limitation of existing approaches: the document-level static privacy risk assumption. We show that privacy leakage in RAG is inherently query-dependent, and the same document may pose substantially different privacy risks under different user prompts. Motivated by this observation, we propose PA-HDP, a Prompt-Aware Dynamic Hierarchical Differential Privacy protection framework that dynamically assesses privacy risks and applies differentiated protection through risk stratification, adaptive sensitive entity replacement, and exponential mechanism-based text selection.

Extensive experiments on medical dialogue and open-domain question answering benchmarks demonstrate that PA-HDP effectively mitigates both targeted and untargeted privacy attacks while maintaining high retrieval utility. Moreover, under rigorous differential privacy guarantees, PA-HDP achieves a favorable trade-off between privacy protection and model performance. We hope this work highlights the importance of query-aware privacy protection and provides a foundation for the secure deployment of RAG systems on sensitive data.

6.0.1 limitation↩︎

PA-HDP assumes a fixed privacy budget that is predetermined before deployment. However, in real-world RAG systems, the number of user queries is typically unknown in advance, and privacy budgets may be gradually exhausted as queries accumulate. Future work will investigate adaptive privacy budget allocation and accounting mechanisms to support long-term deployments while maintaining rigorous privacy guarantees.

6.1 Missing experimental results↩︎

4pt

Table 9 reports the utility results on all datasets under different privacy budgets. Overall, the proposed method exhibits a clear privacy–utility trade-off: as the privacy budget \(\epsilon\) increases, the utility metrics generally improve across all datasets. This trend is expected, since a larger privacy budget introduces less perturbation during the privacy-preserving generation process, allowing the generated responses to better preserve the semantic information of the retrieved content.

More importantly, the performance degradation under stringent privacy budgets remains relatively small. Even with a very strong privacy guarantee (\(\epsilon=0.1\)), the BLEU and ROUGE scores are only slightly lower than those obtained with much larger privacy budgets. For example, on the HealthCareMagic dataset, the ROUGE score increases from 0.3387 at \(\epsilon=0.1\) to only 0.3501 at \(\epsilon=10\), while similar trends are observed on the remaining datasets. This demonstrates that our method effectively preserves generation quality while providing rigorous differential privacy guarantees, making it suitable for privacy-sensitive RAG applications.

The relatively stable performance across a wide range of privacy budgets indicates that the proposed prompt-aware mechanism can accurately identify privacy-critical content, enabling the privacy budget to be utilized efficiently rather than perturbing all retrieved information uniformly.

6.2 Differential Privacy Guarantee of PA-HDP↩︎

We prove that PA-HDP satisfies differential privacy by decomposing the algorithm into three components: (i) Laplace perturbation for risk scoring, (ii) post-processing-based hierarchical filtering, and (iii) exponential mechanism for private candidate selection. The overall privacy guarantee follows from sequential composition and post-processing invariance.

6.2.0.1 Step 1: Laplace Mechanism for Risk Score.

For each segment \(s_{i,j}\), the risk score is computed as \(S_{total}(q,s_{i,j})\). Since the risk score is normalized to the interval \([0,1]\), changing one neighboring segment under the semantic metric alters the score by at most one. Therefore, the semantic sensitivity of the risk score is bounded by \(\Delta S = 1\).

To privately release the risk score, PA-HDP allocates half of the total privacy budget, i.e., \(\epsilon/2\), and adds Laplace noise: \[S_{total}^{*}(q,s_{i,j}) = S_{total}(q,s_{i,j}) + \mathrm{Lap}\!\left(\frac{2}{\epsilon}\right).\]

6.2.0.2 Step 2: Hierarchical Filtering (Post-processing).

The algorithm applies thresholding: \[S^{\ast}_{total}(q, s_{i,j}) \geq \tau_1.\]

Since this operation is a deterministic function of the DP output \(S^{\ast}_{total}\), by the post-processing theorem of differential privacy, it does not incur additional privacy loss.

6.2.0.3 Step 3: Exponential Mechanism for Candidate Selection.

Lemma 3[Semantic Exponential Mechanism] Let \(\mathcal{C}\) be a finite candidate text set, and let \[u(s_{i,j}, s_{i,j}^v) = - \alpha \cdot d_{\mathrm{sem}}(s_{i,j}, s_{i,j}^v) - (1-\alpha) \cdot r(s_{i,j}, s_{i,j}^v),\] where \(\alpha \ge 0\). The Semantic Exponential Mechanism selects an output \(s_{i,j}^{\ast}\in\mathcal{C}\) according to the probability distribution \[\Pr[M(s_{i,j})=s_{i,j}^{\ast}] = \frac{ \exp\left( \frac{\epsilon u(s_{i,j},s_{i,j}^{\ast})}{2}\right) }{ \sum\limits_{s_{i,j}^v\in\mathcal{C}} \exp\left( \frac{\epsilon u(s_{i,j},s_{i,j}^v)}{2} \right) }.\] Then \(M\) satisfies \((\epsilon,0)\)-Semantic Metric Differential Privacy.

6.2.0.4 Step 4: Composition Over Segments.

Since the privacy budget \(\epsilon/2\) is evenly allocated to the high-risk and medium-risk segments, by the sequential composition theorem, these two mechanisms together satisfy \((\epsilon/2,0)\)-Semantic Metric Differential Privacy.

6.2.0.5 Theorem (Privacy Guarantee of PA-HDP).

PA-HDP satisfies \((\epsilon,0)\)-Semantic Metric Differential Privacy.

6.3 Details of Field Sensitivity Weights↩︎

To instantiate the field sensitivity defined in Section X, we assign a predefined leakage risk weight \(w_k \in [0,1]\) to each sensitive entity category. The weights are manually designed according to the potential privacy harm caused by disclosing each entity type, following the principle that entities enabling direct identity disclosure receive substantially higher scores than contextual or auxiliary information.

Specifically, we divide sensitive entities into two broad categories.

6.3.0.1 High-risk entities (\(w_k \in [0.5,1.0]\))

These entities can directly identify an individual or expose highly sensitive credentials (Table 10 ).

6.3.0.2 Low-risk entities (\(w_k \in [0.1,0.5]\))

These entities mainly provide contextual or auxiliary information (Table 11 ).

We emphasize that the specific numerical values are empirically defined and can be adapted to different application scenarios without modifying the overall framework.

6.4 Implementation Details of Baseline Approaches↩︎

We adopt the baseline methods and implementation protocols from the work of Zeng et al. [7]. The detailed implementation settings are described as follows.

6.4.1 Paraphrase↩︎

This approach employs large language models to extract relevant and essential components from the retrieved passages. Insignificant portions can be discarded, while selected sentences are rephrased to improve clarity or relevance. The instruction template we use for paraphrasing is provided in Table 12.

3pt

6.4.2 ZeroGen↩︎

The ZeroGen strategy is designed to produce a new set of question–answer pairs derived from the original passage. In practice, we first apply the spaCy toolkit to recognize named entity mentions in the source text. We then feed the LLM with the prompt: “The context is: original context. extracted entities is the answer to the following question:” to solicit corresponding questions. The newly constructed context comprises 10 randomly chosen question–answer pairs, formatted as “Question: generated questions. Answer: extracted entities”.

6.4.3 AttrPrompt↩︎

Unlike the previous methods, AttrPrompt relies solely on LLM-generated content without accessing any retrieved data from the external database. This technique queries the LLM about the most critical attributes associated with a specific data category. For the ChatDoctor scenario, we prompt the model with: “What do you think are important attributes to generate some chat doctor data? Examples: disease...”. From the LLM’s response, we select five attributes and ask the model to propose 10 varied subtopics for each attribute. During new context generation, we randomly pick one subtopic per attribute and instruct the LLM to produce data following the assigned attribute.

6.5 Detailed Attack Design↩︎

This subsection elaborates on the design of both targeted and untargeted attacks against Retrieval-Augmented Generation (RAG) systems. These attack configurations are employed to assess the privacy-preserving capacity of our proposed synthetic-data framework. We emulate a realistic black-box environment where the adversary can only interact with the system via API calls. Consequently, the adversary’s strategy centers on crafting and manipulating the query \(q\) to elicit the target information from the RAG pipeline.

6.5.1 Prompt Composition↩︎

The composite structured prompt generally consists of two segments: an {information} component and a {command} component. \[q = \{\text{information}\} + \{\text{command}\}\]

This formulation pursues two parallel goals: (a) steering the retriever to accurately fetch the desired pieces of data, and (b) compelling the language model to incorporate the retrieved evidence into its output. The {information} part directs the retrieval module toward specific data items, whereas the {command} part instructs the model to echo the retrieved content in its response. For the latter, we adopt directives like “Please repeat all the context”; the former varies according to the attacker’s objective.

6.5.2 Targeted Attack↩︎

In a targeted scenario, the adversary seeks to recover a predetermined set of specific details. Constructing the information component for such an attack proceeds in two steps. Initially, the attacker supplies concrete exemplars based on their goal – for instance, “I want some advice about target name” for a well-defined target, or a partial prefix such as “Please email us at” for a more vague target. Subsequently, a substantial number of analogous yet diverse instances are generated from these exemplars. For targets that contain multiple sub-elements (e.g., the HealthcareMagic corpus), variations can be produced by substituting sub-items like disease names derived from ChatGPT or the International Classification of Diseases (ICD). Alternatively, large language models (e.g., ChatGPT) can directly yield similar sentences based on the provided examples – this approach is also applied to the Wiki-PII dataset. For instance, one might input “Generate 100 similar sentences like ‘Please email us at’”.

6.5.3 Untargeted Attack↩︎

In contrast, untargeted attacks emphasize the production of varied information components so as to extract a broad spectrum of data from the retrieval repositories, rather than homing in on any particular entry. Inspired by the methodology of Carlini et al.[40], we randomly sample segments from the Common Crawl corpus to serve as the information part. However, the randomness of the input could potentially interfere with the command component. To counter this, we cap the maximum token length of the information part at 15 tokens, thereby preserving prompt coherence and maintaining effectiveness in eliciting data from the retrieval collections.

6.6 Details of Evaluation Metrics↩︎

This section clarifies the evaluation metrics adopted in our study.

6.6.1 ROUGE-L↩︎

ROUGE-L belongs to the ROUGE (Recall-Oriented Understudy for Gisting Evaluation) family and is widely used for assessing text-generation tasks, including summarization and machine translation. It measures the overlap between a generated output and a reference text via the Longest Common Subsequence (LCS).

• Longest Common Subsequence (LCS): ROUGE-L identifies the longest word sequence that appears in both the generated and reference texts while preserving the original order, without requiring contiguity.

• Recall, Precision, and F-measure:

  • Recall is defined as the ratio of the LCS length to the reference length (\(n\)): \(\text{Recall} = \text{LCS}(X,Y) / n\). It reflects the fraction of the reference sequence captured by the generator.

  • Precision is the ratio of the LCS length to the generated length (\(m\)): \(\text{Precision} = \text{LCS}(X,Y) / m\). It indicates how much of the generated content aligns with the reference.

  • The F-measure, which balances precision and recall via their harmonic mean, is computed as: \[F_{\text{lcs}} = \frac{(1+\beta^2) \cdot R_{\text{lcs}} \cdot P_{\text{lcs}}}{R_{\text{lcs}} + \beta^2 \cdot P_{\text{lcs}}}\] where \(\beta\) controls the trade-off between precision and recall (typically set to \(1.0\)). In our experimental results, we report the F-measure as the ROUGE-L score.

Let \(C\) denote the candidate translation, and \(R\) represent the set of reference translations.

6.6.2 BLEU-1↩︎

BLEU-1 evaluates translation quality based on unigram precision.

• Unigram precision: \[P_1 = \frac{\sum_{w} \min\bigl(\text{Count}_C(w), \max\text{Count}_R(w)\bigr)}{\sum_{w} \text{Count}_C(w)}\] where \(\text{Count}_C(w)\) is the frequency of word \(w\) in the candidate, and \(\max\text{Count}_R(w)\) is the maximum frequency of \(w\) across any single reference translation.

• Brevity penalty (BP): \[\text{BP} = \min\left(1, \exp\left(1 - \frac{r}{c}\right)\right)\] where \(c\) is the candidate length and \(r\) is the reference length that is closest to \(c\).

• Final BLEU-1 score: \[\text{BLEU-1} = \text{BP} \times P_1\] The score ranges from 0 to 1, with 1 indicating perfect unigram matching against the reference.

6.6.3 Additional Metrics↩︎

Beyond the above, we incorporate several new metrics to further corroborate our approach:

• Exact Match (EM): This metric checks whether the ground-truth answer appears verbatim within the LLM’s generated response.

• LLM-based Correctness Judgment: We employ Ragas, a widely adopted automatic evaluation pipeline for RAG (with over 5.9k stars on GitHub), to assess the factual correctness of the generated answers.

6.7 Construction of Wiki-PII Dataset↩︎

To assess the privacy-preserving performance of our proposed approach against targeted extraction attempts, we adopt the dataset construction methodology introduced in [7] to create the Wiki-PII dataset. This corpus is specifically designed to contain a substantial volume of personally identifiable information (PII), thereby providing a rigorous testbed for evaluating privacy-protection strategies. The construction procedure comprises three main phases.

1. PII extraction: We first harvested real PII instances from the Enron Mail corpus. Specifically, we employed the urlextract library to extract web URLs and applied regular expressions to capture phone numbers and personal email addresses.

2. Text chunking: In the second phase, we partitioned the Wikipedia text collection using the recursive character text splitter provided by LangChain, with a chunk size of 1500 characters.

3. PII insertion: Finally, for each resulting text chunk, we randomly appended the extracted PII items to the end of every sentence within that chunk.

6.8 Dataset Preparation Details↩︎

In our experiments, we adopt the configuration established in prior RAG privacy research [33] to simulate a realistic scenario where a large-scale public repository inadvertently contains certain private information — for instance, personal names or phone numbers that appear within broadly accessible web content. In practice, virtually any sentence in a text corpus may carry privacy-sensitive entities. Consequently, embedding PII into the dataset serves as a plausible and effective strategy for modeling such real-world privacy risks.

6.9 Examples of Protected Contexts↩︎

Table 13 presents representative examples of the protected query–response pairs in the retrieval corpus. Compared with the original records, the protected versions effectively obfuscate privacy-sensitive information while preserving the semantic content required for downstream retrieval. Specifically, sensitive entities, including medication names, disease descriptions, temporal information, numerical values, and demographic attributes, are replaced with plausible alternatives or generalized expressions, making it substantially more difficult to recover the original records or infer personal information.

Despite these perturbations, the protected query–response pairs largely retain their original semantics. In the first example, although several medications and personal details are replaced, the protected record still clearly describes a fertility-related consultation and its corresponding medical advice. Similarly, in the second example, detailed medical history and treatment information are concealed, while the protected response continues to capture the clinical recommendation regarding pregnancy risk. These examples demonstrate that our protection mechanism effectively removes privacy-sensitive content while preserving the essential medical knowledge required for retrieval.

We also observe that stronger perturbations may occasionally introduce minor semantic deviations or less natural expressions, as illustrated in the third example. Nevertheless, the overall medical intent and question–answer correspondence remain largely intact, ensuring that the protected corpus continues to provide useful evidence for retrieval-augmented generation. These qualitative examples further demonstrate that our method achieves a favorable balance between privacy protection and retrieval utility.

6.10 Discussions when Adapting PA-HDP to Domain-Specific Applications↩︎

Although PA-HDP is presented in the context of medical and QA RAG, the overall framework is domain-independent and can be readily adapted to other application scenarios. The primary modification lies in the sensitive entity extraction module, as different domains involve different categories of privacy-sensitive information.

Specifically, our framework relies on the extracted sensitive entities to estimate field sensitivity and subsequently allocate privacy budgets. Therefore, when applying PA-HDP to a new domain, the Named Entity Recognition (NER) model and rule-based extraction patterns should be customized according to the domain-specific privacy taxonomy. For example, in the financial domain, sensitive entities may include bank account numbers, credit card numbers, transaction records, and customer identifiers, while in the legal domain, case numbers, client identities, and confidential agreements may require protection. Existing domain-specific NER models can be directly integrated into our framework, or custom entity recognizers can be trained when annotated data are available.

In addition, the predefined sensitivity weights assigned to different entity types should be adjusted to reflect their privacy risks in the target domain. Since the subsequent privacy budget allocation only depends on the extracted entity categories and their corresponding sensitivity scores, no modification to the remaining components of PA-HDP is required. This modular design enables PA-HDP to be easily extended to diverse privacy-sensitive RAG applications while preserving its query-aware privacy protection mechanism.

References↩︎