1 Introduction↩︎

For most of their short history, large language models were used as chatbots: a user typed a question, the model produced an answer, and the interaction ended. Privacy questions in that setting concerned what the model had memorized from its training corpus and whether it would repeat it [1]–[4]. The interaction left no state, touched no external data, and took no action in the world.

This setting no longer describes how language models are deployed. Equipped with the ability to reason about a goal, call tools, and observe the results [5], [6], language models now operate as agents that pursue multi-step tasks with little supervision, carry memory across turns and sessions [7], and coordinate with other agents [8], [9]. A large share of these agents exist to work with data. They translate a request into a database query, retrieve passages from a document collection, join results across sources, summarize what they find, write intermediate findings to memory, and act on the result. We refer to such systems as data agents: LLM-driven agents whose purpose is to query, retrieve, transform, remember, and act on data on a user’s behalf [10]–[12]. Data agents already drive enterprise analytics over warehouses [13], [14], scientific and data-science question answering [15], and personal assistants that read a user’s files, mail, and calendar.

The data these agents touch is sensitive. A clinical agent reads patient records, an enterprise analytics agent reads payroll and customer tables, and a personal assistant reads private correspondence. When a model only answered questions, privacy could be enforced at one place, the final answer, by filtering what the model was about to say. A data agent eliminates this single enforcement point for five reasons. First, it touches many heterogeneous data sources at once: relational databases, vector indexes built over private documents, files, external APIs, and its own memory, each exposing data in different ways (Figure 1). Second, it runs multi-step workflows in which sensitive information flows through intermediate artifacts long before any answer is produced: the generated query, the retrieved rows, the arguments passed to a tool, and the text written to a scratchpad, each of which can expose sensitive information [16], [17]. Third, it keeps persistent state: data written to memory in one session can surface in another, to a different user [18], [19]. Fourth, it acts with delegated authority, often holding broader permissions than any single task requires, so a compromised or misdirected agent can reach far more data than its current request [20], [21]. Fifth, every protective transformation trades against utility: the more an agent redacts, abstracts, or withholds, the less useful its answers become, and that tension must be managed rather than resolved once [22], [23]. Privacy for data agents is therefore not a property of a single output. It is a property of an execution that spans data sources, intermediate channels, memory, and actions.

Each of these risks has an established literature, but these literatures have developed largely in isolation, in research communities with little overlap. Work on natural language interfaces to databases measures and defends against sensitive data leaking through generated SQL [22], [24], [25]. Work on retrieval-augmented generation shows that private corpora can be extracted or have their membership inferred through an agent’s queries [26]–[28]. Work on agent memory demonstrates extraction and poisoning of stored state [18], [19], [29]. Work on prompt injection shows that untrusted data and tools can hijack an agent into exfiltrating what it sees [30]–[32]. A separate line builds access control and information-flow control for agents [33]–[35], and another studies whether agents respect contextual norms of appropriate information flow [36]–[38]. None of them organizes its findings around the data the agent handles, and so their combined account of an end-to-end data agent remains fragmented.

We unify these lines under a single, data-centric view of LLM agents. Rather than organize by attack type, as broad agent-security surveys do [39]–[41], we organize around the data itself: where it resides, the privacy risks it incurs, and how to govern it. Concretely, we make the following contributions.

• We motivate a data-centric view of agent privacy and explain why an LLM agent that works with data, which we call a data agent, has privacy properties different from those of stateless chatbots and generic tool-using agents (Sections 1 and 2).

• We give three taxonomies and the tables that connect them: the data surfaces a data agent touches (Section 3), the privacy risks each surface creates (Section 4), and the governance mechanisms that address them (Section 5). Three tables connect the taxonomies: surfaces to the disclosure outcomes they create (Table 3), outcomes to the vectors that trigger them (Table 4), and outcomes to the controls that address them (Table 5). They are intended as a reusable reference for both system builders and reviewers.

• We map the benchmarks that measure these risks, compare what each one covers, and identify the gap that no current benchmark fills: an end-to-end data-agent workflow evaluated under an explicit privacy policy (Section 6).

• We survey where data agents are deployed (Section 7) and set out the open problems for the field (Section 8).

Two practical conclusions recur. A builder who can adopt only one advanced mechanism should adopt information-flow control, the single mechanism in our map that covers both compositional leakage and inference across a session. And the instrument the field most lacks is an end-to-end benchmark that drives an agent across its data surfaces under one explicit privacy policy; we argue for it in Section 6.

Several recent surveys treat the security and privacy of LLM agents broadly and organize by attack type [39]–[42]. Two works share our premise that agent privacy extends across the whole execution rather than the final output: a position paper that reframes privacy beyond training-data memorization [43], and a survey that decomposes agentic data leakage by architectural component [44]. Closest in scope is a literature review of data privacy for language models and their agents [45], which organizes by privacy threat and protection technique across the model lifecycle and treats the data-systems surfaces a data agent spans, text-to-SQL interfaces, database access control, and retrieval-index attacks, only briefly. A systematization of retrieval-augmented generation privacy treats one of our data surfaces in depth [46]. We differ from all of these in what we organize around. They treat the agent as a general language model with tools and group findings by component, attack, or mechanism. We organize by the data the agent works with, and we cover the data-systems literature these surveys treat only in passing: text-to-SQL privacy, database access control, retrieval and index attacks, and enterprise data governance. In doing so we connect two research communities that have approached agent privacy separately: agent security and data management. Table 1 summarizes the difference.

The rest of this survey is organized as follows. Section 2 provides background on LLM agents, data agents, and the privacy foundations the rest of the survey draws on. Section 3 taxonomizes data surfaces, Section 4 the privacy risks over those surfaces, and Section 5 the governance mechanisms that address them. Three tables connect them: surfaces to the outcomes they create (Table 3), outcomes to the vectors that trigger them (Table 4), and outcomes to the controls that address them (Table 5); we omit a surfaces-to-controls table because Section 5 notes each mechanism’s surface applicability in prose. Section 6 surveys evaluation and benchmarks, Section 7 the application domains, and Section 8 the open problems. Section 9 concludes. Figure 2 summarizes this organization.

Scope and method. We survey work at the intersection of LLM agents and data privacy appearing primarily between 2023 and 2026, drawn from venues in security and privacy, data management, and machine learning and natural language processing, together with arXiv preprints reporting results not yet in proceedings. We include a work if it measures, attacks, or defends a privacy property of a system in which a language model reads from or writes to an external data store. We exclude work on training-data memorization in standalone models, except where it bears on an agent’s data surfaces. Because the field moves quickly and much relevant work exists only as preprints, we treat preprint results as provisional and note where a result has not been independently reproduced.

2 Background↩︎

This section fixes terminology. We describe what an LLM agent is, what distinguishes a data agent, and the privacy foundations the later sections build on. Readers familiar with agent architectures may skip to Section 2.3.

2.1 LLM agents↩︎

An LLM agent couples a language model with the ability to act. Rather than emit a single response, the model is placed in a loop in which it reasons about a goal, chooses an action, observes the outcome, and repeats until the task is done. The ReAct framework [5] established the basic pattern of interleaving reasoning traces with actions on an external environment, and tool-use methods showed that models can learn when to call external functions and incorporate their results [6]. Four capabilities recur across agent designs and matter for privacy. Tool use lets the agent read and write external systems, which is how it accesses data beyond its prompt. Memory lets it store and recall information across steps and sessions, as in the generative-agent architecture that records and reflects on past experience [7]. Planning lets it decompose a goal into a sequence of actions whose intermediate steps the user never sees. Multi-agent collaboration lets several agents exchange messages and divide labor [8], [9]. Each capability enlarges the set of locations where data is read, written, or transmitted, and the later sections trace privacy risk to exactly these capabilities.

2.2 Data agents↩︎

A data agent is an LLM agent whose task is to work with data: to query, retrieve, transform, join, summarize, and act on it. The defining feature is that data, rather than open-ended conversation, is both the input and the object of the work. Recent treatments place data agents on a spectrum of autonomy. Luo et al. [10] propose levels from a natural language interface that translates a single request into a query, through agents that plan and execute multi-step analyses, to agents that proactively decide what to analyze. Surveys of agents as data analysts and of data-science agents catalog the capabilities along this spectrum [11], [47], and a growing set of systems implements them: agents that perform end-to-end data science [15], [48], agentic analytics over heterogeneous stores [13], and enterprise text-to-SQL over real warehouses [14]. A survey of these systems finds that the great majority are released without explicit trust or safety mechanisms [47], which is part of what motivates this survey. Whether data agents are a durable paradigm or a transient term is itself debated [49]; our claims depend only on the existence of systems that query, retrieve, and remember over sensitive data, not on the durability of the term.

Autonomy level matters for privacy because it determines how much of the data work happens without the user observing it. A non-autonomous interface issues one query the user could read. A higher-autonomy agent constructs its own queries, chooses which sources to join, retrieves more than it returns, and composes results across steps, all without a human inspecting each action. The set of data stores on which privacy must be enforced grows with autonomy, and the parts of those stores the user can directly observe shrink.

2.3 Privacy foundations↩︎

The mechanisms surveyed in Section 5 draw on several foundations developed long before LLM agents, each in data management or formal privacy. We introduce them here so that later sections can refer to them without re-deriving them.

Data minimization and purpose limitation are the governing principles of data-protection regulation: collect, retain, and disclose no more than is necessary for a stated purpose. Hippocratic databases first proposed building these principles into the data system itself, with purpose and consent attached to data and enforced at access time [50].

Access control decides who may see what. Role-based access control assigns permissions to roles, attribute-based access control conditions them on attributes of the requester and resource, and purpose-based access control conditions them on the purpose of the access, with the access checked against the purpose for which data was collected [51]. These models specify access at the level of rows, columns, and documents.

Information-flow control tracks data as it moves rather than gating it once at the source. Each value carries a label, the label propagates as data is combined, and a policy constrains where labeled data may flow. Sticky policies extend this idea across organizational boundaries by attaching usage constraints that travel with the data [52].

Differential privacy gives a formal, composable guarantee on what a release reveals about any individual: a mechanism is differentially private if its output distribution barely changes when one record is added or removed [53]. Its two properties that matter here are clean composition of privacy loss across multiple releases and an explicit parameter that trades privacy against accuracy. Policy-aware refinements such as Pufferfish [54] and Blowfish [55] let the guarantee be tuned to what a deployment needs to protect.

Utility-first privacy concerns how the privacy-utility trade-off is navigated. The privacy-first paradigm, used in most production deployments, fixes a privacy budget in advance and returns the most accurate answer that budget permits. The utility-first, or accuracy-first, paradigm inverts the relationship: the analyst specifies a required level of utility and the mechanism spends the least privacy that meets it [56]–[59]. This is the natural fit when the answer drives a consequential decision, since it guarantees the quality of that answer while minimizing disclosure, and the line has been extended to complex multi-condition queries [60]. The same tension reappears for data agents, which must satisfy an analytical requirement while disclosing as little as possible, and we return to it in Sections 5 and 8.

Contextual integrity frames privacy as appropriate information flow: whether a disclosure is acceptable depends on the context, the roles of sender and recipient, and the norms governing transmission, not on secrecy alone [61]. Efforts to crowdsource and formalize these norms aim to turn them into rules a system can check [62]. Contextual integrity supplies the vocabulary for judging whether an agent that is technically permitted to share a piece of data should share it in a given situation.

These foundations were designed for data at rest and for one-shot releases. A data agent shifts the disclosure being protected from a single release to a multi-step interaction over live data, which is why the later sections revisit each foundation in the agent setting rather than applying it unchanged.

3 Data Surfaces↩︎

A data agent does not hold sensitive data in one place. It reads and writes across several distinct stores, each with its own structure, its own access model, and its own way of leaking data. We call these stores data surfaces. We distinguish surfaces by their native access model and characteristic way of leaking data rather than by storage format, so a relational table and a flat file are separate surfaces because one carries an access policy and the other does not. Two of the six, agent memory and the inter-agent channel, are both stores and transports; we keep them as surfaces because each holds a distinct sensitive unit and exposes it through a distinct leakage path, and we flag where they also serve as channels. This section enumerates the surfaces a data agent touches and, for each, names the sensitive unit it holds and how the agent interacts with it. Table 2 summarizes the enumeration; the risks specific to each surface are deferred to Section 4 and the controls to Section 5.

Databases and warehouses. The canonical data agent translates a natural language request into a query over a relational database or warehouse and returns the result [64], [65]. The sensitive units are the rows and columns of the underlying tables, but the schema is sensitive too: table and column names, types, and relationships reveal what an organization records about people. The agent touches this surface by constructing a query, choosing predicates and join keys, executing it, and reading the rows it returns. Each of these is a point of potential disclosure, because the generated query encodes the agent’s intent and the returned rows carry the data itself. Real text-to-SQL systems already run against enterprise warehouses with multi-step workflows [13], [14], so this surface is already in production use.

Tabular and file data. Many data agents work over files directly: spreadsheets, CSVs, logs, and documents loaded into a dataframe and computed over in code [15], [48]. The sensitive units are the individual records and fields, and, for free-text documents, the contents themselves. Unlike a database, file data usually carries no access model of its own, so the agent can read in full any file it can open. The agent touches this surface by loading files into its working environment and executing code over them, which means sensitive values pass through generated code and intermediate variables as well as the final summary.

RAG corpora and vector stores. Retrieval-augmented generation grounds an agent’s answers in a private corpus by embedding documents into a vector index and retrieving the passages nearest to a query [26]. The sensitive units are the passages, the embeddings that encode them, and the retrieval scores that rank them. The agent touches this surface by embedding its query, retrieving the top passages, and conditioning its answer on them. The corpus is exposed in three distinct ways: through the passages placed into context, through the embeddings, which can be inverted to recover text, and through the scores, which reveal whether a given document is present. Each of these is the subject of its own attack literature, recently systematized for retrieval systems [46] and surveyed as one surface among several in Section 4.

Tools and external APIs. Beyond data stores, a data agent calls tools: search functions, calculators, internal services, and increasingly other services reached through standard agent protocols. These tools increasingly arrive packaged as third-party components: browser plugins and custom GPTs whose bundled instructions and data access the user rarely audits [66], and increasingly Model Context Protocol servers and agent skills. The sensitive units are the arguments the agent passes, the responses it receives, and the side effects the calls produce, such as sending mail. The agent touches this surface whenever it invokes a function, and the arguments it constructs often carry sensitive data assembled from other surfaces. Tools are also the surface most exposed to untrusted input, because a tool’s response can carry attacker-controlled instructions [30], [32].

Agent memory. To act coherently over time, an agent writes selected information to a memory store and recalls it later [7]. The sensitive units are the stored facts, prior results, and accumulated user history. Memory differs from the other surfaces in two ways that heighten its risk. It persists across sessions, so data written for one user or task can resurface for another [18], and it is writable by the agent during operation, so untrusted input can plant content that is recalled later [19], [29]. Memory is therefore both a store of sensitive data and a vector through which other surfaces are attacked. Persistence also creates an obligation the other surfaces do not, to delete or revoke data already written, which no surveyed control yet meets (Section 8).

Multi-agent communication. When several agents collaborate, they exchange messages and share context, and that channel is itself a surface [8]. The sensitive units are the inter-agent messages and any shared working state. The agent touches this surface by sending results to peers and forwarding what it receives. This internal channel is frequently overlooked because it is invisible to the end user, yet information benign in isolation can become disclosing once several agents pool what they hold [63], [67]. When the agents belong to different organizations, this channel also crosses a trust boundary, so each party observes only its own side of the exchange.

Two properties characterize the enumeration. First, the surfaces are heterogeneous: a row in a database, an embedding in a vector index, and a fact in memory leak in different ways and need different controls. Second, several surfaces are both stores and channels: memory and the inter-agent channel hold sensitive data and also serve as paths by which data from other surfaces is moved or attacked. The next section turns from which surfaces hold data to the risks specific to each surface.

4 Privacy Risks↩︎

This section surveys the privacy risks that arise over the surfaces of Section 3. We first fix a threat model so that each risk can be stated against a defined adversary. We then separate two orthogonal axes that the literature often conflates: the outcome, what information is disclosed, and the vector, how the disclosure is triggered. Section 4.2 takes the outcomes in turn and Section 4.3 the vectors, and Table 4 records which vector can trigger which outcome. Figure 3 previews where these risks arise along a typical execution.

4.1 Threat model↩︎

Privacy risks in data agents come from adversaries in several positions, with different goals and different views of the execution. By intent, an adversary may be honest but curious, a data or tool provider that follows the protocol while trying to learn more than it should, or malicious, actively trying to extract or exfiltrate data. By position, the adversary may be an external party that interacts with the agent only through its inputs and outputs, an internal over-permissioned user or agent that already holds some access, a compromised peer agent in a multi-agent system, an untrusted tool or external service the agent calls, or a poisoned document or memory entry the agent reads. By observation, the key distinction is whether the adversary sees only the final answer or also the intermediate channels: the queries, retrieved results, tool arguments, reasoning traces, memory writes, and inter-agent messages. Much of what follows depends on this last point, because controls that inspect only the final output miss adversaries that observe or inject on the intermediate channels. Unless an outcome notes otherwise, the outcomes below assume an honest-but-curious holder of a surface, while the vectors of Section 4.3 assume a malicious party that can plant content the agent reads. The control matrix of Section 5 inherits this assumption, and we note where a control that holds against a curious provider fails against a compromised peer.

4.2 Disclosure outcomes↩︎

An outcome is the kind of sensitive information a disclosure reveals, independent of how it was triggered. Six recur for data agents.

Direct output leakage. The simplest outcome is that the agent states sensitive data in its answer to a party not entitled to it. This is the failure mode inherited from chatbots, and it persists for data agents because the agent now has direct access to private stores rather than only to its training data. Benchmarks for natural language interfaces to databases show that models will return data they should have withheld when asked in the right way [24], and studies of conversational agents show frequent disclosure of information that the context made inappropriate to share [36]. Current output filters target this outcome, so the remaining outcomes, which bypass the output, are the ones this section emphasizes.

Intermediate-step leakage. A data agent produces a trail of intermediate artifacts before it answers, and each carries sensitive data. The generated query encodes which records the agent sought, the retrieved rows hold the data itself, the arguments passed to a tool may contain assembled personal information, and the reasoning trace can restate any of it. An adversary positioned on one of these channels, a logging system, a compromised tool, or a peer agent, reads sensitive data that never appears in the final answer, and the prompt-injection vector of Section 4.3 routinely turns such a channel into an exfiltration path. A full-stack benchmark of multi-agent systems finds that internal channels raise total system exposure well above the output-only level, so an audit that inspects only the answer misses much of it [16]. Emulated-sandbox testing surfaces such failures, including private-data leakage, at scale by running an agent against simulated tools and scoring the outcomes [68].

Schema and metadata leakage. Sensitive information leaks not only from data values but from the structure around them. The schema of a database, its table and column names, types, and relationships, reveals what an organization records, and an agent that exposes it through generated queries or error messages leaks that structure even when no row is returned. Schema inference attacks reconstruct hidden database schemas by probing a text-to-SQL model, recovering table and column information at high accuracy [69]. For retrieval systems, the analogous metadata is the structure of the index and the retrieval scores, which reveal which documents exist and how they relate to a query.

Inference and aggregation leakage. An agent can disclose sensitive information it never returns verbatim, by letting an adversary infer it. Membership inference determines whether a particular record is present in a store: attacks against retrieval-augmented generation decide whether a target document is in the corpus using only ordinary queries, including stealthy variants that evade detection with few queries [27], [70]. Reconstruction recovers content from derived artifacts: embeddings can be inverted to recover the text they encode, and prompt-injection-driven extraction can pull large fractions of a private corpus out of a deployed system [28], [71]. Stored memory is no exception: extraction attacks craft inputs that make an agent reveal memory contents, including records belonging to other users, so data written in one interaction surfaces in another [18]. Aggregation is the data-systems version of the same problem: statistics released from a single store can, in combination, expose an individual record even when each statistic is within policy, which is the classic reason aggregate releases are regulated [53]. We reserve leakage that accumulates across steps, sessions, or agents for compositional leakage below.

Compositional leakage. Multi-step and multi-agent operation creates an outcome that no single step exhibits, generalizing the aggregation problem above from a single store to a whole execution. Responses that are each within policy can, in combination, disclose something none of them does alone, whether the combination happens across the steps of one agent or across the messages of several. The distinction from inference is the number of releases involved: a single response that lets an adversary deduce an unseen attribute is inference, while a disclosure that emerges only from combining two or more responses is composition. A study of multi-agent collaboration defines and measures this compositional leakage and shows that benign individual contributions add up to a violation [67], and a benchmark in which private information is essential to the shared task finds that state-of-the-art agents leak a large fraction of it during legitimate collaboration [63]. Formal treatments bound how local leakage accumulates along a sequential pipeline of agents [72]. Composition is the outcome most specific to agentic operation and the least addressed by per-step controls.

Policy and consent violations. A disclosure can be a privacy violation even when it leaks nothing in the information-theoretic sense, if it sends the right data to the wrong recipient, for the wrong purpose, or against a subject’s consent. This is the contextual integrity view of privacy [61], and benchmarks built on it show that agents frequently take actions that violate the norms of the context they operate in, sharing data that is accurate and available but inappropriate [38], [73]. Studies of real third-party LLM app ecosystems trace how personal data flows to parties a user did not anticipate, and how poorly app behavior matches stated policy [66]. For data agents this maps directly onto regulatory obligations such as purpose limitation and consent, which the governance mechanisms of the next section must enforce.

4.3 Attack vectors↩︎

An attack vector is how a disclosure is triggered. Four recur for data agents, and each can produce several of the outcomes above; for each we note the control that counters it.

Prompt injection. Data agents read untrusted content: web pages, retrieved documents, tool responses, and the messages of other agents. Indirect prompt injection embeds instructions in that content so the agent follows the attacker rather than the user [30]. Its natural target is the data the agent can reach: injection benchmarks include data exfiltration as a primary attack class [31], [32], a simple injection suffices to redirect an agent’s data access into an exfiltration channel [17], and for database agents it specializes into prompt-to-SQL injection [74] and text-to-SQL backdoors [75]. Runtime governance and information-flow control are the matched controls.

Memory poisoning. An adversary plants content in an agent’s memory or knowledge base through a normal interaction, without write access, so that it is retrieved and acted on later [19], [29]. Because memory persists and is writable during operation, poisoning links sessions and turns a one-time plant into a standing influence over later outcomes. Memory scoping and provenance counter it.

Over-permissioned access. An agent that holds broader permissions than any single task requires turns a misdirected or compromised step into access far beyond the request [20]. The remedy is least-privilege access control that scopes the agent to the task at hand [21].

Compromised peer. In a multi-agent system, a malicious or subverted peer can actively solicit data over the inter-agent channel, accumulating across messages what no single one reveals [16]. Information-flow control over inter-agent messages is the matched control.

Outcomes and vectors are not independent: the same injection that exfiltrates a result, an intermediate-step outcome, can also probe a schema or pool data across agents. Schema and metadata leakage is the exception with no primary trigger in Table 4, since it arises chiefly from the agent’s own generated queries and error messages rather than from an adversary’s action. Table 3 maps outcomes onto the surfaces of Section 3 and Table 4 onto the vectors that trigger them; together with the control map of Section 5 they let a deployment determine, for a given surface, which outcomes arise, how they are triggered, and what addresses them.

5 Governance Mechanisms↩︎

This section surveys the mechanisms that govern privacy in data agents. We group them by what they do rather than by which community produced them, and for each we note the risks of Section 4 it addresses and the surfaces of Section 3 it applies to. Table 5 collects the mapping. No single mechanism covers the full range of risks, which is the central observation the table is intended to convey. Figure 3 shows where each mechanism acts along the agent’s execution, paired with the risk it addresses.

Access control. Access control decides which data the agent may reach, and it is the primary control because it bounds exposure before any leak can occur. Classical role, attribute, and purpose-based models apply directly, enforced at the level of rows, columns, and documents [51]. The open question for agents is whether the model itself can be trusted to respect a policy when it constructs queries. Benchmarks of role-based access reasoning find that even strong models handle conflicting permissions and role-conditioned access poorly [76], [77], and access-aware text-to-SQL checks generated queries against a policy before execution rather than trusting the model to self-enforce [25]. A line of work argues that static allow-or-deny is the wrong frame for agents and recasts access control as context-dependent governance over information flow [21], with attribute-based frameworks and mandatory controls proposed to contain over-permissioned and privilege-escalating agents [20], [78].

For enterprise deployments, agent access control connects to a longer line of data governance the generic agent literature seldom addresses. Purpose-based access control has been enforced at warehouse scale through automatically generated masking views that bind each access to its declared purpose [79], extending the Hippocratic-database principle of attaching purpose and consent to data and checking them at the point of use [50], [51], and automated tools check data handling against regulation [80]. A data agent that constructs its own queries must operate within this governance rather than adjacent to it, so that the policy the warehouse already enforces also binds the queries the agent generates. This is the intersection of agent autonomy and enterprise data governance, addressed by neither the agent-security surveys nor the data-governance literature in isolation.

Information-flow control. Where access control gates data at the source, information-flow control follows it through the execution and constrains where it may go. Each value carries a label, the label propagates as data passes through tool calls and memory, and a policy blocks flows that would send labeled data to an unauthorized sink. Recent systems apply this discipline to agents: a formal model labels an agent planner’s data with confidentiality and integrity levels and enforces flow policies deterministically [33], a capability-based design extracts control and data flow from a trusted plan and provably blocks exfiltration through unauthorized flows [34], earlier system-level work disaggregates the agent so a monitor can enforce flow properties [81], and a tool-based agent system attaches confidentiality labels to data and gates the tool calls that would leak them [82]. Information-flow control is the mechanism best matched to intermediate-step and compositional leakage, because it reasons about the path data takes rather than only its final destination.

Runtime governance and policy enforcement. A complementary line places a deterministic enforcement layer around the agent that checks each action against a policy at run time. Programmable privilege-control systems confine an agent’s tool calls to a least-privilege policy and cut injection-driven attack success sharply [35], runtime-enforcement languages express safety and privacy constraints that a monitor evaluates as the agent acts [83], execution-isolation architectures confine third-party agent components and mediate their interactions [84], and production guardrail frameworks combine injection detection with action checks [85]. A guard agent translates natural-language safety and access rules into executable checks on another agent’s actions [86], and dedicated execution environments interpose on an agent’s data access to enforce user-data policies [87]. At the prompt level, lighter defenses separate trusted instructions from untrusted data through structured queries [88], by marking untrusted spans so the model discounts them [89], and through design patterns that constrain what an agent may do with untrusted input [90]. The shared principle is to move enforcement off the model and onto infrastructure that does not depend on correct model behavior, and to attach policies to execution paths rather than single calls [91].

Privacy-preserving transformations. Rather than block data, transformations reduce its sensitivity before it is used. Redaction removes sensitive spans, pseudonymization replaces identifiers, abstraction replaces a value with a less specific one, and summarization retains the substance while dropping detail. Abstraction has been applied to text-to-SQL to protect sensitive constants while preserving the structure a query needs [22]. Synthetic substitution goes further by replacing private records with generated substitutes, applied to retrieval corpora to retain utility while cutting leakage [92]. The common limitation is that a transformation strong enough to protect can be strong enough to break the task, which is a concrete instance of the privacy-utility tension and the reason these mechanisms combine with the negotiation discussed below.

Differential privacy and formal privacy. For releases that aggregate over many records, differential privacy gives a formal bound on what the release reveals about any individual [53]. It has been adapted to the agent’s data surfaces: differentially private retrieval-augmented generation spends a privacy budget over retrieved content [23], and differentially private synthetic generation builds a reusable private corpus an agent can query freely [93]. In-context and prompt-level variants extend the guarantee to how private examples are used at inference time [94], [95], differentially private fine-tuning trains the model itself under a budget [96], and the resulting guarantee can be empirically audited rather than only claimed [97], [98]. The principal difficulty for agents is composition: a multi-step session issues many queries, and the privacy budget must be accounted across the whole session rather than per query, which is an open problem we return to in Section 8.

Confidential and cryptographic retrieval. When the data store or the model runs on infrastructure the data owner does not trust, cryptographic and hardware mechanisms keep data confidential in use. Private information retrieval hides which document an agent retrieves, applied to retrieval-augmented generation so the index learns nothing about the query [99]. Trusted execution runs the retrieval pipeline inside hardware enclaves so queries and documents stay confidential across trust boundaries, including in federated deployments [100]. End-to-end designs aim to keep both documents and queries confidential in cloud-hosted retrieval [101]. A lighter, query-monitoring defense detects and blocks queries that probe whether a specific document is present, exploiting that such queries sit abnormally close to a single target [102]. These mechanisms address the case where the threat is the infrastructure itself rather than the agent’s behavior.

Contextual privacy reasoning. Some violations are not about secrecy but about appropriateness, and addressing them requires the agent to reason about context. A line of work builds this capability and measures it: benchmarks ground contextual integrity in synthetic scenarios and in privacy law [103], [104], methods steer an agent to share only what the context permits, as in data minimization driven by contextual integrity [37], [105], and reinforcement learning trains the reasoning into the model rather than only measuring it [106]. A position paper cautions that contextual integrity is often applied to language models in ways that depart from the theory, which matters for how these results should be read [107]. A complementary direction uses a multi-agent pipeline that decomposes the judgment into extraction, classification, and a final check, reducing contextual leakage relative to a single pass [108].

Human oversight and privacy-utility negotiation. Because every protective transformation costs utility, some decisions are best returned to a person, and the trade-off itself can be negotiated rather than fixed. Human-in-the-loop approval gates high-stakes actions before they take effect. The trade-off can also be made an explicit, policy-bounded quantity: policy-driven privacy definitions let a deployment state which trade-offs are acceptable rather than fixing one in advance [54], [55], and utility-first mechanisms invert the usual order by fixing a utility requirement and spending only as much privacy as it demands [56], [58]. The same idea extends naturally to an interactive setting, in which a data consumer could request controlled relaxation of a privacy parameter within policy-set bounds so that utility is recovered without overriding a subject’s choice. We are not aware of a deployed agent mechanism that negotiates the privacy-utility trade-off in this way, and we return to it as an open problem in Section 8.

Auditing, provenance, and accountability. The mechanisms above prevent or limit leaks; auditing makes them accountable after the fact. Recording what data the agent accessed, what it passed where, and under which policy supports detection, forensics, and the evidence a deployment needs to demonstrate compliance. Detection of data over-exposure across an agent’s tool chain uses regulation-grounded tracking to flag when an agent forwards more than a task needs [109], and provenance that travels with data, as in sticky policies, lets later stages check the constraints attached to what they received [52]. Auditing is necessary but not sufficient: it observes violations rather than preventing them, which is why it complements the preventive mechanisms above rather than replacing them.

Table 5 shows that the outcomes are unevenly covered. Direct output leakage and policy violations have many applicable mechanisms, while compositional leakage and inference over a session are addressed by only a few, and those few are the least mature. The table also shows that no row dominates: a deployment needs several mechanisms, chosen so that their coverage overlaps on the outcomes its data and threat model make most acute. One obligation falls outside the table: no surveyed mechanism deletes or revokes data already written to memory or an index, a gap we return to in Section 8. How well these mechanisms actually work, and how we would know, is the subject of the next section.

6 Evaluation and Benchmarks↩︎

A claim that a data agent protects privacy is no stronger than the evaluation supporting it. This section identifies what should be measured, surveys the benchmarks that measure parts of it, and identifies the gap that none of them addresses.

6.1 Dimensions of Evaluation↩︎

The risks of Section 4 imply a checklist that a complete evaluation would cover. It would measure leakage not only in the final output but on the intermediate channels, the queries, tool arguments, reasoning traces, and inter-agent messages, which carry exposure the output alone does not [16], [17]. It would measure leakage from memory across sessions, exposure of schema and index metadata, success of inference and reconstruction attacks, and resistance to injection-driven exfiltration. It would measure policy compliance, whether the agent respects purpose, recipient, and consent constraints, not only whether it withholds secrets. It would measure all of this against utility, since a trivial agent that returns nothing leaks nothing, and it would account for the user burden that privacy controls impose. Few evaluations cover more than two or three of these dimensions. Table 6 scores the three that most separate the field, surface coverage, multi-step execution, and evaluation against an explicit policy, and the prose notes where a benchmark also reaches the others.

6.2 Existing Benchmarks↩︎

Benchmarks exist for most individual risks, each strong within its scope. Table 6 maps the main families against the surface they exercise, whether they evaluate a multi-step or multi-agent execution, and whether they evaluate against an explicit privacy policy rather than a generic notion of sensitivity.

For databases, SecureSQL evaluates whether a natural language interface leaks sensitive data through generated queries [24]. For retrieval, membership-inference and extraction benchmarks measure whether a corpus can be detected or extracted through ordinary queries [27], [28]. For tool use, AgentDojo and InjecAgent evaluate agents under prompt injection with data exfiltration as an attack class [31], [32], Agent Security Bench formalizes a broad attack and defense suite [110], an emulated sandbox surfaces high-stakes failures including leakage without live tools [68], and a harmfulness benchmark scores whether an agent refuses malicious multi-step requests [111]. For memory, the extraction attack that defines MEXTRA doubles as a benchmark for memory leakage [18]. For multi-agent systems, AgentLeak measures leakage across internal channels and MAGPIE measures it during collaboration on tasks where private data is essential [16], [63]. For contextual norms, ConfAIde and PrivacyLens evaluate whether an agent shares information appropriately, the latter over realistic agent actions [36], [38], and PrivaCI-Bench ties the evaluation to legal compliance [73]. For web agents, AgentDAM measures privacy leakage end to end under a data-minimization principle and ST-WebAgentBench scores agents against explicit policies [112], [113].¹

6.3 The Missing Benchmark↩︎

Table 6 identifies the gap. Each benchmark exercises one surface, and only a few evaluate a multi-step execution against an explicit policy. None evaluates a data agent end to end across the surfaces it actually spans, a database, a retrieval corpus, tools, memory, and inter-agent channels, within a single workflow governed by a stated privacy policy. As a result, a system can pass the text-to-SQL benchmark and the injection benchmark and the memory benchmark while still leaking through a path that crosses surfaces, for example a value read from the database, written to memory, and later forwarded to a peer agent. The compositional and cross-surface risks of Section 4, which are the ones most specific to data agents, are precisely the ones current benchmarks do not measure. The closest attempts increase realism without addressing the gap: a live evaluation turns static contextual-integrity benchmarks into dynamic agent interactions and pairs them with a mitigation [115], and a simulation framework searches for attacks and defenses through repeated privacy-critical agent exchanges [116]. Both raise the realism of the interaction, but each stays within an assistant’s tool actions and does not drive the agent across a database, a retrieval corpus, memory, and inter-agent channels under one stated policy. Addressing the gap requires a benchmark whose tasks drive the agent across surfaces under a policy, and whose scoring inspects every channel rather than only the output. We treat building it as an open problem in the next section.

7 Application Domains↩︎

The risks and controls surveyed above are not uniform across deployments. Each application domain stresses a different part of the mapping of data surfaces to risks, and the domain determines which controls matter most. We survey five domains where data agents already operate on sensitive data, summarized along the axes that distinguish them in Table 7. Figure 4 walks through one deployment as a worked example.

Enterprise analytics. The most mature data agents answer business questions over warehouses and lakehouses, translating a request into queries, joining across tables, and summarizing the result [13], [14]. The data is governed by existing access policies the agent must respect, so the dominant risks are leakage through generated queries, over-return of rows beyond a user’s entitlement, and inference across joined results, countered by query-level access control, information-flow control over results, and differential privacy on aggregates [23], [77], [79]. This domain presents the most direct conflict between agent autonomy and enterprise governance, because the agent constructs queries no human reviewed against the access policy.

Personal assistants. A personal assistant reads a user’s mail, files, calendar, and history to act on their behalf, and connects to external services through tools. The data is intimate and the agent is highly permissioned, so the dominant risks are injection-driven exfiltration from untrusted content and inappropriate sharing that violates contextual norms [17], [38], countered by contextual privacy reasoning, data minimization that releases only what a task needs [37], and runtime checks on the actions the assistant takes. Cross-session memory makes this domain particularly exposed to persistent leakage, since the assistant accumulates a detailed model of one person over time.

Scientific and institutional data agents. Agents that perform data science and analysis over institutional data [15], [48] operate on records that are sensitive and often regulated, such as research cohorts and administrative data. They load files directly and compute over them in code, so sensitive values pass through intermediate variables and generated code as well as the final report. The dominant risks are intermediate-step leakage and reconstruction from outputs that look aggregate, countered by formal privacy on released statistics and confidential execution when the compute is not trusted [53], [100].

Smart-space and sensor-data agents. Agents embedded in smart buildings, campuses, and homes act on continuous sensor streams, occupancy, location, and environmental data, to deliver safety, energy, and comfort services. The data is collected about many subjects who did not individually consent to each use, and sensitive facts such as a person’s location are inferred from streams that look innocuous in isolation, so the dominant risks are inference from aggregated streams and consent violations across a shared infrastructure, a setting whose privacy norms have been mapped through contextual integrity and whose users report weak consent [118], [119]. Recent work envisions privacy-preserving smart spaces that mediate this sharing through policy at the infrastructure level [117]. The controls that fit build on purpose-based access control and contextual-integrity reasoning [51], [61]: least-privilege release that withholds data until a service needs it, alongside two not-yet-realized controls, an ingestion-time mapping of a subject’s high-level preference onto the sensor streams that reveal it and a negotiation step that recovers utility within policy bounds. We are not aware of a deployed agent that realizes the preference-to-stream mapping or negotiation, and we return to both as open problems in Section 8.

Data-governance and compliance assistants. A growing class of agents helps organizations govern their own data: discovering and classifying sensitive fields, checking processing against regulation, and answering compliance questions [80], [109]. Here privacy is the task rather than a constraint on it, and the agent’s own access to the data it audits becomes the risk, countered by strong auditing and provenance so its actions are themselves accountable and purpose-bound access so it reads only what a compliance task requires.

Table 7 makes the cross-domain pattern visible: the hardest control gap differs by domain, composition for smart-space and multi-agent settings, over-return for enterprise, and inappropriate sharing for assistants, and the least-mature controls cluster on the domains that handle the most continuous or cross-session data. Autonomy level further constrains the choice: a non-autonomous query interface exposes one query a person could inspect, while a high-autonomy agent constructs and composes queries no one reviews, which shifts the burden from output filtering onto the access-control and information-flow mechanisms that act where the queries are built.

8 Open Problems and Research Directions↩︎

The surveyed work leaves the central problem of privacy in data agents unsolved: there is no way to state a privacy policy over a whole agentic workflow and enforce it end to end with a guarantee. This section sets out the open problems, grouped by the part of the problem they address, and the directions the surveyed work suggests for each.

Representing and placing policy. Two basic questions have no settled answer. The first is how to represent a privacy policy for a multi-step workflow rather than for a single query or release. A policy must speak about purpose, recipient, and consent across a sequence of actions over heterogeneous surfaces, and the regulatory notions of purpose limitation and minimization must be expressible in it [50], [51]. The second is the unit of enforcement: a policy can be checked at the query, the tool call, the memory write, the action, or the whole execution path, and these choices give different guarantees and different costs. Path-level enforcement is a promising direction [91], but which unit is right for which guarantee is open.

Composing guarantees. The single most important technical gap is composition. A data agent chains a retrieval, a query, an aggregation, a memory write, and a message to a peer, and each step may have its own protection, yet there is no framework that composes those protections into an end-to-end guarantee. Differential privacy composes cleanly for sequences of releases [53], and recent work bounds leakage along a pipeline of agents [72], but composing a differential-privacy budget with an access-control decision with a retrieval protection, across the different steps of one execution, is not solved. Until it is, a deployment can only reason about its steps in isolation, the condition that the compositional risks of Section 4 exploit.

Preserving utility. Every control costs utility, and a data agent that protects too aggressively ceases to be useful. The open problem is to minimize data exposure while preserving the utility a task requires, which calls for privacy-aware planning that scopes queries and joins to what the task needs, and for treating the trade-off as a quantity negotiated per task rather than a fixed budget. The utility-first view from differentially private data exploration, in which a utility requirement is fixed and privacy is spent only as needed, is a natural starting point for agents that must meet an analytical target [57], [58], [60].

Detecting violations in intermediate channels. Because a substantial share of leakage occurs on channels the user never sees [16], [17], a deployment needs monitors that inspect the queries, the tool arguments, the memory writes, and the inter-agent messages, not only the output. Building such monitors, and deciding what a violation looks like on each channel, is largely unaddressed.

Reasoning about context. Many violations are about appropriateness rather than secrecy, and getting them right requires the agent to reason about purpose, consent, and social norms. Contextual integrity provides the framing and early benchmarks provide measurement [38], [73], but reliable norm-aware behavior, and a faithful application of the theory to agents [107], remain open.

Forgetting and revocation. Persistent memory creates an obligation that data systems have long recognized and agents have not: the right to have data removed. An agent must support forgetting a fact, revoking access already granted, and rolling memory back to a prior state, across caches and derived artifacts as well as the primary store. Machine unlearning gives techniques for removing data from a model [120]–[122], recent work proposes governing an agent’s evolving memory through temporal decay and consistency checks [123], and the right-to-be-forgotten literature frames the obligation [124], but applying any of these to a running agent’s memory and indexes is open.

Enforcement outside the model. A recurring lesson of the surveyed defenses is that enforcement which depends on correct model behavior is fragile, and that deterministic infrastructure around the model is more trustworthy [33]–[35]. How far governance can be moved off the model and onto such infrastructure, without losing the flexibility that makes agents useful, is an open design question.

Governing third-party capabilities. Agents increasingly extend themselves with capabilities packaged by third parties as plugins, custom GPTs, Model Context Protocol servers, and skills, each granting data access the user did not review [66]. Vetting these components, scoping the data a skill may reach, and monitoring what it does with that data is an open supply-chain problem for agent privacy, and one the data surfaces and controls of this survey apply to directly.

Evaluation and evidence. Two evaluation gaps remain. The field needs the end-to-end, cross-surface, policy-governed benchmark identified in Section 6, built on realistic enterprise and personal workflows. And it needs to define the evidence an agent should produce to demonstrate that it complied with a policy, since a deployment in a regulated setting must show compliance, not merely assert it. Provenance and auditing supply the underlying mechanisms [52], [109], but what constitutes sufficient proof of compliance for an agentic workflow is undefined.

9 Conclusion↩︎

In this paper, we surveyed the privacy of LLM agents from a data-centric view. As data agents query databases, search private corpora, call tools, carry memory, and act, sensitive data can leak at any step of the execution, not only in the final answer, so privacy must be governed over the whole execution rather than at the output alone.

We organized the literature around the data an agent handles. We taxonomized the data surfaces an agent touches (Section 3), the privacy risks each surface creates as disclosure outcomes and attack vectors (Section 4), and the governance mechanisms that address them (Section 5), and we connected the three with cross-tables. We then surveyed the benchmarks that measure these risks (Section 6), the domains where data agents are deployed (Section 7), and the open problems (Section 8).

Two conclusions follow from the cross-tables. First, compositional leakage and inference across a session are the least-covered risks (COMP and IA in Table 5), and information-flow control is the only surveyed mechanism that addresses both, so it is the most useful single control to adopt. Second, every benchmark we surveyed exercises a single surface, so the field most needs an end-to-end benchmark that drives an agent across database, retrieval, tool, memory, and inter-agent surfaces under one stated policy. More broadly, a data agent takes on the obligations of a data system without yet having its governance, and supplying that governance over a dynamic, multi-step, partly autonomous execution is the central open problem this survey frames.

References↩︎