Abstract↩︎

Background: Federated learning (FL) enables multi-institutional model training on clinical text without sharing raw data; however, gradient inversion methods can reconstruct sensitive information from shared model updates. The extent of such privacy leakage in FL applied to radiology reports, and the role of tokenizer design, remains unclear.

Objective: To quantify gradient-based reconstruction of radiology report text in an FL setting and to compare privacy risk across three transformer tokenization strategies in a controlled, tokenizer-aware evaluation.

Methods: Six FL clients trained a GPT-2–style transformer (sequence length 32) on two public radiology corpora comprising 368,751 diagnostic reports, 98,206 discharge summaries, and 1,500 MIMIC-CXR free-text reports. Models were trained using three tokenizers (GPT-2, RadBERT, LLaMA-2) with batch sizes of 64, 128, and 256. An active malicious-server threat model was assumed (the server modifies the shared model architecture before distribution), and analytic gradient inversion was applied to recover text. Reconstruction fidelity was measured over five runs using exact sentence accuracy, S-BLEU, G-BLEU, and ROUGE-L.

Results: Exact sentence reconstruction ranged from 31% to 44% across tokenizers (30.6–43.5% across the 27 tokenizer \(\times\) dataset \(\times\) batch-size cells). At batch size 64 on the Discharge dataset, accuracy was 42.1% (GPT-2), 42.3% (RadBERT), and 39.4% (LLaMA-2), decreasing to 37.3%, 37.2%, and 34.3% at batch size 256. S-BLEU scores declined with increasing batch size (e.g., GPT-2: 0.44\(\rightarrow\)​0.33; RadBERT: 0.48\(\rightarrow\)​0.35; LLaMA-2: 0.39\(\rightarrow\)​0.30). RadBERT yielded higher reconstruction fidelity and greater recovery of clinical terms, but no tokenizer prevented leakage.

Conclusions: Substantial portions of radiology report text can be reconstructed from FL gradients even with larger batch sizes and domain-specific tokenizers. Tokenizer design influences leakage severity and should be incorporated into privacy evaluations for clinical language models. Integrating safeguards such as secure aggregation and differential privacy is likely necessary to meet HIPAA and GDPR requirements when deploying FL for radiology NLP.

Trial Registration: Not applicable.

Keywords: Federated Learning; Radiology; Privacy; Gradient Inversion; Large Language Models; Data Security; Transformer Models; Patient Confidentiality; HIPAA; GDPR

Introduction↩︎

Radiology reports serve as an indispensable tool in medical diagnostics, providing critical insights [1] that complement imaging data. These textual data contain rich clinical information, including patient history, diagnostic impressions, and recommended follow-ups, often bridging gaps left by imaging alone. In recent years, advancements in artificial intelligence (AI), particularly transformer-based large language models (LLMs) [2], have revolutionized natural language processing, making it possible to analyze unstructured textual data at scale. LLMs such as the Generative Pretrained Transformer (GPT) family [3] are now being explored for their potential in radiology to assist in generating summaries, extracting relevant clinical insights, and identifying patterns across large datasets [4–6]. A recent systematic review of LLM evaluations in clinical medicine highlights the rapid growth of these models and underscores the need for robust evaluation frameworks to ensure their safety, reliability, and ethical alignment in healthcare applications [7].

Radiology large language models can be collaboratively developed across multiple institutions, leveraging diverse clinical reports to enhance model robustness and improve generalizability across unseen healthcare settings. Nevertheless, healthcare data, such as radiological data sharing, faces significant challenges due to stringent data protection laws like the Health Insurance Portability and Accountability Act (HIPAA) [8] in the U.S. and the General Data Protection Regulation (GDPR) [9] in the EU, which mandate strict safeguards for the privacy of sensitive patient data. Federated Learning (FL) [10], a decentralized machine learning approach where models are trained across multiple institutions without exchanging raw data, has emerged as a promising solution to address these challenges by ensuring that sensitive data remains within each participating institution while still enabling collaborative model development. Multi-Centric FL clinical collaborations [11–14] are transforming medical research by enabling privacy-preserving data sharing, fostering innovation, and addressing critical concerns around data ownership and security.

Nevertheless, FL’s decentralized nature introduces inherent vulnerabilities that may compromise patient confidentiality [15,16]. While they eliminate direct data sharing, FL systems rely on the exchange of model parameters between institutions and a central server. An active malicious server — one that not only observes gradients but also modifies the shared model architecture before distribution — can exploit these parameters to reconstruct sensitive data, exposing significant vulnerabilities. Existing research has shown that gradient inversion attacks [17–19] can reconstruct private text from models trained on generic language datasets, but such risks remain largely unexplored in the domain of radiology [20].

Recent advances in natural language processing have demonstrated that leveraging domain-specific tokenizers [17] and training LLM on domain-adapted corpora significantly enhances performance in specialized fields such as medicine and genomics. For instance, studies like [21] introduced PubMedBERT, showing that pre-training on biomedical literature yields superior results on downstream clinical tasks compared to generalist models. Similarly, prior studies such as [22] demonstrate that fine-tuning LLMs like BioBERT and MedAlpaca with domain-specific data and tailored tokenization strategies substantially enhances performance in medical tasks. These adaptations enable the models to capture nuanced, domain-relevant semantics that general-purpose LLMs typically overlook.

To specifically investigate the role of tokenization in privacy leakage within federated learning, we isolate tokenizer design as the primary variable while keeping the underlying model architecture constant. While most prior work compares entire language models [7], we argue that vocabulary segmentation alone can influence the susceptibility of models to gradient inversion attacks. Tokenizers trained on domain-specific corpora (e.g., RadBERT) are more likely to encode medical terms as single tokens, increasing their semantic coherence and making them easier to reconstruct. In contrast, general-purpose tokenizers tend to fragment clinical phrases into multiple subwords, which may reduce both interpretability and recoverability. This design choice not only impacts downstream clinical utility [21] but also alters the re-identifiability of sensitive entities during model inversion. By decoupling the tokenizer from the model architecture, our approach provides a targeted evaluation of how vocabulary structure mediates the trade-off between semantic fidelity and patient privacy.

In our study, RadBERT [23], a radiology-specific transformer model, demonstrated consistently higher reconstruction fidelity under attack compared to general-purpose models like GPT-2 [24] and LLaMA-2 [25]. While this highlights the advantages of domain-adapted pretraining for capturing clinical semantics, it also reveals a concerning trade-off: specialized models may be more susceptible to privacy leakage in FL settings. This observation aligns with growing evidence that tokenizer and model design choices can significantly influence both performance and risk in domain-specific applications. Building on this insight, we investigate the extent to which radiology report data can be reconstructed from transformer-based models trained in a simulated FL environment. By orchestrating targeted attacks in a multi-client setup using publicly available radiological datasets, we expose critical vulnerabilities in current FL frameworks. Our study deliberately focuses on the attack surface and does not evaluate defense mechanisms or privacy-preserving strategies. Rather, it aims to characterize worst-case risks and motivate future work toward robust, domain-aware privacy protections for safe deployment of clinical foundation models.

Scope and Contributions. This work establishes a foundational analysis of the privacy vulnerabilities inherent in applying transformer-based federated learning (FL) to radiology reports. We focus on characterizing the attack surface by quantifying the reconstruction risk under a worst-case gradient-inversion scenario. To this end, we provide a systematic comparison of information leakage across major tokenizers, namely, GPT-2, RadBERT, and LLaMA-2. Our central contribution is the discovery that domain-specific tokenizers like RadBERT, while improving clinical performance, can paradoxically heighten the risk of leaking sensitive patient data. These findings provide a necessary benchmark and guidepost for developing effective privacy-preserving defenses in medical FL systems. Clinical utility is not directly evaluated in the present study and is inferred from prior work establishing improved downstream performance with domain-specific clinical tokenizers.

Methods↩︎

Ethical Considerations↩︎

This study is a secondary computational analysis of two publicly available, pre-de-identified clinical text corpora: the Dischargesum dataset [28] and the MIMIC-CXR free-text radiology reports accessed via PhysioNet [29] under the standard PhysioNet credentialed-access Data Use Agreement. No new human-subjects data were collected as part of this work. No identifiable individuals appear in any figure, table, or supplementary material. No participants were recruited and no compensation was provided. Per institutional policy at the German Cancer Research Center (DKFZ) and consistent with PhysioNet’s terms of use, secondary computational analysis of these de-identified public corpora qualifies for ethics-board exemption; the original IRB approvals covering the primary collection of MIMIC-CXR (Beth Israel Deaconess Medical Center) and Dischargesum extend to secondary research use without additional consent. All experiments were performed on local DKFZ compute infrastructure and no patient data were transmitted to external services.

Federated Learning and Transformer Models↩︎

Federated Learning (FL) [10] is a decentralized machine learning paradigm that allows multiple institutions to collaboratively train a common model without exchanging raw data. Each client trains locally and sends only gradient updates to a central server for aggregation. In our implementation, we simulated this multi-institutional setup using a GPT-2–style transformer (instantiated as a compact 3-layer encoder with token-embedding dimension \(m = 96\), 8 attention heads, feed-forward hidden size 1536, and approximately 13.4 M trainable parameters, following the Field Guide to Federated Learning baseline architecture [44]) combined with three different tokenizers:

• GPT-2 tokenizer — 50,257 tokens

• StanfordAIMI RadBERT tokenizer — 30,522 tokens

• LLaMA-2-7B tokenizer — 32,000 tokens

The overall attack pathway is illustrated in Figure 1.

All tokenizers operated with a fixed sequence length of 32 tokens, so any observed privacy differences are attributable to vocabulary segmentation rather than to input shape or representational scale. Holding the foundation-model architecture fixed at this small, well-characterized baseline allows the tokenizer to serve as the sole independent variable across our GPT-2, RadBERT, and LLaMA-2 comparisons. The 32-token sequence length follows the Field Guide to Federated Learning baseline configuration [44] and is consistent with prior gradient-inversion experiments on text models [41]; it provides a controlled, computationally tractable regime in which the imprint construction’s bin disentanglement is well-characterized. To emulate real multi-institutional training, we randomly partitioned two radiology text datasets across six cloud-based clients: the public Dischargesum corpus (98,206 discharge summaries; 368,751 diagnostic reports) and a locally curated sample of 1500 free-text reports from MIMIC-CXR. Training was repeated with batch sizes of 64, 128, and 256 sequences to measure how aggregation granularity affects privacy leakage in shared gradients. Batch sizes refer throughout to 32-token sequences, not whole reports: each report is tokenized and split into multiple non-overlapping 32-token windows; a single discharge summary or radiology report typically contributes between 5 and 40 sequences depending on its length, so the per-client local datasets contain \(\ge\) 5,000 sequences and comfortably exceed the largest batch size of 256. Gradient-inversion attacks of the type considered here operate on the gradient produced by a single client during a single training round, which constitutes the standard threat-surface unit in the gradient-leakage literature [42,41]; privacy leakage was therefore quantified per (client, round) rather than over the aggregated federation-level update.

Threat Model↩︎

We characterize the adversary in this study as an active malicious server, following the worst-case threat-model conventions used in prior FL gradient-inversion analyses [43,46]. Formally, the adversary controls the central FL server and possesses the following capabilities. (C1) Architecture modification (active, white-box on the server side): the server may insert auxiliary parameters into the shared model before the first communication round, provided the resulting model still trains end-to-end. We exercise this capability by inserting a single analytic imprint module (Section Background & Preliminaries) immediately before the positional embedding. (C2) Per-round per-client gradient observation: the server observes the full set of weight and bias gradients \(\nabla_\theta \mathcal{L}\) returned by each client at each round. We do not assume access to intermediate activations, hidden states, or any non-gradient signal — i.e., the protocol is standard FedAvg/FedSGD, not split learning. (C3) Knowledge of the public model and tokenizer: the server knows the model architecture (since it distributed it) and the tokenizer’s embedding table (since it is part of the shared model). The adversary does not observe raw client data, client-side optimization state, or any cryptographic key material. We assume no defenses are enabled at any layer of the FL stack: no differential privacy, no secure aggregation, no client-side architecture verification, no gradient anomaly detection. This intentionally maximal threat model isolates the upper-bound leakage attributable to gradient inversion alone, which is the quantity our experiments seek to characterize.

Detectability and stealthiness. The architecture modification in (C1) is in principle detectable by any client that compares the received model’s parameter signature to a published reference. In current production FL frameworks, no such signature verification is mandatory, and small architectural modifications could in principle be disguised by representing the imprint module as a benign feature-extractor or input-normalization layer. However, the imprint module itself adds approximately \((1000 \times 96) + (96 \times 1000) + 1000 \approx 193\) K parameters — roughly 1.4 % of the 13.4 M-parameter base model — and would therefore be detectable by any routine client-side parameter-size audit or cryptographic attestation of the published model signature. We therefore treat detectability as a deployment-policy mitigation, not a fundamental defense, and we recommend in Practical and Regulatory Implications that mandatory client-side architecture attestation be considered as part of clinical FL audit protocols.

Scope of these findings. The reported reconstruction rates correspond to a worst-case adversary: a malicious central server that (i) owns and modifies the shared model architecture before distributing it to clients, (ii) observes per-round per-client gradients on weight and bias parameters, and (iii) operates in a setting with no privacy defenses enabled — no differential privacy, no secure aggregation, no gradient clipping, and no client-side model integrity verification. Under typical real-world cross-silo FL deployments, where institutional clients verify the received model architecture, secure aggregation conceals individual client updates, and lightweight differential-privacy noise is applied, these reconstruction rates would be expected to fall substantially. We therefore interpret our numbers as an upper bound on residual leakage when defenses are absent, not as expected leakage in defended deployments. This framing is consistent with the worst-case methodology of prior gradient-inversion studies [42,41].

Background & Preliminaries↩︎

Let the token-embedding dimension be \(m\) and the probe output dimension be \(k\). For a single token position, the hidden vector \(x \in \mathbb{R}^m\) is produced by the embedding table before positional addition and attention mixing. We insert a small linear probe (an “imprint” module in the sense of [42]) on top of this hidden state:

\[y = W_2 \cdot \mathrm{ReLU}(W_0 x + b_0),\]

with \(W_0 \in \mathbb{R}^{K \times m}\), \(b_0 \in \mathbb{R}^K\), and \(W_2 \in \mathbb{R}^{m \times K}\), where \(K = k\) denotes the number of probe bins. The probe is appended once at initialization and trained jointly with the rest of the model. As in standard cross-silo federated learning (FedAvg; [10]), each client transmits only the parameter gradient \(\nabla_\theta \mathcal{L}\) of its local loss back to the server; no intermediate activations or hidden states are exchanged. This rules out split-learning style leakage and isolates the threat surface to the gradient update itself.

We initialize \(b_0\) to the cumulative quantiles of a Laplace prior over the expected pre-activation distribution and \(W_0\) to a random Gaussian matrix; \(W_2\) is initialized so that \(y \approx x\) in expectation, leaving downstream training behaviour essentially unchanged. Under this construction the bins partition the input space into cumulative half-spaces of geometrically decreasing measure, so that for sufficiently large \(K\) at most one sample in a mini-batch activates each bin after consecutive-bin differencing. In general, batch aggregation destroys per-sample information: the transmitted gradient \(\nabla_\theta \mathcal{L}\) averages over all \(B\) samples in the batch and is not invertible without optimization [42]. The cumulative-bin imprint construction circumvents this barrier by analytic design rather than by optimization — the contribution of each individual token to \(\nabla W_0\) and \(\nabla b_0\) is structurally separated by bin index \(i\), so the server can isolate per-sample gradients even though it only ever observes the batch sum.

From these two captured gradients, the input hidden vector \(x\) can be recovered in closed form without optimization:

\[\hat{x}_j = \frac{\nabla W_{0,i}}{\nabla b_{0,i}},\]

where \(i\) is the bin uniquely activated by token \(j\) after the cumulative-bin differencing step \(\nabla W_{0,i} \leftarrow \nabla W_{0,i} - \nabla W_{0,i-1}\) (and analogously for \(\nabla b_{0,i}\)). We apply this per token and per sample in a batch; sentence reconstructions are formed by decoding each \(\hat{x}_j\) to the nearest token in the active tokenizer’s embedding table \(E \in \mathbb{R}^{V \times m}\) under cosine similarity, where \(V\) is the tokenizer vocabulary size. We set \(K = 1000\) bins and \(m = 96\), with \(\hat{x}_j\) L2-normalized before decoding; when \(\|\nabla b_{0,i}\|_2\) is below a numerical threshold \(\varepsilon\) we add a ridge term \(\varepsilon\) to the denominator for numerical stability.

Ties or near-ties (cosine margin \(< \delta\)) among candidate tokens are broken by re-scoring with the model’s language-model-head logits in a server-side forward pass conditioned on the previously-decoded prefix of the same sample — feasible because the malicious server holds the full shared model and can run forward passes on partial reconstructions. Token \(j\) is therefore resolved in left-to-right context rather than in isolation; the very first position falls back to the unconditional logit prior.

Attack Implementation↩︎

We implement the gradient-inversion attack as a deterministic, server-side procedure operating on per-round gradients. The analytic probe and closed-form inversion are defined in Section Background & Preliminaries; below we describe the practical instrumentation, data capture, and decoding steps used in our experiments.

Step 1 — Instrumentation. The server augments the shared transformer with a lightweight linear probe inserted immediately after the token embedding lookup and before positional additions and attention mixing (see Background & Preliminaries). This placement exposes token-level representations prior to any token mixing, ensuring reconstructed vectors reflect tokenizer segmentation rather than later architectural effects.

Step 2 — Gradient capture. During each client training round, the server records the probe gradients returned in the aggregation step — specifically, the per-round weight gradient \(\nabla_{W_0}\mathcal{L} \in \mathbb{R}^{K \times m}\) and bias gradient \(\nabla_{b_0}\mathcal{L} \in \mathbb{R}^K\) of the imprint module’s first linear layer (notation as in Background & Preliminaries). The “upstream signal” \(\partial\mathcal{L}/\partial y\) referenced in prior gradient-inversion literature is not transmitted by clients; it is reconstructed server-side from the captured weight and bias gradients under the cumulative-bin construction. These per-round gradients are stored for subsequent inversion and auditing; we retain round identifiers and batch metadata to link reconstructions to training conditions (dataset, tokenizer, batch size).

Step 3 — Closed-form inversion and stabilization. For each captured token gradient in a round, we compute the closed-form embedding estimate \(\hat{x}\) following the expression in Background & Preliminaries. After inversion, each \(\hat{x}\) is L2-normalized to match embedding norm scaling; when \(\|g\|_2\) is near zero we add a small ridge term \(\varepsilon\) to the denominator for numerical stability:

\[\hat{x} = \frac{\left(\dfrac{\partial\mathcal{L}}{\partial W}\right)^{\!\top} g}{g^{\top} g + \varepsilon}, \qquad \varepsilon = 10^{-8}.\]

Step 4 — Token decoding. Each recovered vector \(\hat{x}\) is decoded by nearest-neighbour search over the active tokenizer’s embedding table using cosine similarity (top-1). When multiple vocabulary entries are within a small cosine-similarity margin \(\delta\) of the maximum, the server breaks the tie by re-scoring candidates using the model’s language-model-head logit at that position, computed in a server-side forward pass conditioned on the previously-decoded prefix. This is feasible because the malicious server holds the full shared model (it distributed the model in the first place) and can execute forward passes on partial reconstructions; the very first position falls back to the unconditional logit prior. Decoded tokens are concatenated in positional order to yield sentence-level reconstructions.

Clarification on positional order recovery. Positional order is determined by an optimal-assignment matching between recovered breached embeddings and the 32 known positional embeddings of the sequence, implemented as scipy.optimize.linear_sum_assignment (the Hungarian algorithm) in the breaching library, following Fowl et al. [42]. The language-model-head logit re-scoring described above operates within each assigned position to resolve vocabulary ambiguity; it does not itself determine position. The phrase “immediately before the positional embedding” refers to topological placement within the encoder stack (before the attention and token-mixing layers); the recovered breached embeddings nonetheless carry the positional signature used by the matching stage above.

Step 5 — Design for fair tokenizer comparison. To isolate tokenization effects from architecture or training hyperparameters we hold the model architecture, embedding dimension \(m\), probe output dimension \(k\), and probe placement constant across experiments. The same inversion and decoding hyperparameters (ridge \(\varepsilon\), L2 normalization, nearest-neighbour metric) were applied to GPT-2, RadBERT, and LLaMA-2 tokenizer evaluations without per-tokenizer tuning. Because decoding uses each tokenizer’s own embedding table, reconstruction fidelity reflects vocabulary segmentation and tokenizer-induced exposure (single token entities vs fragmented subwords).

Implementation notes and novelty. All inversion and decoding steps were implemented in Python (NumPy/PyTorch). Nearest-neighbor lookup used a brute-force cosine search for reproducibility; for large vocabularies, the same pipeline is compatible with approximate nearest-neighbor libraries (FAISS). Per-round artifacts (timestamps, batch composition) were retained to permit round-level analyses and the violin plots reported in Figure 3 and Multimedia Appendices 3–5. The analytic probe itself follows prior work [26]; our novelty lies in (i) its embedding-stack placement to isolate tokenizer-specific effects, (ii) a tokenizer-controlled, architecture-fixed evaluation revealing a privacy–utility trade-off, and (iii) clinical entity-level leakage quantification using MedGemma NER [27], which distinguishes boilerplate text from clinically re-identifiable content.

Clarification on the cumulative-bin regime. The \(K = 1000\) bins are allocated globally over the batch rather than per token position, so the expected per-bin occupancy \(B/K\) is computed over the full pool of \(B \times T\) tokens (\(T = 32\) here). The strict single-occupancy regime (\(B < K\)) is exact at our smallest batch (\(B = 64\)); at the largest configuration (\(B = 256\) sequences = 8192 individual tokens) some bins are multiply occupied. The analytic recovery rule \(\hat{x}_j = \nabla W_i / \nabla b_i\) is exact only in the single-occupancy regime; under multi-occupancy it yields a per-bin weighted average of the active samples’ embeddings, producing a blurred but non-vacuous reconstruction. Positional disambiguation in this regime is performed by the optimal-assignment matching stage described in the Attack Implementation (Step 4) operating on the recovered continuous embeddings. The reconstruction-accuracy degradation we report across batch sizes \(64 \rightarrow 128 \rightarrow 256\) is consistent with this graceful degradation rather than a strict pigeonhole failure.

Dataset and Experimental Setup↩︎

We evaluated privacy leakage using three radiology text corpora. The Dischargesum [28] dataset contained 98,206 discharge summaries and 368,751 diagnostic reports, which were evenly distributed across six simulated institutional clients. To complement these structured reports, we also included a subset of 1,500 free-text MIMIC-CXR [29] radiology reports, similarly partitioned across the six clients.

MIMIC-CXR sample. From the full MIMIC-CXR free-text corpus (\(\approx\) 227,000 reports), we sampled 1,500 reports uniformly at random without replacement, conditional on each report being non-empty after standard whitespace stripping. No filtering was applied by clinical content, report length, institutional source, or temporal range. The sample identifier list is included in our reproducibility package and available from the corresponding author on reasonable request, subject to the PhysioNet Data Use Agreement.

Each client trained a GPT-2–style transformer locally and transmitted only gradient updates to the central server for aggregation, emulating a standard cross-institutional FL workflow. Reports were uniformly sampled at random across the six clients, producing an approximately IID partition for the purposes of this study. Experiments systematically varied the local batch size (64, 128, and 256 sequences) to evaluate its effect on gradient-level privacy leakage. Every configuration was repeated across five independent runs to account for stochastic training variability and to ensure reproducibility; 95% confidence intervals across runs are reported in Multimedia Appendix 6; exact paired t-tests on a verification re-run are reported in Multimedia Appendix 8. Training details and reproducibility configuration are reported in Multimedia Appendix 7. At each measurement point, the malicious server captures the gradient transmitted by one client in a given round and applies the analytic reconstruction described in Background & Preliminaries to that single update, in line with the standard single-step formulation adopted in the gradient-inversion literature [42]. Each batch-size configuration was evaluated under this per-client, per-round setting across the five independent runs.

Evaluation Metrics↩︎

We assessed reconstruction fidelity using four complementary metrics chosen to capture both exact and partial text recovery, following established practice in prior gradient-inversion and text reconstruction studies [13–16, 23]. These metrics together quantify the extent to which patient-identifiable information can be re-created from shared model updates.

• Exact Sentence Accuracy. The percentage of reconstructed sentences that match the ground truth exactly, reflecting the proportion of complete exposures of patient text. A reconstructed sentence is counted as an exact match if, after the following normalization steps, every token at every position equals the corresponding token in the original sentence: (i) Unicode NFKC normalization; (ii) lower-casing; (iii) collapsing all runs of internal whitespace to a single space; (iv) preserving punctuation as part of the token sequence rather than stripping it. Sentences are extracted from the concatenated stream of decoded 32-token windows by segmenting at terminal punctuation; the metric is therefore evaluated on these reconstructed sentence units rather than directly on the raw 32-token chunks. We report a stricter case-sensitive variant on the same five-run experiment for completeness; the qualitative ordering of tokenizers is unchanged across both variants. Exact-sentence accuracy is reported as the conservative upper bound on full-sentence exposure: token-level partial recovery (a single recovered diagnostic noun phrase, for instance) can constitute clinically meaningful leakage even when the full sentence is not reconstructed exactly; this regime is captured by the \(n\)-gram and longest-common-subsequence metrics described below. The two metric families together bracket the true leakage rate.

• Sentence-Level BLEU (S-BLEU) [30–31]: Calculates \(n\)-gram precision, where an \(n\)-gram is a contiguous sequence of \(n\) words (e.g., unigrams, bigrams, trigrams, and four-grams). BLEU compares 1–4 word sequences in the reconstructed sentence against the reference, with a brevity penalty to discourage overly short outputs. Higher scores indicate stronger local overlap and preservation of clinical phrasing.

• ROUGE-L [33]: Measures recall based on the longest common subsequence (LCS) between reconstruction and reference, capturing the extent to which key clinical terms and their order are retained in longer spans.

• Global BLEU (G-BLEU) [30]: Aggregates all reconstructed and reference texts into a single corpus and computes BLEU precision across the entire set. BLEU precision measures how many \(n\)-gram sequences (1–4 words) in the reconstructed text are also present in the reference text, reflecting the accuracy of content reproduction. By evaluating over the full corpus rather than sentence-by-sentence, G-BLEU captures overall reconstruction consistency and helps identify systematic leakage patterns in model outputs.

By combining these four metrics, we quantify both the frequency of perfect reconstructions and the degree of partial text recovery, offering a comprehensive assessment of patient data exposure under our federated learning attack.

Choice of metrics. BLEU and ROUGE-L were originally proposed for translation and summarization quality, where their numerical values are bounded by human-level \(n\)-gram overlap. In the gradient-inversion context, we use them as information-recovery metrics rather than as quality metrics: a higher BLEU or ROUGE-L between a reconstructed sentence and the original indicates more shared \(n\)-gram content — i.e., a greater amount of original wording recovered. Exact-sentence accuracy provides the strict-recovery upper bound; S-BLEU and ROUGE-L provide the partial-recovery signal that captures cases where most of a sentence is recovered but a small number of tokens are mis-decoded. This complementary use of strict and partial metrics is consistent with prior gradient-inversion text studies [45,41]. We deliberately do not adopt semantic-similarity metrics such as BERTScore in the main results because they can credit paraphrastic matches that do not constitute literal information leakage, which is the privacy question of interest here.

Named Entity Recall Evaluation↩︎

We quantified recovery of clinically meaningful content using the Google MedGemma medical NER model [27]. MedGemma was run on a random sample of 100 original discharge summaries to build a reference vocabulary of 1,440 unique entity surface forms (diagnoses, procedures, anatomical structures, medications). The same model was applied to reconstructed summaries for each tokenizer configuration. The 100-report sample size was chosen on the basis of clinical-entity vocabulary saturation: in pilot internal analyses the cumulative count of unique MedGemma-extracted entities plateaus near 1,400 unique surface forms by sample = 70 reports and is approximately flat by sample = 100, indicating that this sample is sufficient to characterize the recoverable clinical-entity vocabulary at the corpus level. Larger samples (e.g., 500+ reports) would yield diminishing returns in unique-entity coverage at substantially higher computational cost.

Normalization and matching. Entity strings were lowercased and whitespace-trimmed; internal punctuation was preserved. An entity was counted as recovered if its surface form exactly matched any term in the 1,440-item reference (case-insensitive). No fuzzy matching, synonym expansion, or manual adjudication was used.

Metric. For each tokenizer, we report Recovered Terms (unique count) and Recall Rate (%): Recall Rate (%) = (Recovered Terms / 1,440) \(\times\) 100. We also report the union across tokenizers (“Total Unique Terms Recovered”) computed against the same 1,440-term reference.

Results↩︎

Quantitative Reconstruction Success↩︎

Exact sentence-level reconstruction accuracy on the Discharge dataset was:

• GPT-2 tokenizer: 42% / 40% / 37% at batch sizes 64 / 128 / 256

• RadBERT tokenizer: 42.3% / 40% / 37%

• LLaMA-2 tokenizer: 39.4% / 37% / 34%

On MIMIC-CXR, reconstruction ranged 33%–41% sentences (Figure 2), showing that neither tokenizer choice nor larger batch sizes fully mitigates data leakage. Detailed accuracy results are provided in Multimedia Appendix 1. All reported reconstruction metrics represent the mean of five independent runs; 95% confidence intervals across runs are reported in Multimedia Appendix 6; exact paired t-tests on a verification re-run are reported in Multimedia Appendix 8. Across all configurations, standard deviations were below 1 percentage point, confirming that the observed trends are statistically stable.

Impact of Batch Size on S-BLEU↩︎

Increasing the batch size led to a clear reduction in average reconstruction fidelity across all metrics (Table 2; Figure 3). For discharge reports, mean S-BLEU decreased from 0.44 at a batch size of 64 to 0.33 at 256; diagnosis reports fell from 0.43 to 0.32; and MIMIC-CXR from 0.45 to 0.32. Parallel trends were observed for ROUGE-L and G-BLEU, confirming that coarser gradient aggregation generally mitigates privacy leakage. Notably, across all datasets and batch sizes, the RadBERT tokenizer consistently achieved higher reconstruction scores than GPT-2 and LLaMA-2. This suggests that domain-specific tokenization may facilitate more accurate recovery of clinical language—enhancing semantic fidelity but potentially increasing privacy risk. Corresponding mean S-BLEU, G-BLEU, and ROUGE-L scores are summarized in Multimedia Appendix 2.

Variability in Gradient Inversion Severity Across Training Rounds↩︎

While average reconstruction fidelity declined with larger batch sizes across all metrics, individual training rounds revealed substantial fluctuations in leakage severity. We illustrate this variability using S-BLEU in Figure 3, as it provides a balanced measure of local \(n\)-gram precision and sentence-level structure, making it particularly sensitive to round-to-round differences. However, similar variability patterns were also observed for Exact Sentence Reconstruction Accuracy, ROUGE-L, and G-BLEU, which are summarized in Multimedia Appendices 3–5. Even at the highest batch size (256), certain rounds produced high-fidelity reconstructions comparable to those at batch size 64, underscoring a persistent privacy risk. Coarser gradient aggregation therefore reduces but does not eliminate patient information exposure in federated learning.

In real-world medical AI deployments, even a single outlier round with high leakage could compromise patient confidentiality. Therefore, it is essential to characterize not just average-case leakage but also variability across rounds, to ensure that system-level guarantees account for worst-case scenarios.

Clinical Term Recall in Reconstructed Text↩︎

As presented in Table 4, RadBERT reconstructions recovered the highest number of clinical terms (260, or 18.1%), followed by GPT-2 (180 terms, 12.5%) and LLaMA-2 (136 terms, 9.4%). In total, 380 unique terms were leaked across all models, corresponding to a cumulative recovery rate of 26.4%.

These findings demonstrate that gradient inversion attacks can reveal not only structural sentence fragments but also clinically meaningful entities, posing a tangible privacy risk. The elevated recall from RadBERT, despite its clinical utility, indicates that domain adaptation may heighten vulnerability in federated training settings.

Qualitative Reconstruction↩︎

Tables 1–3 showcase original and reconstructed outputs from the Dischargesum, Radiology, and MIMIC-CXR datasets, highlighting tokenizer-specific vulnerabilities. GPT-2 reconstructions retained general sentence structure but frequently dropped key clinical terms. RadBERT, by contrast, preserved more medically relevant entities such as “catheter” and “nodularity,” reflecting its domain-specific tokenization. LLaMA-2 exhibited the most fragmented outputs, with high token dropout and less consistent semantic coherence. These qualitative results reinforce the quantitative findings, demonstrating that even domain-adapted transformer models remain susceptible to gradient-based inversion attacks, recovering not only template phrases but also clinically meaningful patient information.

Discussion↩︎

Principal Results↩︎

This study demonstrates that even with domain-specific tokenization and larger batch sizes, transformer models in an FL setup leak substantial patient information. The active server reconstructed up to 42 % of radiology report sentences: GPT-2 yielded 42% / 40% / 37% accuracy at batch sizes 64 / 128 / 256; RadBERT, 42.3% / 40% / 37%; and LLaMA-2, 39.4% / 37% / 34%. Sentence-level S-BLEU scores similarly declined (e.g., GPT-2 from 0.44 \(\rightarrow\) 0.38 \(\rightarrow\) 0.33), but even at the largest batch sizes, nearly one-third of sentences were fully recovered. Across all batch sizes and datasets, RadBERT consistently achieved higher reconstruction scores than GPT-2 and LLaMA-2—for example, reaching an S-BLEU of 0.52—indicating improved fidelity in recovering sentence structure and phrasing. RadBERT preserved more clinical keywords (e.g., anatomical and diagnostic terms) than the generalist models, yet did not eliminate leakage. These findings confirm that neither increasing batch sizes nor specialized tokenizers alone can ensure patient privacy in FL-trained LLMs.

To further disambiguate templated language from truly clinical leakage, we applied MedGemma NER to 100 reconstructed summaries. Across all models, 380 clinical terms were recovered out of 1440 unique entities. These findings confirm that privacy risks extend beyond boilerplate phrasing to include clinical concepts relevant to diagnosis and care.

Our findings underscore that tokenizer design plays a critical role in privacy leakage under gradient inversion attacks. While RadBERT’s domain-specific tokenizer improved the retention of clinically meaningful terms, it also increased the recoverability of sensitive information, leaking 260 out of 1440 reference entities (18.1%) compared to 12.5% and 9.4% for GPT-2 and LLaMA-2, respectively. This highlights a fundamental trade-off: the same vocabulary segmentation that enhances clinical utility may inadvertently make protected health information (PHI) easier to reconstruct. In this light, tokenizer choice should be treated not merely as a performance optimization, but as a privacy-sensitive design decision. Future privacy audits and risk assessments of federated clinical models should explicitly account for tokenization strategies, especially as domain-specific LLMs become more prevalent in medical AI.

Mechanism — why domain-specific tokenization heightens reconstruction. The reconstruction step decodes each recovered token-embedding vector back to the nearest token in the active tokenizer’s embedding table. A clinical concept such as “pneumothorax” is encoded as a single token in RadBERT’s domain-specific vocabulary, so a successful nearest-neighbour decode yields the entire concept verbatim. The same concept is segmented into multiple subwords (e.g., “p / neu / moth / orax”) in GPT-2’s byte-pair encoding; recovering the concept therefore requires all of its subword embeddings to be reconstructed correctly and placed in the correct positional order. Each fragmented subword introduces an additional point of failure, and any one mis-decode yields a corrupted concept. Domain-specific tokenization thus reduces the attack-surface diameter per clinical entity — a property that improves training utility on clinical text but, by the same mechanism, raises gradient-inversion vulnerability.

Clinical concepts versus directly identifiable PHI. The MedGemma named-entity recognition model used in this study captures clinical concepts (diagnoses, procedures, anatomical structures, medications) rather than direct identifiers under the HIPAA Safe Harbor enumeration (names, MRNs, dates more granular than year, full-face photographic images, etc.). Direct identifiers are largely already redacted in the public Dischargesum and MIMIC-CXR corpora before release. The recovered terms therefore reflect re-identification potential through clinical context — for example, an unusually rare diagnosis, a unique procedure pattern, or a specific medication regimen that, in combination with auxiliary recovered context, could plausibly contribute to re-identifying a patient — rather than direct exposure of an enumerated identifier. We make this distinction explicit because the regulatory framing of “PHI” is broader than “direct identifier”; HIPAA’s expert-determination standard (45 CFR §164.514(b)(1)) treats clinical context recoverable from a record as part of identifiability risk, and the same logic is reflected in the GDPR’s notion of indirect identifiability via singling out.

Why batch size reduces leakage in this attack. The imprint module’s analytic recovery rule \(\hat{x}_j = \nabla W_{0,i} / \nabla b_{0,i}\) relies on the assumption that bin \(i\) is activated by at most one sample in the batch. With \(K = 1000\) bins and batch size \(B\), the expected number of samples per bin is \(B/K\), and the probability that any given bin is hit by \(\ge 2\) samples grows approximately quadratically in \(B\) (a birthday-paradox argument). When two or more samples share a bin, the recovered ratio \(\nabla W_{0,i} / \nabla b_{0,i}\) becomes a gradient-weighted average of distinct embeddings rather than a clean single-sample recovery, and the corresponding nearest-neighbour decode is corrupted. This explains why exact-sentence accuracy decays with batch size while never reaching zero: as long as some bins remain singly occupied, the corresponding samples are recovered cleanly. The same argument predicts that increasing \(K\) relative to \(B\) would partially compensate for batch growth — a direction we leave to future work.

Key Contribution. We provide the first systematic evaluation of how different transformer tokenizers impact gradient-inversion vulnerability on radiology text, showing that domain-specific tokenization (RadBERT) may heighten leakage risk even as it better preserves clinical terminology.

Comparison With Prior Work↩︎

Our results extend prior demonstrations of gradient-inversion attacks in generic natural language processing models [17,18] and imaging domains [19] to the structured and template-rich text of radiology reports. Earlier works such as DLG [47] and iDLG [26] reconstructed generic sentences or pixel-level images from shared gradients but did not examine how tokenizer design mediates privacy leakage. Subsequent privacy-preserving FL frameworks in medical imaging [17–19] primarily focused on architectural defenses rather than language-specific vulnerabilities. Unlike these approaches, our study isolates the tokenizer as a controllable privacy variable by holding the transformer architecture and training hyperparameters constant across GPT-2, RadBERT, and LLaMA-2. This tokenizer-controlled setup reveals that vocabulary segmentation alone can significantly influence gradient-based reconstruction fidelity. This comparison against prior gradient-inversion literature is summarized in Table 5. While Akinci et al [20] first discussed text leakage risks in clinical LLMs, no prior work systematically compared multiple tokenization schemes or evaluated reconstruction across both discharge and imaging-report corpora. Our findings therefore provide the first quantitative evidence that domain-adapted tokenizers—though beneficial for clinical utility—may paradoxically amplify privacy leakage in federated radiology LLMs.

Practical significance. Earlier closed-form gradient inversion attacks on text models report exact-sentence recovery rates of approximately 30–50 % at small batch sizes on generic English corpora [45,41]; our radiology-domain experiments fall within this range (31–44 %), confirming that the threat extends from generic to clinical NLP without amplification at the strict-recovery level. The practical significance is, however, qualitatively different in the clinical setting: radiology corpora encode clinical context (diagnoses, procedure types, anatomical findings) at higher density per token than newswire or web text, so equivalent reconstruction rates translate to higher clinical informativeness per recovered sentence. A single recovered sentence in a clinical context may contain a rare diagnosis or unusual procedure combination that contributes meaningfully to re-identification risk, whereas a single recovered sentence from generic web text typically carries no analogous re-identifying signal.

Scope of the tokenizer comparison. Because we hold the foundation-model architecture fixed at a small, well-characterized baseline, our findings reflect the effect of vocabulary segmentation on gradient-inversion leakage, not the privacy properties of the GPT-2, RadBERT, or LLaMA-2 foundation models themselves. Larger frontier-scale foundation models exhibit different gradient sparsity and noise dynamics that can change inversion vulnerability in either direction; we do not claim that our results extend to those regimes, and explicit characterization at frontier scale is an important target for future work.

Practical and Regulatory Implications↩︎

Focus on Attack Characterization. This study deliberately isolates the gradient-inversion attack vector and does not evaluate or compare privacy defenses. Instead, we quantify the worst-case leakage risk under an unconstrained adversary. Subsequent work should build on these findings to empirically evaluate and optimize defense mechanisms in realistic radiology FL pipelines. Our results demonstrate that generic federated learning defenses such as differential privacy [34], secure aggregation [35] and gradient-anomaly detection have yet to be empirically validated for the structured, high-risk text found in radiology reports. Injecting noise via differential privacy may reduce reconstruction fidelity but requires careful calibration to avoid degrading critical diagnostic language. Similarly, cryptographic secure aggregation can obscure individual updates but may introduce prohibitive latency in hospital networks. Real-time gradient-anomaly detection promises early warning of inversion attacks, yet its thresholds must be tuned to the unique update patterns of clinical text to prevent both false alarms and missed breaches. To ensure compliance with HIPAA and GDPR, healthcare organizations may benefit from running pilot evaluations of these defenses under realistic conditions and document performance trade-offs. Regulatory bodies could consider developing formal audit protocols for federated learning systems, encompassing threat modeling, reconstruction testing, and defense efficacy, before granting approval for clinical deployment.

Defenses and Practical Mitigations↩︎

While this study focused on characterizing gradient inversion risk, effective deployment of federated radiology models requires integrating strong privacy defenses [35]. Common mitigation strategies such as synthetic data generation and differential privacy (DP) alone remain insufficient for large language models [36–38]. Synthetic data can leak statistical artifacts and fail to preserve downstream clinical performance [36,37], whereas DP often requires large privacy budgets to maintain utility, limiting its protective value [39].

In practice, more robust protection arises from hybrid approaches that combine secure aggregation to mask individual updates with task-tuned DP noise, gradient clipping, and auditable privacy monitoring. Cryptographic protocols such as homomorphic encryption or secure multiparty computation offer mathematically proven safeguards but may introduce computational overhead in hospital networks [40]. Future implementations of federated clinical NLP should adopt layered defenses that balance efficiency, regulatory compliance, and measurable privacy guarantees [35,40].

Limitations↩︎

This study was designed as a worst-case evaluation, assuming a fully active malicious-server capable of modifying model architecture and exploiting gradient updates. While extreme, this assumption is appropriate for international or cross-consortium federated settings, where governance may be inconsistent and insider threats cannot be ruled out.

Several constraints limit the generalizability of our findings:

• Dataset Scope. We used publicly available radiology corpora (Dischargesum and MIMIC-CXR), which lack multimodal components (e.g., images), free-text notes, and operational metadata such as timestamps or identifiers. Future evaluations should test gradient inversion on richer, production-like datasets to better estimate clinical leakage risk.

• Within-batch sequence correlation. Reports were split into non-overlapping 32-token windows; multiple windows from the same report may co-occur within a single training batch. This violates the i.i.d.assumption implicit in our bin-collision analysis and may either inflate or deflate empirical reconstruction success relative to the theoretical per-token collision rate. A correlation-aware extension of the analytic inversion bound is left to future work.

• Clinical-entity recall scope. The MedGemma NER analysis was applied to a saturation-justified sample of 100 reconstructed discharge summaries (Dischargesum); extension of this analysis to the Diagnosis and MIMIC-CXR corpora — which differ in report length, vocabulary density, and clinical specialty — was beyond the compute budget of the present study and is left for future work. The clinical-entity leakage claims should therefore be interpreted as established for the discharge-summary regime and as a hypothesis to be tested in the other two report types.

• Re-execution for the paired t-test analysis. The paired t-test analysis reported in Multimedia Appendix 8 is computed from an independent re-execution of the experimental grid. The primary reported reconstruction values in Appendices 1 and 2 are those from the original five-run experiment, on which all substantive conclusions of the manuscript rely. The re-execution reproduces the largest-batch (\(B = 256\)) sBLEU values within 0.05 of the originally reported Appendix 2 values in 8 of 9 (tokenizer, dataset) cells (the RadBERT / MIMIC-CXR cell differs by 0.064) and preserves the declining-with-batch-size trend; however, at smaller batches the re-execution yields exact-sentence-accuracy values that differ substantially from those in Appendix 1 (for example, 64.69% versus 42.1% for GPT-2 on Discharge at \(B = 64\)), and it therefore should not be interpreted as reproducing the exact-sentence-accuracy values reported in the manuscript. Multimedia Appendix 8 is provided solely as the basis for the exact paired t-test analysis.

• Retrospective Attack Focus. This study focused solely on post hoc attack analysis. No privacy-preserving defenses (e.g., differential privacy, secure aggregation, anomaly detection) were implemented or evaluated. While this allows for a clean assessment of leakage potential, it leaves open questions about mitigation feasibility and deployment cost.

• Tokenizer Isolation vs.Full Model Effects. We controlled for model architecture to isolate the effect of tokenization on inversion risk. While informative, real-world systems often use both tokenizer and model co-adaptation. Thus, privacy risks may be higher or lower depending on the full pipeline design.

• Data partitioning. Our experiments use an approximately IID partition across six simulated clients. Real-world cross-institutional federated deployments are typically non-IID — site-specific protocols, demographic skew, and specialty-specific report distributions can introduce heterogeneity that affects gradient sparsity and consequently inversion vulnerability. The direction of this effect is not a priori clear (non-IID gradients carry stronger per-client signal but also higher variance), and a systematic non-IID evaluation is left for future work.

• Sequence length. The 32-token regime adopted here is shorter than typical clinical narrative passages. Longer sequences increase the input dimensionality \(m \times L\) of each per-sample reconstruction, which raises the number of bins \(K\) required for unambiguous disentanglement and thereby reduces single-step reconstruction success in practice. Our reported leakage rates are therefore an upper bound for the 32-token regime; longer-sequence regimes are expected to be harder to attack with a single fixed \(K\), all else equal. Extending the analysis to longer-sequence training is left for future work.

• Client count. Our experiments fix the simulated client count at six, reflecting a typical small-consortium clinical collaboration. The analytic gradient-inversion attack characterized in this study operates on a single client’s gradient at a single round, and the per-client per-round recovery rate is independent of the number of other clients participating in the federation. Aggregation effects across clients (which arise specifically under cryptographic secure aggregation or weighted averaging schemes that obscure individual client updates) are absent from this evaluation by design and are an explicit target for future defense-oriented work.

• Probe placement. The imprint module is inserted immediately before the positional embedding so that token-level representations are exposed prior to any token-mixing layer (positional addition, attention). Placement at later layers would conflate tokenizer-segmentation effects with the dynamics of attention and feed-forward mixing, and would not be informative for the central question of this study. A systematic placement ablation across pre-embedding, post-embedding, and post-attention positions — holding the tokenizer fixed — would isolate the contribution of placement itself and is identified here as a target for future work.

Taken together, these limitations do not diminish the central contribution highlighting a structural vulnerability in federated clinical NLP but emphasize the need for prospective, defense-integrated evaluations in more realistic medical AI pipelines.

Future Work↩︎

Several extensions to this study are warranted. First, systematic ablation of the imprint module’s placement (pre-embedding, post-embedding, post-attention) would empirically validate the design rationale used here. Second, evaluation of differential privacy as a defense — including utility–privacy trade-off curves at multiple \(\varepsilon\) values, per-example clipping schedules, and interaction with secure aggregation — is a substantial study in its own right and is the explicit subject of our planned follow-up work. Third, extension of clinical-entity recall to additional report types (Diagnosis, MIMIC-CXR) and to non-IID federated partitions would strengthen the external validity of the leakage estimates reported here.

Conclusions↩︎

Federated learning holds great promise for enabling collaborative, multi-site radiology AI without centralizing patient records. However, our gradient-inversion experiments demonstrate that current FL pipelines can leak substantial portions of patient text, raising significant concerns under HIPAA and GDPR standards. In worst-case scenarios, up to 44% of sentences were exactly reconstructed from shared gradients, and 18% of clinical terms, including diagnoses, procedures, and medications, were recovered from the reconstructed text.

Importantly, domain-adapted tokenizers such as RadBERT, while improving semantic fidelity, also showed greater vulnerability to data reconstruction attacks compared to general-purpose alternatives like GPT-2 and LLaMA-2. This highlights a critical trade-off between clinical performance and privacy risk.

To ensure safe deployment of federated clinical models, radiology departments and AI developers should rigorously evaluate privacy vulnerabilities in real-world settings, balancing utility with exposure risk. Regulatory bodies and standardization agencies may consider establishing clear audit pathways and certification frameworks tailored to the unique threats posed by clinical text.

Acknowledgements↩︎

This work was supported by Grant Number 01ZZ2316A-O. The authors thank Maximilian Zenk (Division of Medical Image Computing, DKFZ) for his valuable insights during manuscript preparation. We also appreciate the guidance and prior work of Jonas Geiping and Liam Fowl in developing federated transformer methods. We acknowledge the developers of the breaching library for providing the gradient-inversion framework used in this study. Generative AI tools were used to assist with language editing, grammar checking, and formatting. All scientific content, analyses, and conclusions are the authors’ own.

Funding Statement↩︎

This work was partially supported by the PrivateAIM project, funded under the Medical Informatics Initiative by the German Federal Ministry of Education and Research (funding code 01ZZ2316A-O). The funder had no involvement in the study design, data collection, analysis, interpretation, or writing of the manuscript.

Conflicts of Interest↩︎

The authors declare that they have no competing interests.

Data Availability↩︎

The radiology report corpora analyzed in this study are publicly available: the Dischargesum dataset and the MIMIC-CXR via PhysioNet. The code implementing the federated learning experiments and gradient inversion attacks is available from the corresponding author upon reasonable request. All other data generated or analyzed during this study are included in this published article and its supplementary information files.

Authors’ Contributions↩︎

SP: Conceptualization, Methodology, Software, Validation, Formal Analysis, Investigation, Data Curation, Writing — Original Draft, Writing — Review & Editing, Visualization, Project Administration. AM: Visualization, Writing — Review & Editing. DB: Writing — Review & Editing. SS: Resources, Supervision, Writing — Review & Editing. KMH: Supervision, Writing — Review & Editing. RF: Conceptualization, Resources, Supervision, Writing — Review & Editing.

Abbreviations↩︎

• AI: artificial intelligence

• BLEU: bilingual evaluation understudy

• DP: differential privacy

• FL: federated learning

• G-BLEU: global BLEU

• GDPR: General Data Protection Regulation

• HIPAA: Health Insurance Portability and Accountability Act

• LLM: large language model

• PHI: protected health information

• ROUGE-L: recall-oriented understudy for gisting evaluation

References↩︎

Multimedia Appendices↩︎

Multimedia Appendix 1↩︎

Exact sentence-level reconstruction accuracy (%) across batch sizes (64, 128, 256) for three tokenizers (GPT-2, RadBERT, and LLaMA-2) on three radiology datasets (Discharge, Diagnosis, and MIMIC-CXR). Accuracy is computed as the proportion of sentences perfectly reconstructed from shared gradients.

Multimedia Appendix 2↩︎

Mean reconstruction scores for S-BLEU (S), G-BLEU (G), and ROUGE-L (R) across batch sizes (64, 128, 256) for three tokenizers on three radiology datasets. Higher scores indicate greater recovery of clinical phrasing.

Multimedia Appendix 3↩︎

Distribution of exact sentence-level reconstruction accuracy across datasets, tokenizers, and batch sizes (violin plots over the five independent runs).

Multimedia Appendix 4↩︎

Distribution of ROUGE-L scores across datasets, tokenizers, and batch sizes (violin plots over the five independent runs).

Multimedia Appendix 5↩︎

Distribution of G-BLEU scores across datasets, tokenizers, and batch sizes (violin plots over the five independent runs).

Multimedia Appendix 6↩︎

95% confidence intervals for exact sentence-level reconstruction accuracy across batch sizes and tokenizers, derived from the five-run experiment underlying the main Results.

Confidence intervals were computed using Student’s \(t\)-distribution with \(n = 5\) independent runs per cell (degrees of freedom = 4, critical value \(t_{0.025} = 2.776\)). The standard deviation upper bound of 1 percentage point was used, as reported in the main Results (Section Quantitative Reconstruction Success); where the true per-cell standard deviation is below this upper bound, reported CI half-widths are conservative. Means are reproduced exactly from Multimedia Appendix 1.

Multimedia Appendix 7↩︎

Training details and reproducibility configuration. Hyperparameters and infrastructure-relevant settings used for all experiments reported in this study.

Appendix reference.
Geiping J, Fowl L, Huang WR, Czaja W, Taylor G, Moeller M, Goldstein T. Decepticons: corrupted transformers breach privacy in federated learning for language models. arXiv preprint arXiv:2201.12675; 2022. Available from: https://arxiv.org/abs/2201.12675

Multimedia Appendix 8↩︎

Verification Re-run — Cell Means and Exact Paired t-tests.

This appendix reports a re-execution of the experimental grid. The re-execution covers the full grid (3 tokenizers \(\times\) 3 batch sizes \(\times\) 3 datasets \(\times\) 5 seeds = 135 attack runs) and uses the same attack pipeline as the originally reported experiment (the imprint module of the breaching library, with \(K = 1000\) bins, 32-token sequences, and per-position cumulative-bin disentanglement). The per-run output is retained in a format suitable for direct paired-\(t\)-test computation. Seeds 0–4 were used for the re-execution. Cell means are reported in the first table below; exact paired \(t\)-tests on the per-seed exact-sentence reconstruction accuracy (in percentage points) are reported in the second table.

Comparison with originally reported values. The largest-batch values (\(B = 256\)) reproduce within 0.05 sBLEU of the originally reported Appendix 2 values in 8 of 9 (tokenizer, dataset) cells (the RadBERT / MIMIC-CXR cell differs by 0.064), and the declining-with-batch-size trend is preserved throughout. At smaller batches (\(B = 64\), 128) the re-execution produced sBLEU values somewhat higher than originally reported; we view this appendix as consistent with the qualitative findings of the manuscript rather than as a replacement for the originally reported reconstruction values, which remain in Appendices 1–2. Of 27 pairwise tokenizer comparisons at \(\alpha = 0.05\), one is statistically significant. With \(n = 5\) seeds per cell and observed per-cell standard deviation in the 2–7 percentage-point range, the power to detect tokenizer-level effect-size differences in the 2–4 pp range is limited; the absence of significance therefore reflects sampling power rather than absence of effect. Cell-level effect sizes preserve the manuscript’s qualitative ordering at the largest batch on the Discharge dataset (RadBERT 28.52%, GPT-2 27.34%, LLaMA-2 27.50% mean exact-sentence accuracy).