Radiology reports serve as an indispensable tool in medical diagnostics, providing critical insights [1] that complement imaging data. These textual data contain rich clinical information, including patient history, diagnostic impressions, and recommended follow-ups, often bridging gaps left by imaging alone. In recent years, advancements in AI, particularly transformer-based large language models (LLMs) [2], have revolutionized natural language processing (NLP), making it possible to analyze unstructured textual data at scale. LLMs such as the GPT family [3] are now being explored for their potential in radiology to assist in generating summaries [4], extracting relevant clinical insights, and identifying patterns across large datasets [5-7]. A recent systematic review of LLM evaluations in clinical medicine highlights the rapid growth of these models and underscores the need for robust evaluation frameworks to ensure their safety, reliability, and ethical alignment in health care applications [8].

Radiology LLMs can be collaboratively developed across multiple institutions, leveraging diverse clinical reports to enhance model robustness and improve generalizability across unseen health care settings. Nevertheless, health care data, such as radiological data sharing, faces significant challenges due to stringent data protection laws such as the Health Insurance Portability and Accountability Act (HIPAA) [9] in the United States and the General Data Protection Regulation (GDPR) [10] in the European Union, which mandate strict safeguards for the privacy of sensitive patient data. Federated learning (FL) [11], a decentralized machine learning approach where models are trained across multiple institutions without exchanging raw data, has emerged as a promising solution to address these challenges by ensuring that sensitive data remain within each participating institution while still enabling collaborative model development. Multicentric FL clinical collaborations [12-15] are transforming medical research by enabling privacy-preserving data sharing, fostering innovation, and addressing critical concerns around data ownership and security.

Nevertheless, FL’s decentralized nature introduces inherent vulnerabilities that may compromise patient confidentiality [16,17]. While they eliminate direct data sharing, FL systems rely on the exchange of model parameters between institutions and a central server. An active malicious server—one that not only observes gradients but also modifies the shared model architecture before distribution—can exploit these parameters to reconstruct sensitive data, exposing significant vulnerabilities. Existing research has shown that gradient-inversion attacks [18-20] can reconstruct private text from models trained on generic language datasets, but such risks remain largely unexplored in the domain of radiology [21].

Recent advances in NLP have demonstrated that leveraging domain-specific tokenizers [18] and training LLMs on domain-adapted corpora significantly enhances performance in specialized fields such as medicine and genomics. For instance, studies such as Gu et al [22] introduced PubMedBERT, showing that pretraining on biomedical literature yields superior results on downstream clinical tasks compared to generalist models. Similarly, prior studies such as Zhang et al [23] demonstrate that fine-tuning LLMs such as BioBERT and MedAlpaca with domain-specific data and tailored tokenization strategies substantially enhances performance in medical tasks. These adaptations enable the models to capture nuanced, domain-relevant semantics that general-purpose LLMs typically overlook.

To specifically investigate the role of tokenization in privacy leakage within FL, we isolate tokenizer design as the primary variable while keeping the underlying model architecture constant. While most prior work compares entire language models [8], we argue that vocabulary segmentation alone can influence the susceptibility of models to gradient-inversion attacks. Tokenizers trained on domain-specific corpora (eg, RadBERT) are more likely to encode medical terms as single tokens, increasing their semantic coherence; we hypothesized that this would also make such terms easier to reconstruct—a prediction our controlled experiments ultimately did not support (see the “Results” section). In contrast, general-purpose tokenizers tend to fragment clinical phrases into multiple subwords, which may reduce both interpretability and recoverability. This design choice not only impacts downstream clinical utility [22] but might, we hypothesized, also alter the reidentifiability of sensitive entities during model inversion—a prediction our controlled experiments did not support. By decoupling the tokenizer from the model architecture, our approach provides a targeted evaluation of how vocabulary structure relates to the balance between semantic fidelity and patient privacy.

In our controlled evaluation, the radiology-specific RadBERT [24] tokenizer did not yield significantly higher reconstruction fidelity under attack than general-purpose tokenizers such as GPT-2 [25] and LLaMA-2 [26]. This indicates that the advantages of domain-adapted pretraining for capturing clinical semantics do not come at the cost of measurably greater gradient-inversion leakage; the vulnerability is intrinsic to the attack and is governed primarily by batch size—though under this imprint-probe construction the batch-size effect operates proximately through increased bin-collision burden rather than as a general law of federated training. This observation aligns with evidence that tokenizer and model design choices can influence performance in domain-specific applications, raising the question of whether they likewise affect privacy risk under gradient inversion. Building on this insight, we investigate the extent to which radiology report data can be reconstructed from transformer-based models trained in a simulated FL environment. By orchestrating targeted attacks in a multiclient setup using publicly available radiological datasets, we expose critical vulnerabilities in current FL frameworks. Our study deliberately focuses on the attack surface and does not evaluate defense mechanisms or privacy-preserving strategies. Rather, it aims to characterize worst-case risks and motivate future work toward robust, domain-aware privacy protections for safe deployment of clinical foundation models.

This work establishes a foundational analysis of the privacy vulnerabilities inherent in applying transformer-based FL to radiology reports. We focus on characterizing the attack surface by quantifying the reconstruction risk under a worst-case gradient-inversion scenario. To this end, we provide a systematic comparison of information leakage across major tokenizers, namely, GPT-2, RadBERT, and LLaMA-2. Our central contribution is a controlled demonstration that, with model architecture held fixed, tokenizer choice—including domain-specific tokenizers like RadBERT—does not significantly change the risk of leaking sensitive patient data, whereas batch size does; practitioners therefore should not, in this setting, rely on tokenizer selection as a privacy safeguard. These findings provide a necessary benchmark and guidepost for developing effective privacy-preserving defenses in medical FL systems. Clinical utility is not directly evaluated in this study and is inferred from prior work establishing improved downstream performance with domain-specific clinical tokenizers.

This study is a secondary computational analysis of 2 publicly available, previously deidentified clinical text corpora: the dischargesum dataset [27] and the MIMIC-CXR (Medical Information Mart for Intensive Care Chest X-Ray) free-text radiology reports accessed via PhysioNet [28] under the standard PhysioNet credentialed-access Data Use Agreement. No new data from human participants were collected as part of this work. No identifiable individuals appear in any figure, table, or supplementary material. No participants were recruited, and no compensation was provided. Per institutional policy at the German Cancer Research Center (DKFZ) and consistent with PhysioNet’s terms of use, secondary computational analysis of these deidentified public corpora qualifies for ethics board exemption; the original institutional review board approvals covering the primary collection of MIMIC-CXR (Beth Israel Deaconess Medical Center) and Dischargesum extend to secondary research use without additional consent. All experiments were performed on local DKFZ compute infrastructure and no patient data were transmitted to external services.

FL [11] is a decentralized machine learning paradigm that allows multiple institutions to collaboratively train a common model without exchanging raw data. Each client trains locally and sends only gradient updates to a central server for aggregation. In our implementation, we simulated this multi-institutional setup using a GPT-2–style transformer (instantiated as a compact 3-layer encoder with token-embedding dimension m=96, 8 attention heads, feed-forward hidden size 1536, and approximately 13.4 M trainable parameters, following the field guide to FL baseline architecture [29]) combined with 3 different tokenizers. The overall attack pathway is illustrated in Figure 1.

All tokenizers operated with a fixed sequence length of 32 tokens, so any observed privacy differences are attributable to vocabulary segmentation rather than to input shape or representational scale. Holding the foundation-model architecture fixed at this small, well-characterized baseline allows the tokenizer to serve as the sole independent variable across our GPT-2, RadBERT, and LLaMA-2 comparisons. The 32-token sequence length follows the field guide to FL baseline configuration [29] and is consistent with prior gradient-inversion experiments on text models [30]; it provides a controlled, computationally tractable regime in which the imprint construction’s bin disentanglement is well-characterized. To emulate real multi-institutional training, we randomly partitioned 2 radiology text datasets across 6 cloud‐based clients: the public Dischargesum corpus (98,206 discharge summaries; 368,751 diagnostic reports) and a locally curated sample of 1500 free-text reports from MIMIC-CXR. Training was repeated with batch sizes of 64, 128, and 256 sequences to measure how aggregation granularity affects privacy leakage in shared gradients. Batch sizes refer throughout to 32-token sequences, not whole reports: each report is tokenized and split into multiple nonoverlapping 32-token windows; a single discharge summary or radiology report typically contributes between 5 and 40 sequences depending on its length, so the per-client local datasets contain ≥5000 sequences and comfortably exceed the largest batch size of 256. Gradient-inversion attacks of the type considered here operate on the gradient produced by a single client during a single training round, which constitutes the standard threat-surface unit in the gradient-leakage literature [30]; privacy leakage was therefore quantified per (client and round) rather than over the aggregated federation-level update.

Let the token-embedding dimension be m, and the probe output dimension be k. For a single token position, the hidden vector x ∈ ℝ^m is produced by the embedding table before positional addition and attention mixing. We insert a small linear probe (an “imprint” module in the sense of [30]) on top of this hidden state:

y=W2⋅ReLU(W0x+b0) with W₀ ∈ ℝ^k×m, b₀ ∈ ℝ^k, and W₂ ∈ ℝ^m×k, where k denotes the number of probe bins. The probe is appended once at initialization and trained jointly with the rest of the model. As in standard cross-silo FL (FedAvg [11]), each client transmits only the parameter gradient ∇θ ℒ of its local loss back to the server; no intermediate activations or hidden states are exchanged. This rules out split-learning style leakage and isolates the threat surface to the gradient update itself.

We initialize b₀ to the cumulative quantiles of a Laplace prior over the expected preactivation distribution and W₀ to a random Gaussian matrix; W₂ is initialized so that y≈x in expectation, leaving downstream training behavior essentially unchanged. Under this construction, the bins partition the input space into cumulative half-spaces of geometrically decreasing measure, so that for sufficiently large k at most one sample in a mini-batch activates each bin after consecutive-bin differencing. In general, batch aggregation destroys per-sample information: the transmitted gradient ∇θ ℒ averages over all B samples in the batch and is not invertible without optimization [30]. The cumulative-bin imprint construction circumvents this barrier by analytic design rather than by optimization—the contribution of each individual token to ∇W₀ and ∇b₀ is structurally separated by bin index i, so the server can isolate per-sample gradients even though it only ever observes the batch sum.

From these 2 captured gradients, the input hidden vector x can be recovered in closed form without optimization: x^j=▽W0,i▽b0,i

where i is the bin uniquely activated by token j after the cumulative-bin differencing step ∇W_₀,i ← ∇W_₀,i − ∇W_₀,i−1 (and analogously for ∇b_₀,i). We apply this per token and per sample in a batch; sentence reconstructions are formed by decoding each xj to the nearest token in the active tokenizer’s embedding table E ∈ ℝ^V×m under cosine similarity, where V is the tokenizer vocabulary size. We set k=1000 bins and m=96, with xj L2-normalized before decoding; when ‖∇b₀,i‖₂ is below a numerical threshold ε, we add a ridge term ε to the denominator for numerical stability.

Ties or near-ties (cosine margin < δ) among candidate tokens are broken by rescoring with the model’s language-model-head logits in a server-side forward pass conditioned on the previously-decoded prefix of the same sample—feasible because the malicious server holds the full shared model and can run forward passes on partial reconstructions. Token j is therefore resolved in left-to-right context rather than in isolation; the very first position falls back to the unconditional logit prior.

We assessed reconstruction fidelity using 3 complementary metrics chosen to capture both exact and partial text recovery, following established evaluation practice for reconstructed clinical text [14-17,24]. These metrics together quantify the extent to which patient-identifiable information can be recreated from shared model updates.

By combining these 3 metrics, we quantify both the frequency of perfect reconstructions and the degree of partial text recovery, offering a comprehensive assessment of patient data exposure under our FL attack.

BLEU and ROUGE-L were originally proposed for translation and summarization quality, where their numerical values are bounded by human-level n-gram overlap. In the gradient-inversion context, we use them as information-recovery metrics rather than as quality metrics: a higher BLEU or ROUGE-L between a reconstructed sentence and the original indicates more shared n-gram content—that is, a greater amount of original wording recovered. Exact-sentence accuracy provides the strict-recovery upper bound; S-BLEU and ROUGE-L provide the partial-recovery signal that captures cases where most of a sentence is recovered but a small number of tokens are misdecoded. This complementary use of strict and partial metrics is consistent with prior gradient-inversion text studies [37]. We deliberately do not adopt semantic-similarity metrics such as BERTScore in the main results because they can credit paraphrastic matches that do not constitute literal information leakage, which is the privacy question of interest here.

On the discharge dataset, exact sentence reconstruction accuracy across tokenizers and batch sizes was as follows:

On MIMIC-CXR, reconstruction ranged from 28.7% to 74.7% of sentences (Figure 3), showing that neither tokenizer choice nor larger batch sizes fully mitigates data leakage. Detailed accuracy results are provided in Multimedia Appendix 6. All reported reconstruction metrics represent the mean of 5 independent runs; 95% CIs across runs are reported in Multimedia Appendix 3; exact paired t tests are reported in Multimedia Appendix 4. Across configurations, per-cell SDs ranged from approximately 1.5 to 7 percentage points.

Increasing the batch size led to a clear reduction in average reconstruction fidelity across all metrics (Multimedia Appendix 7; Figure 2). For discharge reports, mean S-BLEU decreased from 0.69 at a batch size of 64 to 0.31 at 256; diagnosis reports fell from 0.68 to 0.31; and MIMIC-CXR from 0.71 to 0.32. Parallel trends were observed for ROUGE-L, confirming that coarser gradient aggregation generally mitigates privacy leakage. Notably, across datasets and batch sizes, the 3 tokenizers achieved statistically indistinguishable reconstruction scores. This indicates that domain-specific tokenization does not measurably facilitate more accurate recovery of clinical language, and that tokenizer choice is not a privacy-relevant design lever in this setting. Corresponding mean S-BLEU and ROUGE-L scores are summarized in Multimedia Appendix 7.

While average reconstruction fidelity declined with larger batch sizes across all metrics, individual training rounds revealed substantial fluctuations in leakage severity. We illustrate this variability using S-BLEU in Figure 2, as it provides a balanced measure of local n-gram precision and sentence-level structure, making it particularly sensitive to round-to-round differences. However, similar variability patterns were also observed for exact sentence reconstruction accuracy and ROUGE-L, which are summarized in Multimedia Appendices 1 and 2. Even at the highest batch size (256), certain rounds produced high-fidelity reconstructions comparable to those at batch size 64, underscoring a persistent privacy risk. Coarser gradient aggregation therefore reduces but does not eliminate patient information exposure in FL.

In real-world medical AI deployments, even a single outlier round with high leakage could compromise patient confidentiality. Therefore, it is essential to characterize not just average-case leakage but also variability across rounds, to ensure that system-level guarantees account for worst-case scenarios.

As presented in Table 1, reference-vocabulary overlap was high and statistically indistinguishable across tokenizers: MedGemma-extracted clinical-concept overlap averaged 75.1% (GPT-2), 74.1% (RadBERT), and 72.9% (LLaMA-2) across datasets and seeds, with no pairwise difference reaching significance.

These findings demonstrate that gradient-inversion attacks can reveal not only structural sentence fragments but also clinically meaningful entities, posing a tangible privacy risk. That overlap remains high (approximately 75%) regardless of the tokenizer indicates that this leakage is intrinsic to the attack rather than a property of any particular tokenizer in federated training settings.

Tables 2-4 showcase original and reconstructed outputs from the dischargesum, radiology, and MIMIC-CXR datasets, illustrating the clinical content recoverable across all 3 tokenizers (aggregate metrics show no significant tokenizer difference). These examples use oracle alignment for readability only: the positional ordering shown is not available to the attacker, who observes an unordered or partially ordered set of recovered tokens, so the aligned display illustrates recoverable content rather than an attacker’s verbatim output. Across the reconstructions, all 3 tokenizers recovered substantial clinical content while also dropping or fragmenting individual terms; in the specific example shown, recovered entities included terms such as “catheter” and “nodularity.” Which terms were retained or dropped varied from example to example, and—consistent with the aggregate analysis, which found no significant tokenizer difference—these single-example observations should not be read as a systematic tokenizer ordering. These qualitative results reinforce the quantitative findings, demonstrating that even domain-adapted transformer models remain susceptible to gradient-based inversion attacks, recovering not only template phrases but also clinically meaningful patient information.

This study demonstrates that even with domain-specific tokenization and larger batch sizes, transformer models in an FL setup leak substantial patient information. The active server reconstructed up to approximately 75% of radiology-report sentences. On the Discharge dataset, exact-sentence accuracy was 64.7%/46.4%/27.3% for GPT-2, 70%/50.8%/28.5% for RadBERT, and 67.5%/48.4%/27.5% for LLaMA-2 at batch sizes 64/128/256. S-BLEU similarly declined with batch size (eg, GPT-2 on discharge reports from 0.66 to 0.31), yet even at the largest batch sizes roughly a quarter of sentences were fully recovered. Across batch sizes and datasets, reconstruction fidelity did not differ significantly among the 3 tokenizers (mean exact-sentence accuracy 48.3% for each; no pairwise comparison significant). After Holm correction for the 27 pairwise comparisons, no tokenizer difference remained significant (all adjusted P=1.00), and a 3-way variance decomposition attributed 94% of the variance in exact-sentence accuracy to batch size versus less than 1% to tokenizer, corroborating batch size—not tokenizer choice—as the dominant factor; the pairwise tokenizer effect sizes were negligible (mean differences ≤0.05 percentage points; Cohen d≤0.03). All 3 tokenizers, including the domain-specific RadBERT, leaked substantial clinical content, and none eliminated leakage. These findings confirm that neither increasing batch sizes nor specialized tokenizers alone can ensure patient privacy in FL‐trained LLMs.

To further disambiguate templated language from truly clinical leakage, we applied MedGemma NER to the reconstructed text across all 3 datasets. Across models, approximately 75% of reference clinical entities were recovered (Table 1). These findings confirm that privacy risks extend beyond boilerplate phrasing to include clinical concepts relevant to diagnosis and care.

Our findings underscore that tokenizer design is not a decisive factor in privacy leakage under gradient inversion attacks. In a controlled named-entity analysis, clinical-concept reference-vocabulary overlap was statistically indistinguishable across tokenizers, with approximately 75% of each dataset’s reference vocabulary appearing under each (GPT-2 75.1%, RadBERT 74.1%, LLaMA-2 72.9%; no pairwise difference significant). This indicates that domain-specific vocabulary segmentation does not, by itself, make reidentification-relevant clinical content easier to reconstruct. Within the evaluated architecture, attack, datasets, and metrics, tokenizer choice should therefore not be treated as a privacy safeguard; batch size and explicit defenses are the privacy-relevant levers. Future privacy audits and risk assessments of federated clinical models should focus on batch size and explicit defenses rather than tokenizer choice, especially as domain-specific LLMs become more prevalent in medical AI.

We initially reasoned that domain-specific tokenization should heighten reconstruction. The reconstruction step decodes each recovered token-embedding vector back to the nearest token in the active tokenizer’s embedding table. A clinical concept such as “pneumothorax” is encoded as a single token in RadBERT’s domain-specific vocabulary, so a successful nearest-neighbor decode would yield the entire concept verbatim, whereas the same concept is segmented into multiple subwords (eg, “p/neu/moth/orax”) in GPT-2’s byte-pair encoding, where recovering it requires every subword embedding to be reconstructed correctly and placed in the correct positional order. Under this reasoning, single-token clinical terms should present fewer points of failure. We tested this prediction directly—including a prespecified analysis restricted to clinical terms that GPT-2 fragments into multiple subwords, where any single-token advantage should be largest—and found no significant tokenizer difference in exact-sentence accuracy, clinical-concept overlap, or the fragmented-term subset. The hypothesized single-token advantage is therefore not borne out empirically: across the datasets and batch sizes studied, the closed-form attack recovers clinical content at comparable rates regardless of tokenizer, and batch size is the dominant factor. This controlled negative result indicates that, in this setting, tokenizer selection should not be treated as a privacy safeguard.

The MedGemma NER model used in this study captures clinical concepts (diagnoses, procedures, anatomical structures, and medications) rather than direct identifiers under the HIPAA Safe Harbor enumeration (names, medical record numbers, dates more granular than year, full-face photographic images, and so on). Direct identifiers are largely already redacted in the public Dischargesum and MIMIC-CXR corpora before release. The recovered terms therefore reflect reidentification potential through clinical context—for example, an unusually rare diagnosis, a unique procedure pattern, or a specific medication regimen that, in combination with auxiliary recovered context, could plausibly contribute to reidentifying a patient—rather than direct exposure of an enumerated identifier. We make this distinction explicit because the regulatory framing of “PHI” is broader than “direct identifier”; HIPAA’s expert-determination standard (45 CFR §164.514(b)(1)) treats clinical context recoverable from a record as part of identifiability risk, and the same logic is reflected in the GDPR’s notion of indirect identifiability via singling out.

The imprint module’s analytic recovery rule x̂_j =∇W_₀,i/∇b_₀,i relies on the assumption that bin i is activated by at most one sample in the batch. With k=1000 bins and batch size B, the expected number of samples per bin is B/k, and the probability that any given bin is hit by ≥2 samples grows approximately quadratically in B (a birthday-paradox argument). When 2 or more samples share a bin, the recovered ratio ∇W_₀,i/∇b_₀,i becomes a gradient-weighted average of distinct embeddings rather than a clean single-sample recovery, and the corresponding nearest-neighbor decode is corrupted. This explains why exact-sentence accuracy decays with batch size while never reaching zero: as long as some bins remain singly occupied, the corresponding samples are recovered cleanly. The same argument predicts that increasing k relative to B would partially compensate for batch growth—a direction we leave to future work.

We provide the first systematic evaluation of how different transformer tokenizers impact gradient‐inversion vulnerability on radiology text, showing that domain‐specific tokenization (RadBERT) does not significantly change leakage risk relative to general-purpose tokenizers, even as it better preserves clinical terminology, whereas batch size does.

This study was designed as a worst-case evaluation, assuming a fully active malicious server capable of modifying model architecture and exploiting gradient updates. While extreme, this assumption is appropriate for international or cross-consortium federated settings, where governance may be inconsistent and insider threats cannot be ruled out.

Several constraints limit the generalizability of our findings:

Taken together, these limitations do not diminish the central contribution highlighting a structural vulnerability in federated clinical NLP but emphasize the need for prospective, defense-integrated evaluations in more realistic medical AI pipelines.

Several extensions to this study are warranted. First, systematic ablation of the imprint module’s placement (preembedding, postembedding, and postattention) would empirically validate the design rationale used here. Second, evaluation of differential privacy as a defense—including utility-privacy trade-off curves at multiple ε values, per-example clipping schedules, and interaction with secure aggregation—is a substantial study in its own right and is the explicit subject of our planned follow-up work. Third, extension of the reference-vocabulary overlap analysis to source-report-aligned entity recovery and to non-IID federated partitions would strengthen the external validity of the leakage estimates reported here.

FL holds great promise for enabling collaborative, multisite radiology AI without centralizing patient records. However, our gradient-inversion experiments demonstrate that current FL pipelines can leak substantial portions of patient text, raising significant concerns under HIPAA and GDPR standards. In worst-case scenarios, up to approximately 75% of sentences were exactly reconstructed from shared gradients, and approximately 75% of clinical entities, including diagnoses, procedures, and medications, were recovered from the reconstructed text.

Importantly, domain-adapted tokenizers such as RadBERT, while improving semantic fidelity, did not show significantly greater vulnerability to data reconstruction attacks than general-purpose alternatives such as GPT-2 and LLaMA-2. The privacy risk is substantial for all tokenizers and is governed by batch size rather than tokenizer choice.

To ensure safe deployment of federated clinical models, radiology departments and AI developers should rigorously evaluate privacy vulnerabilities in real-world settings, balancing utility with exposure risk. Regulatory bodies and standardization agencies may consider establishing clear audit pathways and certification frameworks tailored to the unique threats posed by clinical text.