1 Introduction↩︎

Federated Learning (FL) [1] has emerged as a powerful paradigm for training models collaboratively across decentralized data, under the promise of privacy preservation. Nonetheless, the presence of heterogeneous data across clients poses a significant challenge, often leading to slow convergence and suboptimal model performance. A wide range of approaches have been proposed to address this, from personalized federated learning (pFL) [2]–[7] to data and representation sharing strategies [8]–[13]. Despite their effectiveness, these methods typically require many communication rounds which can be problematic for real-world cross-device deployment.

A particularly attractive direction for communication constrained applications is one-shot federated learning (OSFL), where clients communicate with the server exactly once. Recent OSFL methods leverage the aforementioned data sharing principle, showing the efficacy in improving one-shot performance. Methods such as FedD3 [14], FedSD2C [15], FedCVAE [16], DENSE [17], CoBoosting [18] and [19] approach OSFL by having each client train a local generative model or distill its data, then transmit the result to the server, which aggregates the contributions into a shared synthetic dataset or ensemble. These approaches thus eliminate repeated parameter exchanges, naturally support heterogeneous architectures, and can produce datasets that can be reused for pretraining, distillation, or personalization.

However, existing OSFL methods face a fundamental limitation: few provide formal privacy guarantees as a built-in mechanism with regards to preserving utility. Privacy leakage occurs because clients transmit trained model parameters, generator weights, or distilled data, where information about individual training records can be leaked. Retrofitting differential privacy (DP) [20] onto these approaches is difficult; the high dimensionality of model parameters inflates sensitivity, DP noise degrades already-fragile local models, and multi-step training complicates privacy accounting. Meanwhile, training local generators and models such as in [16]–[18] can also heavily increase computational cost on clients.

This leads to a direct question: Can we devise a framework which leverages the benefits of representation sharing in an OSFL setting with formal privacy guarantees and lightweight client computation?

We observe that modern pretrained autoencoders, originally developed for neural image compression [21], [22], provide exactly the structure needed to answer this question. These models map images to compact latent vectors and back with high fidelity, using publicly available weights that require no adaptation. The key insight is that, given such a shared public encoder, each client can encode its private images in a single forward pass and compute simple per-class statistics in the latent space. These statistics can be aggregated across clients using secure aggregation, which we implement such that partitioning data across any number of clients produces exactly the same as if data were aggregated centrally. Moreover, the latent vectors are also low-dimensional and have bounded sensitivity, making them ideal targets for calibrated Gaussian noise under differential privacy [23].

Building on this insight, we propose FedKT-CSD (Federated Knowledge Transfer via Collaborative Synthetic Data). Each client receives a frozen, publicly pretrained autoencoder and encodes its private images via a single forward pass with no optimization or training on-device. The client computes class-conditional latent statistics, protects them under differential privacy, and transmits only these lightweight summaries to the server. The server aggregates the noisy statistics across all clients, recovers per-class Gaussian distributions in latent space, samples from them, and decodes the samples into a synthetic image dataset. This dataset can then be used to train a global classifier, a feature extractor for personalized FL, or auxiliary data for multi-round methods.

Our approach achieves three critical objectives simultaneously: (1) privacy by design, since the autoencoder is public and frozen and only DP-protected class-conditional statistics are transmitted; (2) communication efficiency, as the approach requires a single communication round where each client uploads only statistics rather than model parameters; and (3) computational efficiency, since clients perform only a forward pass through the encoder with no on-device training, optimization, or gradient computation.

Our contributions are:

• One-shot DP synthetic data generation. We show that deep compression autoencoders provide a shared latent space in which class-conditional statistics can be computed with differential privacy. This enables a one-shot FL pipeline that generates synthetic data with formal \((\varepsilon,\delta)\)-DP guarantees, a single communication round, and no on-device training.

• Competitive with SOTA under DP. Despite operating under DP, FedKT-CSD is competitive and outperforms existing one-shot FL baselines across diverse datasets and heterogeneity settings. Moreover, our method is also invariant to both the degree of heterogeneity and the number of clients, properties that no existing OSFL method possesses.

• Versatile downstream use. The DP synthetic dataset is not limited to server-side training. We demonstrate its effectiveness for downstream personalized FL, where it serves as pretraining data and achieves improvements over multi-round personalized FL baselines.

• Practical efficiency. Each client uploads payloads in the regime of Kilobytes and requires seconds of computation with no GPU training. This makes FedKT-CSD deployable in bandwidth- and compute-constrained settings where existing methods are impractical.

2 Related Works and Preliminaries↩︎

One-Shot Federated Learning. One-shot FL methods can be organized by how clients communicate with the server. One family transmits local models or generators, from which the server extracts knowledge via distillation. DENSE [17] and Co-Boosting [18] follow this approach, aggregating client model ensembles through data-free knowledge distillation. Both evaluate privacy empirically through membership inference but provide no formal guarantees. A second family transmits data, or data-derived features, ranging from distilled datasets, synthetic samples, or parametric distributions. FedD3 [14] constructs compact client summaries via dataset distillation. FedSD2C [15] optimizes synthetic distillates through a pretrained autoencoder to minimize information loss relative to the original data. Similarly, [16] shares locally trained generators to avoid model aggregation, though without privacy analysis. Most closely related to our work is FedPFT [19], which also encodes images through a frozen pretrained model and transmits class-conditional Gaussian mixtures in a single round. Our approach differs in that we aggregate additive statistics rather than per-client mixture models, enabling secure aggregation, exact partition invariance, and cleaner DP accounting. In the FedPFT privacy model, noise is added per-client to high-dimensional GMM parameters, which does not benefit from secure aggregation and is likely to degrade at meaningful privacy budgets. Moreover, we use an autoencoder rather than a discriminative feature extractor, producing a reusable pixel-space synthetic dataset rather than features tied to a specific classifier head, which is more useful for downstream tasks. We discuss the precise differences further in the appendix.

Data and Representation Sharing. Underpinning the recent works in OSFL are works on data sharing. To improve transferability, recent work explores sharing auxiliary information beyond parameters. FedGen [9] and FedFTG [10] use knowledge distillation from client logits to train server-side generators or refine global models. CCVR [11] and FedFed [12] share representation statistics to calibrate classifiers or align features. In addition, [13] use data derived from XAI-attribution methods, and [24] uses diffusion models to account for classifier drift. More recently, foundation models enable high-quality synthetic data generation: Pre-Text [25], dp-prompt [26], and dp-kde [27] leverage LLMs in the text domain. These approaches typically require large pretrained models and may incur significant computational or privacy costs.

3 Methodology↩︎

3.1 Problem Setting↩︎

We consider one-shot federated learning (OSFL). \(N\) clients hold private datasets \(\mathcal{D}_i = \{(x_{i,j}, y_{i,j})\}_{j=1}^{n_i}\) with \(y_{i,j} \in \{1,\dots,K\}\), drawn from heterogeneous distributions. The goal is to produce a global model \(f_\theta\) minimizing \[\label{eq:objective} \min_\theta \; \sum_{i=1}^N \frac{n_i}{\sum_j n_j}\,\mathcal{L}_{\mathcal{D}_i}(\theta),\tag{1}\] subject to (i) a single communication round and (ii) record-level differential privacy. Our key observation is that statistics of latent representations are additive: class-conditional sums computed locally combine across clients without information loss, so each client transmits only lightweight summary statistics from which the server generates a DP synthetic dataset \(X^*\) (1). We follow the standard conditional-generation setting where class labels are treated as public information. The privacy guarantee therefore protects the private feature content of each record, not its class membership although we outline how to achieve this in the appendix.

3.2 Pretrained Autoencoder and Encoding↩︎

We use a publicly pretrained autoencoder with encoder \(\mathcal{E}\colon \mathbb{R}^{H \times W \times C} \to \mathbb{R}^d\) and decoder \(\mathcal{D}\colon \mathbb{R}^d \to \mathbb{R}^{H \times W \times C}\), both frozen throughout.

Each client \(i\) encodes its images via \(z_{i,j} = \mathcal{E}(x_{i,j})\), a single forward pass with no optimization or training on-device. After clipping each latent to \(\ell_2\)-norm at most \(R\) (Section 4.1), the client groups its samples by class label. Let \(\mathcal{I}_{i,c} = \{j : y_{i,j} = c\}\) denote the index set of client \(i\)’s samples belonging to class \(c\). For each class it holds, the client computes three quantities: \[\label{eq:client95stats} \mathbf{m}_c^i = \sum_{j \in \mathcal{I}_{i,c}} \bar{z}_{i,j}, \qquad \mathbf{M}_c^i = \sum_{j \in \mathcal{I}_{i,c}} \bar{z}_{i,j}\,\bar{z}_{i,j}^\top, \qquad n_c^i = |\mathcal{I}_{i,c}|,\tag{2}\] where \(\bar{z}_{i,j} = \mathrm{Clip}_R(z_{i,j})\) is the clipped latent vector. Here \(\mathbf{m}_c^i \in \mathbb{R}^d\) is the sum of clipped latents for class \(c\) (not the mean), \(\mathbf{M}_c^i \in \mathbb{R}^{d \times d}\) is the sum of their outer products, and \(n_c^i\) is the number of class-\(c\) samples held by client \(i\). The triple \((\mathbf{m}_c^i, \mathbf{M}_c^i, n_c^i)\) is all that leaves the client. We transmit sums rather than means because sums are additive: they can be aggregated across clients without knowing per-client sample counts, compose naturally with secure aggregation (Section 3.3), and have bounded per-record sensitivity for DP.

3.3 Aggregation and Noise Injection↩︎

The server collects the per-client triples and aggregates them via secure aggregation, a cryptographic protocol that computes element-wise sums across clients while revealing only the totals to the server, never individual contributions (we describe secure aggregation further in the appendix). The resulting global quantities are: \[\label{eq:global95sums} \mathbf{S}_{\mu,c} = \sum_{i=1}^N \mathbf{m}_c^i \in \mathbb{R}^d, \qquad \mathbf{S}_{\Sigma,c} = \sum_{i=1}^N \mathbf{M}_c^i \in \mathbb{R}^{d \times d}, \qquad n_c = \sum_{i=1}^N n_c^i.\tag{3}\] Because these are simple sums, the result is identical to what would be computed if all class-\(c\) data resided on a single machine; the partitioning across clients has no effect. Gaussian noise calibrated to the sensitivity bounds (Proposition 1) is then added to the aggregated sums: \[\label{eq:noisy95sums} \widetilde{\mathbf{S}}_{\mu,c} = \mathbf{S}_{\mu,c} + \eta_\mu, \qquad \widetilde{\mathbf{S}}_{\Sigma,c} = \mathbf{S}_{\Sigma,c} + \eta_\Sigma,\tag{4}\] where \(\eta_\mu \sim \mathcal{N}(0,\sigma_\mu^2 I_d)\) and \(\eta_\Sigma \sim \mathcal{N}(0,\sigma_\Sigma^2 I_{d \times d})\). After this step, all subsequent operations (normalization, sampling, decoding, model training) are deterministic functions of the noisy sums and incur no additional privacy cost.

3.4 Post-Processing↩︎

Given the noisy sums \((\widetilde{\mathbf{S}}_{\mu,c}, \widetilde{\mathbf{S}}_{\Sigma,c}, n_c)\), the server recovers class-conditional statistics. The DP-protected mean \(\boldsymbol{\mu}_c^{\mathrm{DP}} \in \mathbb{R}^d\) and raw covariance estimate \(\widehat{\boldsymbol{\Sigma}}_c \in \mathbb{R}^{d \times d}\) are: \[\label{eq:dp95mean} \boldsymbol{\mu}_c^{\mathrm{DP}} = \frac{\widetilde{\mathbf{S}}_{\mu,c}}{n_c}, \qquad \widehat{\boldsymbol{\Sigma}}_c = \frac{\widetilde{\mathbf{S}}_{\Sigma,c}}{n_c} - \boldsymbol{\mu}_c^{\mathrm{DP}}\,(\boldsymbol{\mu}_c^{\mathrm{DP}})^\top.\tag{5}\] The first term \(\widetilde{\mathbf{S}}_{\Sigma,c}/n_c\) is the noisy second moment; subtracting the outer product of the mean yields the covariance. Two corrections are needed before \(\widehat{\boldsymbol{\Sigma}}_c\) can be used for sampling:

3.4.0.1 Bias correction.

Because \(\boldsymbol{\mu}_c^{\mathrm{DP}}\) is itself noisy, the subtracted outer product is biased upward: \(\mathbb{E}[\boldsymbol{\mu}_c^{\mathrm{DP}}(\boldsymbol{\mu}_c^{\mathrm{DP}})^\top] = \boldsymbol{\mu}_c\boldsymbol{\mu}_c^\top + (\sigma_\mu/n_c)^2 I_d\), where the second term is the variance of the noise in the mean. This increases the subtracted quantity, decreasing the covariance estimate along the diagonal. We correct analytically: \[\label{eq:bias95correction} \widehat{\boldsymbol{\Sigma}}_c \;\leftarrow\; \widehat{\boldsymbol{\Sigma}}_c + \frac{\sigma_\mu^2}{n_c^2}\,I_d.\tag{6}\]

3.4.0.2 PSD projection.

The additive noise on \(\widetilde{\mathbf{S}}_{\Sigma,c}\) can result in \(\widehat{\boldsymbol{\Sigma}}_c\) being indefinite (some eigenvalues negative), which would prevent sampling. We project onto the positive semidefinite cone and compute the eigendecomposition \(\widehat{\boldsymbol{\Sigma}}_c = Q\Lambda Q^\top\) (with \(Q\) orthogonal and \(\Lambda\) diagonal) and clamp all eigenvalues to a minimum threshold \(\tau > 0\): \[\label{eq:psd95projection} \boldsymbol{\Sigma}_c^{\mathrm{DP}} = Q\,\max(\Lambda,\,\tau I)\,Q^\top,\tag{7}\] where \(\tau\) is small (e.g., \(10^{-6}\)). The result \(\boldsymbol{\Sigma}_c^{\mathrm{DP}}\) is the final DP-protected covariance used for generation. Both corrections are deterministic post-processing of the noisy sums and incur no additional privacy cost.

3.5 Synthetic Data Generation and Downstream Use↩︎

The server now possesses, for each class \(c\), a DP-protected mean \(\boldsymbol{\mu}_c^{\mathrm{DP}} \in \mathbb{R}^d\) and covariance \(\boldsymbol{\Sigma}_c^{\mathrm{DP}} \in \mathbb{R}^{d \times d}\) that together parameterize a class-conditional Gaussian in the autoencoder’s latent space. For each class \(c\), \(N_c\) latent vectors are sampled from this Gaussian and decoded back to pixel space via the frozen decoder \(\mathcal{D}\): \[\label{eq:sampling} z_c^{(k)} \sim \mathcal{N}\!\big(\boldsymbol{\mu}_c^{\mathrm{DP}},\,\boldsymbol{\Sigma}_c^{\mathrm{DP}}\big), \qquad x_c^{(k)} = \mathcal{D}(z_c^{(k)}), \qquad k = 1,\dots,N_c.\tag{8}\] The synthetic dataset \(X^* = \bigcup_{c=1}^K \{(x_c^{(k)}, c)\}_{k=1}^{N_c}\) aggregates decoded images across all \(K\) classes. Since \(X^*\) is derived entirely from the DP-protected statistics, it can be generated in any quantity, producing more samples is just additional sampling and decoding, at no additional privacy cost.

In our experiments we train a neural network on \(X^*\) via standard ERM: \[\label{eq:downstream} \theta^* = \arg\min_\theta \; \frac{1}{|X^*|} \sum_{(x,y) \in X^*} \ell\!\big(f_\theta(x),\, y\big).\tag{9}\] \(X^*\) is equally suitable as pretraining data for multi-round FL, a distillation corpus, or input to any other learning pipeline. We refer do 1 for a visualization of our method.

4 Privacy Analysis↩︎

The DP analysis is straightforward compared to iterative FL methods. Statistics are released exactly once, sensitivity is determined by a public clipping threshold, and all downstream operations are post-processing. This DP framework does not protect class membership and we assume the label space is public knowledge. We show how to privatize class membership in the supplementary materials.

4.1 Clipping↩︎

Before computing statistics, each client clips its encoded latents to \(\ell_2\)-norm to \(R\): \[\label{eq:clipping} \bar{z}_{i,j} = z_{i,j} \cdot \min\!\Big(1,\;\frac{R}{\|z_{i,j}\|_2}\Big).\tag{10}\] Clipping ensures that each individual record’s contribution to the sums \(\mathbf{S}_{\mu,c}\) and \(\mathbf{S}_{\Sigma,c}\) is bounded, which is necessary for calibrating the DP noise. The threshold \(R\) is set data-independently from the autoencoder prior: under \(p(z) = \mathcal{N}(0, I_d)\), the norm \(\|z\|_2\) concentrates near \(\sqrt{d}\), so \(R = \alpha\sqrt{d}\) with a moderate \(\alpha\) (e.g., \(\alpha = 3\)) covers the vast majority of encoded latents without inspecting any private data.

4.2 Sensitivity↩︎

Under the public-label assumption above, we adopt within-class replacement adjacency. Adjacent datasets \(\mathcal{D} \sim \mathcal{D}'\) differ by replacing one class-\(c\) record with another class-\(c\) record, leaving all counts \(n_c\) unchanged. Under this definition, replacing a clipped latent \(\bar{z}\) (with \(\|\bar{z}\|_2 \le R\)) by \(\bar{z}'\) (with \(\|\bar{z}'\|_2 \le R\)) changes the sum \(\mathbf{S}_{\mu,c}\) by \(\bar{z}' - \bar{z}\) and the second-moment sum \(\mathbf{S}_{\Sigma,c}\) by \(\bar{z}'(\bar{z}')^\top - \bar{z}\bar{z}^\top\). The \(\ell_2\)-sensitivities are therefore: \[\label{eq:sensitivity} \Delta_\mu = \sup_{\mathcal{D} \sim \mathcal{D}'} \|\mathbf{S}_{\mu,c}(\mathcal{D}) - \mathbf{S}_{\mu,c}(\mathcal{D}')\|_2 \le 2R, \qquad \Delta_\Sigma = \sup_{\mathcal{D} \sim \mathcal{D}'} \|\mathbf{S}_{\Sigma,c}(\mathcal{D}) - \mathbf{S}_{\Sigma,c}(\mathcal{D}')\|_F \le 2R^2,\tag{11}\] by the triangle inequality and the norm bound on clipped latents.

4.3 Privacy Guarantee↩︎

The mechanism results in two quantities per class: the noisy mean sum \(\widetilde{\mathbf{S}}_{\mu,c}\) and the noisy second-moment sum \(\widetilde{\mathbf{S}}_{\Sigma,c}\), each protected by the Gaussian mechanism. We split the total privacy budget \((\varepsilon, \delta)\) equally between them. We use the analytic Gaussian mechanism [28] rather than the textbook closed-form calibration because it provides a valid and tighter calibration for all \(\varepsilon > 0\), including our experimental regime.

For a query \(q(\mathcal{D})\) with \(\ell_2\)-sensitivity \(\Delta\), we release \[\label{eq:gaussian95release} \widetilde{q}(\mathcal{D}) = q(\mathcal{D}) + \xi, \qquad \xi \sim \mathcal{N}(0, \sigma^2 I),\tag{12}\] where \(\sigma\) is calibrated by the analytic Gaussian mechanism. In our case, this gives \[\widetilde{\mathbf{S}}_{\mu,c} = \mathbf{S}_{\mu,c} + \eta_{\mu,c}, \qquad \widetilde{\mathbf{S}}_{\Sigma,c} = \mathbf{S}_{\Sigma,c} + \eta_{\Sigma,c},\] where \(\eta_{\mu,c} \sim \mathcal{N}(0,\sigma_\mu^2 I_d)\) and, viewing \(\mathbf{S}_{\Sigma,c} \in \mathbb{R}^{d \times d}\) as a vector in \(\mathbb{R}^{d^2}\), \(\eta_{\Sigma,c}\) is isotropic Gaussian noise in that ambient space with variance \(\sigma_\Sigma^2\) per coordinate.

All subsequent operations such as normalization (Eq. 5 ), bias correction (Eq. 6 ), PSD projection (Eq. 7 ), sampling, decoding, and model training are deterministic functions of the released noisy sums and preserve \((\varepsilon,\delta)\)-DP by the post-processing theorem [20]. The effective noise in the mean scales as \(\sigma_\mu/n_c\), so larger per-class counts directly improve utility. Within-class replacement protects the values of individual records (i.e., which specific image a client contributes to class \(c\)) but does not protect class membership. The alternative add/remove adjacency, which does hide class membership, would reduce sensitivity to \(R\) (resp.\(R^2\)) since only one term is added or removed rather than swapped, but requires additional care in handling the variable count \(n_c\). However, class membership can be protected via secure aggregation and shuffling methods; see the appendix and [29]. In addition, we discuss making the class count DP in the appendix.

5 Experimental Results↩︎

Datasets. We evaluate on four image classification benchmarks chosen to cover diverse visual domains: ImageNette (10 classes, 9.5k ImageNet subset) and CIFAR-100 (100 classes, 50k natural images) as standard benchmarks for image classification, and EuroSAT (10 classes, 27k satellite images), and BloodMNIST (8 classes, 12k microscopy images) since these constitute real-world and out-of-distribution datasets for the pretrained VAE encoder-decoder pair. This selection tests whether the pretrained autoencoder, originally trained on natural images, generalizes to other domains.

Heterogeneity simulation. We follow [30] for experimental setups. The practical setting draws per-client label distributions from a Dirichlet distribution with concentration parameter \(\alpha\) where smaller \(\alpha\) means more heterogeneity. The pathological setting assigns each client a disjoint subset of classes (2 classes per client for 10- and 8-class datasets; 10 for CIFAR-100). Together these cover the spectrum from moderate label imbalance to complete class separation.

Implementation. We follow [30] in their implementation. Default client number is 20 clients with full participation. The autoencoder is DC-AE f32c32 [21]. Privacy parameters are \(\varepsilon{=}10\), \(\delta{=}10^{-5}\) unless stated otherwise. Downstream classifier is ResNet-18 trained on synthetic data only and evaluated on real test data of the corresponding dataset. All OSFL baselines use a ResNet-18 unless otherwise specified.

5.1 One-Shot FL Performance Comparison↩︎

Tables 1 compare FedKT-CSD against five one-shot FL baselines (FedSD2C, DENSE, CoBoosting, FedD3, and FedCVAE) across four datasets and three heterogeneity settings. All baselines operate without differential privacy, while FedKT-CSD satisfies \((\varepsilon{=}10, \delta{=}10^{-5})\)-DP. Despite this stricter threat model, FedKT-CSD outperforms every baseline on every dataset and heterogeneity setting.

The reason lies in how each method handles the heterogeneity-communication tradeoff. The baselines train local models (generators, classifiers, or distilled representations) on each client’s skewed data, then aggregate these models on the server. When data is heterogeneous, local models are biased toward the client’s label subset, and single-round aggregation cannot fully correct this bias. This effect is visible in the tables. Every baseline degrades by 20–40% between \(\alpha{=}0.1\) and the pathological setting, because pathological splits produce heavily biased local models.

FedKT-CSD sidesteps this entirely. Because we aggregate per-class statistics rather than model parameters, the server reconstructs the same global class-conditional distributions regardless of how labels are distributed across clients. The three heterogeneity columns within each dataset are identical up to DP noise, a property that no model-aggregation approach can match. This heterogeneity invariance is a structural guarantee of the additive aggregation scheme.

Furthermore, the performance gap is widest on datasets where baseline local models are most fragile. On CIFAR-100, where pathological splits give each client only 10 of 100 classes, baselines collapse below 21% while FedKT-CSD maintains \(\sim\)​29%. On EuroSAT and BloodMNIST, which have fewer classes and more samples per class, baselines fare better but FedKT-CSD still leads by 7–15 points. The pattern is consistent: the harder the heterogeneity makes local model training, the larger our advantage.

5.2 Communication and Compute↩︎

Table 2 profiles the client-side cost of FedKT-CSD for four autoencoder configurations. All timings are wall-clock measurements on a single NVIDIA RTX 5090 processing 1000 images. The pipeline has three stages: encoding, DP statistics computation and sampling, and decoding.

With our default configuration (DC-AE f32c32 at \(64{\times}64\)), the entire pipeline completes in 1.33 seconds, dominated by the encoder and decoder forward passes (0.47s and 0.85s respectively). The statistics and sampling step is negligible at 3ms, since it involves only vector sums and matrix operations in \(\mathbb{R}^{128}\). The lightweight TAESD codec at \(32{\times}32\) finishes in under 0.1 seconds total, making it viable even on CPU-only clients, though at the cost of lower downstream accuracy (Table 5).

Upload cost is the total payload a client transmits to the server. A client sends one mean vector (\(d\) floats) and one symmetric covariance matrix (\(d(d{+}1)/2\) floats) per class. With \(d{=}128\) and 10 classes this amounts to 327.5 KB per client — approximately two orders of magnitude smaller than a single FedAvg model upload (a ResNet-18 is \(\sim\)​45 MB). The high-resolution DC-AE at \(128{\times}128\) incurs a substantially larger upload of 5.0 MB due to the \(d{=}512\) latent dimension, which also degrades DP utility (Table 5), making it unattractive on both axes. Peak GPU memory during encoding is 3.5 GB for DC-AE f32c32, well within the capacity of consumer hardware. For deployment on edge devices without a GPU, the TAESD configuration requires only 333 MB of GPU memory and 21 MB of client RAM.

5.3 Visualization↩︎

Figure 3 shows samples of synthetic EuroSAT images generated from the DP-protected class-conditional Gaussians. The samples are not entirely photorealistic, because each class is modeled as a single Gaussian in a 128-dimensional latent space, yet the decoder produces images that capture the dominant visual modes of each class rather than fine-grained spatial detail. Moreover, class-discriminative structure is clearly visible. Forests and pastures produce uniform green and blue tones, residential areas contain grid-like patterns with visible rooftops, highways exhibit linear features, and industrial regions display angular structures. Annual and permanent crops are distinguishable by texture regularity, while sea/lake samples show consistent dark blue hues. These visual differences align with what a downstream classifier needs to separate the classes, which explains why the synthetic data achieves \(\sim\)​80% accuracy on EuroSAT despite the coarse appearance of individual images. The key insight is that classification does not require photorealism; it requires that the synthetic distribution preserves the inter-class separation present in the real data, which the Gaussian model in latent space achieves effectively. We show more images of other dataset in the appendix.

5.4 Comparison with other DP Synthetic Data Methods↩︎

Since we generate a DP synthetic dataset, we also compare FedKT-CSD against established DP synthetic image generation methods (see  [31] for a comprehensive overview of DP methods for images) at \(\varepsilon{=}10\), in both centralized and federated settings in Table 3. For the centralized setup we run the methods using the implementations in [31]. For the federated setup we simply run each method on the local client split and share the generated data to train a model on the server.

The kernel-based methods (DP-Kernel [32], DP-MERF [33], DP-NTK [34]) operate by matching noisy statistics of random feature embeddings between real and synthetic data. The high-dimensional matching objectives require substantial noise at practical \(\varepsilon\) values. FedKT-CSD outperforms all methods in the federated settings. This is because our pipeline sidesteps iterative optimization entirely. The pretrained DC-AE encoder provides a rich, low-dimensional embedding (\(d{=}128\)) in which simple Gaussian modeling already captures class structure well.

Furthermore, DP-LoRA LDM [35] takes a different approach, fine-tuning a latent diffusion model pretrained on the full ImageNet dataset using DP-SGD. In the centralized setting this is the strongest competitor, since the diffusion model can generate high-fidelity images and the full dataset provides enough gradient signal to absorb DP-SGD noise. However, the federated variant collapses on several datasets (e.g., 20% on BloodMNIST). This is expected, as DP-SGD noise scales with the number of gradient steps and the sensitivity of each step, so when local datasets shrink in a federated split, the signal-to-noise ratio becomes prohibitive. FedKT-CSD does not suffer from this because it never performs gradient-based optimization on private data. The noise is injected once into low-dimensional additive statistics, and the total noise magnitude is independent of how data is partitioned across clients.

5.5 Downstream Tasks↩︎

A key advantage of our approach is that the synthetic dataset \(X^*\) consists of actual images, not features tied to a specific model. Methods such as FedPFT [19] generate synthetic representations in a fixed feature space, meaning any downstream model must use the same feature extractor. By contrast, \(X^*\) is model-agnostic and can train any architecture, serve as a pretraining corpus, or be combined with local data for fine-tuning.

We demonstrate this flexibility by using \(X^*\) as a pretraining resource for personalized federated learning, following the pFL-Bench protocol [30]. We use 20 clients with full participation and a ResNet-18 backbone. Baselines train via multi-round optimization (50 rounds). For FedKT-CSD, the server pretrains ResNet-18 on \(X^*\), distributes it, and each client fine-tunes only the classification layer locally. Crucially, this ResNet-18 is entirely separate from the autoencoder that generated \(X^*\), i.e., the synthetic images transfer across model families.

Table 4 shows results on CIFAR-10 and CIFAR-100 under pathological and practical heterogeneity. The value of global synthetic pretraining scales with the severity of label-space fragmentation. On CIFAR-100 pathological, each client observes a narrow class slice and multi-round methods cannot recover structure that was never represented locally. Our globally aggregated statistics reconstruct the full class-conditional distribution, yielding a nearly 9-point lead. This advantage narrows on CIFAR-10 and under practical heterogeneity for the same reason. As local class overlap increases, the information deficit shrinks and local optimization alone suffices.

6 Privacy Evaluation↩︎

6.0.0.1 Membership inference attack.

Although the formal DP guarantee already bounds any adversary’s advantage, we include a membership inference evaluation [38] for comparability with prior OSFL methods that rely on MIA as their primary privacy metric. Figure 4 (g) shows the ROC curve. The AUC is 0.501, indistinguishable from random guessing, which is the optimal result.

This result is naturally follows from the structure of our framework. Unlike standard MIA settings where the attacker probes a model that was directly trained on private data and may have memorized individual examples, our synthetic images are samples from a Gaussian distribution fit to aggregated statistics with DP. To detect whether a specific image participated, an attacker would need its individual contribution to remain detectable after aggregation across hundreds of samples, addition of calibrated DP noise, random sampling from the resulting distribution, and decoding through a public network. Each of these steps dilutes per-record influence, making membership inference structurally difficult.

6.0.0.2 Privacy-utility tradeoff.

Figure 4 (e) shows downstream accuracy on ImageNette as a function of the privacy budget \(\varepsilon\) (with \(\delta{=}10^{-5}\)). At \(\varepsilon{=}1\), heavy noise reduces accuracy to roughly 25%. Performance climbs steeply through \(\varepsilon{=}3\)–\(5\) and reaches \(\sim\)​62% at our default \(\varepsilon{=}10\), with diminishing returns beyond that (the non-DP centralized upper bound is \(\sim\)​65%). The steep gains at moderate \(\varepsilon\) reflect the low dimensionality of the latent statistics. Because noise is injected into \(d{=}128\) dimensional vectors rather than high-dimensional model parameters, even a modest privacy budget yields a favorable signal-to-noise ratio.

7 Ablation of Method and Experimental Parameters↩︎

We ablate key design choices on ImageNette with 20 clients under practical heterogeneity (\(\alpha{=}0.1\)) unless stated otherwise. All baselines are run without DP. Results are shown in Figure 4.

7.0.0.1 Number of clients.

Figure 4 (b) varies the federation size from 20 to 1000. All baselines degrade substantially as data is spread more thinly: FedSD2C drops from 57% to 42%, and weaker methods fall below 20%. Our method is invariant to federation size because the global class-conditional sums are identical regardless of how data is partitioned across clients. The small fluctuations in our curve (\(\sim\)​1%) are purely due to downstream training variance.

7.0.0.2 Heterogeneity.

Figure 4 (c) varies the Dirichlet concentration from \(\alpha{=}0.5\) (mild heterogeneity) to \(\alpha{=}0.001\) (extreme). Every baseline degrades monotonically: FedSD2C drops from \(\sim\)​64% to \(\sim\)​36%, and the other methods can drop below 20%. Our method stays at \(\sim\)​62% across all settings because per-class statistics are aggregated identically regardless of how labels are distributed across clients. This heterogeneity invariance is a direct consequence of the additive structure of the pipeline.

7.0.0.3 Client participation rate.

Figure 4 (d) varies the fraction of clients that participate in a single round from 10% to 100%. Our method degrades gracefully. Even at 10% participation (2 out of 20 clients), accuracy remains above 48%, compared to 28% for FedSD2C and below 20% for the other baseline methods. The degradation in our case is small because the statistics from participating clients still combine additively. The main effect of lower participation is fewer samples contributing to the global sums, which increases the relative magnitude of DP noise.

7.0.0.4 Number of synthetic samples.

Figure 4 (f) varies the total number of synthetic images generated from the DP-protected Gaussians. Accuracy climbs from \(\sim\)​39% at 1k samples to \(\sim\)​62% at 10k, then plateaus. The saturation point roughly matches the size of the real dataset (9.5k images), which is expected; once the synthetic dataset is large enough to cover the learned Gaussian distributions, additional samples provide diminishing returns. Baselines follow a similar saturation pattern but at their respectively lower ceilings.

7.0.0.5 Autoencoder Architecture Ablation

Table 5 ablates the autoencoder architecture. The key design question is choosing the latent dimension \(d\). Higher \(d\) gives the encoder more capacity to represent images, however the DP noise injected into each coordinate accumulates across dimensions. Specifically, when independent Gaussian noise of scale \(\sigma_\mu\) is added to a \(d\)-dimensional mean vector, the total \(L_2\) magnitude of the noise increases as \(\sigma_\mu\sqrt{d} \propto C\sqrt{d}\), where \(C\) is the per-class clipping sensitivity. We report \(C\sqrt{d}\) as a signal-to-noise proxy. Lower values mean less total noise relative to the statistics.

The table reveals a clear tradeoff. At \(d{=}512\) (the raw DC-AE representation at \(128{\times}128\) resolution), \(C\sqrt{d}{=}1934\) and accuracy collapses to 38–58% across datasets. The latent space is expressive but noise strongly dominates as it overlays the signal. At \(d{=}128\) with \(C\sqrt{d} \approx 250\)–\(400\), accuracy peaks, showing that the latent space is expressive enough and the noise is moderate. Reducing to \(d{=}64\) via PCA further lowers \(C\sqrt{d}\) to 241, but discarding half the latent dimensions sacrifices representational capacity, resulting in 3–7% lower accuracy. At \(d{=}32\) the noise is minimal (\(C\sqrt{d}{=}119\)) but the bottleneck is too narrow to capture class structure.

A second trend is that resolution and reconstruction quality matter independently of \(d\). DC-AE at \(64{\times}64\) consistently outperforms TAESD at \(32{\times}32\) despite both using \(d{=}64\)–\(128\), because higher-resolution decoding recovers more visual detail for the downstream classifier. The best overall configuration is DC-AE f32c32-in at \(64{\times}64\) with \(d{=}128\), which balances representational capacity, reconstruction quality, and noise tolerance.

8 Limitations↩︎

Our method assumes labeled classification tasks where each client knows the class identity of its samples. Extending to unsupervised or self-supervised settings, where no class labels are available to condition the statistics on, remains an open problem for our approach and for data-sharing methods in FL more broadly. Furthermore, the Gaussian model assumes reasonable class separability in the latent space. Under noisy labels or highly overlapping class distributions, the per-class statistics would degrade, though this sensitivity is shared by most class-conditional generative approaches. Moreover, single Gaussian per class could limit the quality of generated data. However, while a GMM would offer more expressive power, it would require per-component assignments that are not additively aggregable across clients without revealing additional information. Finally, classes with very few samples yield noisy statistics. With \(n_c\) small, the effective noise in the mean scales as \(\sigma_\mu / n_c\), which can dominate through overlaying the signal. This is, however, inherent to DP mechanisms that calibrate noise to worst-case sensitivity.

9 Conclusion↩︎

We have presented FedKT-CSD, a one-shot federated learning framework that generates DP synthetic data by aggregating class-conditional latent statistics from a frozen pretrained autoencoder. The method requires a single communication round, no on-device training, and uploads in the KB regime per client, while providing formal \((\varepsilon, \delta)\)-DP guarantees with clean accounting via the Gaussian mechanism. Empirically, FedKT-CSD outperforms existing one-shot FL baselines that operate without any privacy protection, is competitive with or superior to centralized DP synthetic data methods, and scales to 1000+ clients with no performance degradation. The synthetic dataset is versatile: we demonstrate its use for both centralized classifier training and downstream personalized FL, where it achieves state-of-the-art results from a single round of communication. The core insight, that pretrained autoencoders provide a shared low-dimensional space in which simple additive statistics suffice for high-quality data generation, suggests a broader design principle for privacy-preserving federated systems.

10 Scaling with more Clients↩︎

Table 6 examines the effect of increasing the number of clients to 1000 under \(\alpha{=}0.1\). All baselines degrade substantially as each client’s local dataset shrinks and model-based one-shot methods must aggregate increasingly poor local models. FedSD2C, the most resilient baseline, drops from 76% to 62% on EuroSAT. FedKT-CSD, by contrast, is invariant to federation size: because the transmitted sums are additive, the global sums, and therefore the synthetic dataset, are identical regardless of how data is partitioned across clients.

11 Comparison of Output Modality: Pixel-Space vs.Feature-Space Synthesis↩︎

A fundamental distinction between FedKT-CSD and other methods such as FedPFT [19] or FedSD2C [15] lies in the modality of the synthesized output. FedPFT generates synthetic features in the embedding space of a specific frozen foundation model (e.g.CLIP), and FedSD2C generates only small distilled datasets. By contrast, FedKT-CSD decodes its DP-protected latent statistics back to pixel space via the frozen decoder, producing a synthetic image dataset \(X^*\) that is fully model-agnostic.

11.0.0.1 Implications for downstream use.

The pixel-space output of FedKT-CSD unlocks a strictly broader set of downstream applications than feature-space synthesis:

• Architecture flexibility. Any model architecture can be trained on \(X^*\) directly, including convolutional networks, vision transformers, or task-specific models, without any dependency on the autoencoder used to generate it. We demonstrated this on a p-FL downstream task.

• Pretraining and transfer learning. \(X^*\) can serve as a pretraining corpus for any feature extractor, enabling transfer to downstream tasks beyond the original classification problem.

• Knowledge distillation. \(X^*\) can be used as a distillation corpus for any teacher-student pair, independent of the teacher’s architecture.

• Data augmentation. \(X^*\) can be combined with local client data for fine-tuning or mixed into any training pipeline as auxiliary data.

• Multi-task reuse. Because \(X^*\) consists of images rather than task-specific features, it can be reused across multiple downstream tasks without additional privacy cost — generating more samples from the DP-protected Gaussians incurs no further privacy expenditure by the post-processing theorem.

• Third-party sharing. \(X^*\) can be shared with parties who have no knowledge of the autoencoder architecture or weights, since it is simply a labeled image dataset.

12 Detailed Comparison with FedPFT↩︎

FedPFT [19] is the most closely related prior work: both methods encode images through a frozen pretrained model, compute class-conditional distributions, and transmit them in one round. FedPFT includes a theoretical treatment of DP and some privacy evaluation, but it is not a core architectural design choice. As we detail below, the way noise is applied in FedPFT is likely to degrade rapidly at meaningful privacy budgets. The comparison below reflects our interpretation of their described mechanism; we encourage readers to consult the original work.

The most consequential difference is noise scaling. In FedPFT, each client adds noise to its own GMM parameters before uploading. With \(N\) clients each holding \(n/N\) samples, each client’s noise is calibrated to its local sample size. For \(K{>}1\) components the server concatenates noisy GMMs without averaging, so each synthetic feature carries one client’s full noise. For \(K{=}1\) the server can average, reducing noise by \(\sqrt{N}\), but without secure aggregation the server still observes individual contributions. In our approach, clients upload masked additive sums via secure aggregation; the server sees only the global total, and the effective noise matches the centralized setting with all \(n\) samples. With 100 clients this yields roughly \(10\times\) better SNR than FedPFT \(K{=}1\).

Second, FedPFT assumes \(\|f\|_2 \leq 1\), which holds for CLIP but not for general extractors. Our autoencoder prior provides a known norm distribution from which we derive a data-independent clipping radius, making the guarantee unconditional on model choice. Third, FedPFT does not account for composition across classes, and their \(K{>}1\) GMM fitting involves sequential steps (K-means then per-component estimation) that require sequential composition. Our single-pass additive sums avoid this complexity: for each class we release only two aggregated statistics in the main text (or three with privatized counts), so the privacy accounting is simple. Finally, our decoder produces reusable pixel-space images rather than features tied to a specific classifier head.

13 Parallel Composition and Privacy Accounting↩︎

A key structural property of our pipeline is that, under the public-label assumption, each record contributes to exactly one class-specific mechanism. This enables parallel composition across classes, while the multiple releases within a class compose sequentially.

In our setting, the dataset decomposes across classes as \(D = D_{c=1} \cup \dots \cup D_{c=K}\), since each sample has exactly one class label. For a fixed class \(c\), the mechanism releases two noisy quantities in the main text: the mean sum \(\widetilde{\mathbf{S}}_{\mu,c}\) and the second-moment sum \(\widetilde{\mathbf{S}}_{\Sigma,c}\). These two releases access the same class-\(c\) records and therefore compose sequentially within the class. In the add/remove extension of Section 15, the privatized count \(\tilde{n}_c\) adds a third sequentially composed release.

Across classes, however, the class-specific datasets are disjoint, so the classwise mechanisms compose in parallel. Therefore, under within-class replacement adjacency and public labels, the overall privacy cost is the privacy cost of a single class-specific mechanism: \((\varepsilon,\delta)\) in the main text, and \((\varepsilon,\delta)\) under the budget split used in Section 15.

Secure aggregation does not change the DP accounting itself; it only hides individual client contributions before the global classwise sums are formed.

14 Further Visualization and Justification of Image Quality↩︎

We show generated images of our method (5) for all datasets used, using at most 10 classes from each dataset for better visualization. For EuroSAT and BloodMNIST the images are clearly recognizable with respect to the original datasets and their respective classes. However, for ImageNette and CIFAR-100 the generated images are more abstract representations. There are several reasons why these representations remain useful for downstream classification. Prior work has established that deep networks do not require human-recognizable image quality to learn effective representations. [39] showed that convolutional networks rely heavily on local texture statistics rather than global shape, achieving high accuracy even when shape information is disrupted. [40] demonstrated that patterns appearing as noise to humans can elicit near-perfect classifier confidence, confirming that networks latch onto features orthogonal to human visual criteria. On the resolution front, [41] showed that embeddings learned from images as small as \(16\times16\) pixels transfer competitively to standard benchmarks. More broadly, [42] found that mid-to-upper convolutional features remain highly reusable across domains regardless of source image appearance, and [43] visualized intermediate activations that, while lacking human interpretability, are instrumental for classification. Our synthetic images preserve exactly the kind of local texture and class-conditional structure that these studies identify as sufficient for representation learning.

15 Extension to Add/Remove Adjacency via Count Privatization↩︎

The main text adopts within-class replacement adjacency, which fixes the per-class count \(n_c\) and protects the values of individual records but not participation itself. Here we show that a straightforward extension recovers full add/remove record-level DP.

15.0.0.1 Add/remove adjacency.

Under add/remove adjacency, adjacent datasets \(\mathcal{D} \sim \mathcal{D}'\) differ by the addition or removal of a single class-\(c\) record. This changes three quantities: \(\mathbf{S}_{\mu,c}\) by at most \(R\) in \(\ell_2\)-norm, \(\mathbf{S}_{\Sigma,c}\) by at most \(R^2\) in Frobenius norm, and \(n_c\) by exactly \(1\). The sensitivities are therefore: \[\Delta_\mu^{\rm ao} = R, \qquad \Delta_\Sigma^{\rm ao} = R^2, \qquad \Delta_n = 1.\] Note that the sensitivities on \(\mathbf{S}_{\mu,c}\) and \(\mathbf{S}_{\Sigma,c}\) are halved compared to replacement adjacency in the main text, since only one term is added or removed rather than swapped.

15.0.0.2 Budget allocation.

We partition the total privacy budget \((\varepsilon, \delta)\) as \(\varepsilon = \varepsilon_{\rm stats} + \varepsilon_n\) and \(\delta = \delta_{\rm stats} + \delta_n\), allocating \((\varepsilon_{\rm stats},\delta_{\rm stats})\) to the mean and covariance sums and \((\varepsilon_n,\delta_n)\) to the count. Splitting \((\varepsilon_{\rm stats},\delta_{\rm stats})\) equally between the two statistics, we set \(\sigma_\mu^{\rm ao}\), \(\sigma_\Sigma^{\rm ao}\), and \(\sigma_n\) to be the smallest positive values satisfying \[\Phi\!\left(\frac{\Delta}{2\sigma} - \frac{\varepsilon' \sigma}{\Delta}\right) - e^{\varepsilon'} \Phi\!\left(-\frac{\Delta}{2\sigma} - \frac{\varepsilon' \sigma}{\Delta}\right) \le \delta',\] with \((\varepsilon',\delta') = (\varepsilon_{\rm stats}/2,\delta_{\rm stats}/2)\) and \(\Delta = R\) for \(\sigma_\mu^{\rm ao}\), \((\varepsilon',\delta') = (\varepsilon_{\rm stats}/2,\delta_{\rm stats}/2)\) and \(\Delta = R^2\) for \(\sigma_\Sigma^{\rm ao}\), and \((\varepsilon',\delta') = (\varepsilon_n,\delta_n)\) and \(\Delta = 1\) for \(\sigma_n\). The noisy count \(\tilde{n}_c = n_c + \eta_n\) with \(\eta_n \sim \mathcal{N}(0,\sigma_n^2)\) is then used in place of \(n_c\) throughout the post-processing steps in the main text.

We note that compared to the replacement adjacency in the main text, the noise on \(\mathbf{S}_{\mu,c}\) and \(\mathbf{S}_{\Sigma,c}\) is reduced by a factor of two due to the halved sensitivity, partially offsetting the additional budget allocated to the count. The utility impact of count privatization depends on the ratio \(\sigma_n / n_c\) and is left for future empirical investigation.

16 Secure Aggregation Details↩︎

We provide additional detail on how the secure aggregation protocol of [44] applies to our setting. We assume an honest-but-curious server: the server follows the protocol correctly but attempts to infer as much as possible from the messages it receives.

16.0.0.1 Privacy scope.

The differential privacy guarantee in the main text applies to the released noisy statistics (and any downstream synthetic data derived from them). In the protocol described here, secure aggregation hides individual client contributions, but the server recovers the exact global sums before adding DP noise. Accordingly, our threat model assumes that the server is trusted to execute the DP mechanism on the aggregated statistics after secure aggregation. An alternative is distributed noise addition, in which no raw aggregate is ever revealed; we leave this variant for future work.

16.0.0.2 Setting.

Each client \(i \in \{1,\dots,N\}\) holds a private vector \(\mathbf{m}_c^i \in \mathbb{R}^d\) (the class-\(c\) latent sum) and a private matrix \(\mathbf{M}_c^i \in \mathbb{R}^{d \times d}\) (the class-\(c\) outer-product sum). The server requires only the element-wise totals \(\mathbf{S}_{\mu,c} = \sum_i \mathbf{m}_c^i\) and \(\mathbf{S}_{\Sigma,c} = \sum_i \mathbf{M}_c^i\), and should learn nothing about any individual client’s contribution beyond what is inferable from these totals. We describe the protocol for \(\mathbf{m}_c^i\); the same procedure is applied independently to each entry of the symmetric upper triangle of \(\mathbf{M}_c^i\) and to \(n_c^i\).

16.0.0.3 Masking with pairwise perturbations.

The basic idea, following [44], is to mask each client’s input with random perturbations that cancel upon summation. Each pair of clients \((i,j)\) agrees on a shared random seed \(s_{i,j} = s_{j,i}\) via Diffie-Hellman key agreement, mediated by the server. From this seed, both clients derive a pseudorandom vector \(\mathbf{p}_{i,j} \in \mathbb{R}^d\) using a cryptographically secure pseudorandom generator. Client \(i\) computes a perturbation by summing over all other clients: \[\mathbf{r}_i = \sum_{\substack{j \neq i \\ j > i}} \mathbf{p}_{i,j} - \sum_{\substack{j \neq i \\ j < i}} \mathbf{p}_{i,j},\] so that by construction \(\sum_{i=1}^N \mathbf{r}_i = \mathbf{0}\), since each pairwise term \(\mathbf{p}_{i,j}\) appears once with a positive sign (for the client with smaller index) and once with a negative sign (for the client with larger index). Each client uploads the masked value \(\mathbf{y}_i = \mathbf{m}_c^i + \mathbf{r}_i\) to the server, which computes: \[\sum_{i=1}^N \mathbf{y}_i = \sum_{i=1}^N \mathbf{m}_c^i + \sum_{i=1}^N \mathbf{r}_i = \sum_{i=1}^N \mathbf{m}_c^i = \mathbf{S}_{\mu,c}.\] The server recovers the exact global sum. Under the assumption that the pseudorandom generator is cryptographically secure, each individual \(\mathbf{y}_i\) is computationally indistinguishable from uniform randomness to the server, revealing nothing about \(\mathbf{m}_c^i\) beyond what is inferable from the global sum \(\mathbf{S}_{\mu,c}\).

16.0.0.4 Dropout resilience.

If a client \(i\) drops out before uploading \(\mathbf{y}_i\), its perturbations would not cancel, corrupting the sum. To handle this, each client secret-shares its private key (used to derive the pairwise seeds) among all other clients using a \((t,N)\)-threshold scheme such as Shamir’s Secret Sharing, where \(t > N/2\). If client \(i\) drops out, the surviving clients contribute their shares of \(i\)’s key, allowing the server to reconstruct \(i\)’s perturbations and subtract them from the running total. This ensures correctness as long as at least \(t\) clients complete the protocol. However, if the server were to reconstruct a surviving client’s pairwise perturbations, it could potentially unmask that client’s input — this is addressed by double masking.

16.0.0.5 Double masking.

Each client \(i\) adds a second independent mask \(\mathbf{b}_i\) (also secret-shared among other clients) to its upload: \(\mathbf{y}_i = \mathbf{m}_c^i + \mathbf{b}_i + \mathbf{r}_i\). During unmasking, the server must choose for each client: either recover the pairwise perturbations \(\mathbf{r}_i\) (for genuine dropouts) or recover \(\mathbf{b}_i\) (for surviving clients). An honest client will never reveal both types of shares for the same user, ensuring that each client’s input remains masked by at least one layer at all times, even if the server falsely claims a client dropped out.

16.0.0.6 Why this fits our method.

Standard SecAgg was designed for aggregating model gradients in federated learning, where each client’s contribution is a high-dimensional vector and the server needs only the sum. Our setting is a natural fit for the same reason: the per-class statistics \((\mathbf{m}_c^i, \mathbf{M}_c^i, n_c^i)\) are precisely the kind of additive quantities that SecAgg is designed for, and the server never needs to inspect individual contributions. Exploiting the symmetry of \(\mathbf{M}_c^i\), each client’s payload consists of \(K \times (d + d(d{+}1)/2 + 1)\) floats. With \(d{=}128\) and \(K{=}10\) classes this amounts to approximately 82k floats per client, or 327.5 KB at 4 bytes per float — roughly three orders of magnitude smaller than a typical ResNet-18 gradient upload (\({\sim}45\) MB). This makes the pairwise key agreement and masking steps lightweight in both communication and computation.

References↩︎