[Paper Review] DPD-fVAE: Synthetic Data Generation Using Federated Variational Autoencoders With Differentially-Private Decoder

Bjarne Pfitzner, Bert Arnrich|arXiv (Cornell University)|Nov 21, 2022

Privacy-Preserving Technologies in Data4 citations

TL;DR

DPD-fVAE proposes a federated variational autoencoder that applies differential privacy only to the decoder, reducing noise requirements and improving synthetic data quality. It achieves competitive FID scores and classifier accuracy on MNIST, Fashion-MNIST, and CelebA while preserving client data privacy via federated learning and differential privacy.

ABSTRACT

Federated learning (FL) is getting increased attention for processing sensitive, distributed datasets common to domains such as healthcare. Instead of directly training classification models on these datasets, recent works have considered training data generators capable of synthesising a new dataset which is not protected by any privacy restrictions. Thus, the synthetic data can be made available to anyone, which enables further evaluation of machine learning architectures and research questions off-site. As an additional layer of privacy-preservation, differential privacy can be introduced into the training process. We propose DPD-fVAE, a federated Variational Autoencoder with Differentially-Private Decoder, to synthesise a new, labelled dataset for subsequent machine learning tasks. By synchronising only the decoder component with FL, we can reduce the privacy cost per epoch and thus enable better data generators. In our evaluation on MNIST, Fashion-MNIST and CelebA, we show the benefits of DPD-fVAE and report competitive performance to related work in terms of Fréchet Inception Distance and accuracy of classifiers trained on the synthesised dataset.

Motivation & Objective

• To address the challenge of generating high-quality synthetic data from distributed, privacy-sensitive datasets, such as those in healthcare.
• To reduce the privacy cost of differential privacy in federated learning by applying it selectively to the decoder component of a VAE.
• To enable the training of data generators in federated settings that are both privacy-preserving and effective for downstream machine learning tasks.
• To evaluate the trade-off between privacy budget, model performance, and data quality in synthetic data generation under central and local differential privacy.

Proposed method

• The method uses a federated VAE architecture where the encoder remains local and private, while only the decoder is shared and updated via federated averaging.
• Differential privacy is applied exclusively to the decoder's gradients using L2-norm clipping and noise injection, minimizing the privacy cost per client.
• The model employs conditional VAEs that incorporate class labels to generate class-specific synthetic images.
• Two privacy regimes are evaluated: central differential privacy (CDP) and local differential privacy (LDP), with hyperparameters (ε=10, δ=10−5) used for comparison.
• The training process uses standard VAE objectives, including the reparameterization trick and KL divergence regularization in the latent space.
• Model performance is evaluated using Fréchet Inception Distance (FID) and downstream classifier accuracy on generated data.

Experimental results

Research questions

• RQ1 Can applying differential privacy only to the decoder in a federated VAE reduce the noise burden while maintaining high-quality synthetic data generation?
• RQ2 How does the performance of DPD-fVAE compare to non-private and centrally private baselines in terms of FID and classifier accuracy?
• RQ3 What is the impact of local differential privacy on model convergence and synthetic data quality in a federated setting?
• RQ4 Can DPD-fVAE generate high-quality, class-balanced synthetic data that supports downstream model evaluation without exposing raw client data?

Key findings

• DPD-fVAE achieved a Fréchet Inception Distance (FID) of 56.9 on MNIST and 84.4 on Fashion-MNIST under (10,10−5)-DP, outperforming several prior methods in the same setting.
• On MNIST, the CNN classifier trained on DPD-fVAE-generated data achieved 91.3% accuracy, comparable to state-of-the-art non-federated methods.
• The model successfully generated high-quality images even under strict local differential privacy (LDP), whereas standard federated VAEs failed to converge under the same conditions.
• CelebA synthetic data achieved an FID of 261.7 and a CNN accuracy of 69.0%, demonstrating feasibility despite low client data availability.
• CDP-based DPD-fVAE achieved a FID of 76.4 on MNIST and 88.1% classifier accuracy, showing strong performance under central privacy guarantees.
• The approach significantly reduces the noise required for differential privacy by isolating it to the decoder, enabling better model performance with the same privacy budget.