[Paper Review] Federated Learning with Differential Privacy

Adrien Banse, Jan Kreischer+1 more|arXiv (Cornell University)|Feb 3, 2024

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper evaluates federated learning with differential privacy (DP) on MNIST, FEMNIST, and a real-world medical dataset, demonstrating that DP significantly degrades model accuracy—especially under non-i.i.d. data or small datasets—due to noise injection and gradient clipping, despite maintaining privacy guarantees.

ABSTRACT

Federated learning (FL), as a type of distributed machine learning, is capable of significantly preserving client's private data from being shared among different parties. Nevertheless, private information can still be divulged by analyzing uploaded parameter weights from clients. In this report, we showcase our empirical benchmark of the effect of the number of clients and the addition of differential privacy (DP) mechanisms on the performance of the model on different types of data. Our results show that non-i.i.d and small datasets have the highest decrease in performance in a distributed and differentially private setting.

Motivation & Objective

• To investigate the impact of data distribution (i.i.d. vs. non-i.i.d.) on model performance in federated learning with differential privacy.
• To evaluate how varying privacy budgets (ε) affect model accuracy and convergence in a DP-FL setting.
• To assess the feasibility of applying DP-FL to small, realistic medical datasets commonly found in practice.
• To identify performance trade-offs between privacy and model utility in cross-silo federated learning scenarios.

Proposed method

• Employed FedAvg algorithm for centralized federated learning with model aggregation across clients.
• Applied differential privacy via gradient perturbation using PyTorch's Opacus library with adaptive composition and gradient clipping.
• Used a fixed privacy budget per round, dividing ε across training rounds to maintain overall (ε, δ)-DP guarantees.
• Trained convolutional neural networks (CNNs) on MNIST, FEMNIST, and a small medical dataset with varying numbers of clients and ε values.
• Measured model performance via testing accuracy and loss across training rounds under different configurations.
• Used a non-i.i.d. partitioning strategy on FEMNIST based on writer identity to simulate realistic data heterogeneity.

(a) Accuracy and loss for different numbers of clients.

Experimental results

Research questions

• RQ1 How does data non-i.i.d. distribution affect model convergence and accuracy in DP-FL?
• RQ2 How does varying the privacy budget ε influence model performance in federated learning with DP?
• RQ3 Can DP-FL achieve acceptable model utility on small, realistic medical datasets?
• RQ4 What is the trade-off between privacy (DP) and model utility (accuracy) in cross-silo federated learning?

Key findings

• On MNIST with i.i.d. data, DP reduced model accuracy from over 95% to approximately 75% even with ε=100, indicating a 20% performance drop.
• For FEMNIST (non-i.i.d.), DP caused the training process to fail to converge, resulting in poor model utility despite high client counts.
• On the small medical dataset, training showed higher variance with more clients, and DP did not improve accuracy—only ε=10 showed a slight loss reduction.
• The performance degradation was most severe under non-i.i.d. data and small datasets, where DP mechanisms severely hindered learning.
• Even with large privacy budgets (ε=100), model accuracy did not recover to non-private levels, highlighting a fundamental trade-off.
• Standard ε values like 0.1 were too restrictive for small datasets, requiring larger ε to enable any training progress.

(b) Accuracy and loss for different values of $\epsilon$ , with 10 clients.