[Paper Review] Dopamine: Differentially Private Federated Learning on Medical Data

Mohammad Malekzadeh, Burak Hasırcıoglu+4 more|arXiv (Cornell University)|Jan 27, 2021

Privacy-Preserving Technologies in Data36 references42 citations

TL;DR

Dopamine combines federated learning with differentially private stochastic gradient descent and secure aggregation to train DNNs on distributed medical images, achieving a privacy guarantee close to centralized DP with improved accuracy over standard FL with parallel DP.

ABSTRACT

While rich medical datasets are hosted in hospitals distributed across the world, concerns on patients' privacy is a barrier against using such data to train deep neural networks (DNNs) for medical diagnostics. We propose Dopamine, a system to train DNNs on distributed datasets, which employs federated learning (FL) with differentially-private stochastic gradient descent (DPSGD), and, in combination with secure aggregation, can establish a better trade-off between differential privacy (DP) guarantee and DNN's accuracy than other approaches. Results on a diabetic retinopathy~(DR) task show that Dopamine provides a DP guarantee close to the centralized training counterpart, while achieving a better classification accuracy than FL with parallel DP where DPSGD is applied without coordination. Code is available at https://github.com/ipc-lab/private-ml-for-health.

Motivation & Objective

• Motivate privacy concerns in training DNNs on distributed medical datasets.
• Develop a FL-based training framework with differential privacy guarantees for record-level DP.
• Improve privacy-utility trade-off using secure aggregation and momentum-enabled DPSGD.
• Provide theoretical privacy guarantees and empirical evaluation on a diabetic retinopathy task.

Proposed method

• Proposes Dopamine, a customized DPSGD for FL with secure aggregation.
• Clips per-sample gradients to a bound C and adds Gaussian noise with scale sigma.
• Uses momentum in gradient updates to improve accuracy.
• Tracks privacy loss per round using the Moments Accountant (MA) to enforce (epsilon, delta)-DP.
• Demonstrates record-level DP against the server and DP against hospitals under secure aggregation.
• Evaluates on a diabetic retinopathy dataset with SqueezeNet as the base model.

Experimental results

Research questions

• RQ1 Can Dopamine provide a record-level DP guarantee during federated training on distributed medical data?
• RQ2 Does secure aggregation allow achieving a privacy-utility trade-off closer to centralized DP than standard FL with DPSGD?
• RQ3 How does incorporating momentum and per-round MA-based privacy accounting affect model accuracy and privacy loss?

Key findings

• Dopamine achieves a DP bound close to centralized training (CDP) and often better at early epochs.
• Dopamine yields higher classification accuracy than FL with parallel DP (FPDP) under the same privacy level.
• Secure aggregation reduces the noise burden on each hospital, improving utility compared to FPDP.
• Compared to non-private FL, Dopamine maintains competitive accuracy while offering formal DP guarantees.
• The approach guarantees (epsilon, delta)-DP against the server and (epsilon*sqrt(K/(K-1)), delta)-DP against hospitals in the presented setting.