[Paper Review] Unlocking Accuracy and Fairness in Differentially Private Image Classification

Leonard Berrada, Soham De+7 more|arXiv (Cornell University)|Aug 21, 2023

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper demonstrates that fine-tuning large pre-trained vision models with differentially private stochastic gradient descent (DP-SGD) can achieve accuracy within a few percent of non-private state-of-the-art models across four datasets, including medical imaging benchmarks, while maintaining fairness by showing no significant increase in performance disparities across demographic subgroups compared to non-private models.

ABSTRACT

Privacy-preserving machine learning aims to train models on private data without leaking sensitive information. Differential privacy (DP) is considered the gold standard framework for privacy-preserving training, as it provides formal privacy guarantees. However, compared to their non-private counterparts, models trained with DP often have significantly reduced accuracy. Private classifiers are also believed to exhibit larger performance disparities across subpopulations, raising fairness concerns. The poor performance of classifiers trained with DP has prevented the widespread adoption of privacy preserving machine learning in industry. Here we show that pre-trained foundation models fine-tuned with DP can achieve similar accuracy to non-private classifiers, even in the presence of significant distribution shifts between pre-training data and downstream tasks. We achieve private accuracies within a few percent of the non-private state of the art across four datasets, including two medical imaging benchmarks. Furthermore, our private medical classifiers do not exhibit larger performance disparities across demographic groups than non-private models. This milestone to make DP training a practical and reliable technology has the potential to widely enable machine learning practitioners to train safely on sensitive datasets while protecting individuals' privacy.

Motivation & Objective

• Address the major barrier in deploying privacy-preserving machine learning: the significant accuracy drop in differentially private (DP) models compared to non-private counterparts.
• Investigate whether DP training exacerbates performance disparities across subpopulations, particularly in sensitive applications like healthcare.
• Demonstrate that fine-tuning foundation models with DP-SGD can achieve high accuracy and fairness simultaneously, even under distribution shifts.
• Provide empirical evidence that DP training does not inherently worsen fairness, challenging the assumption of a trade-off between privacy and fairness.

Proposed method

• Fine-tune large-scale pre-trained vision models (e.g., ViT, ResNet) on downstream image classification tasks using DP-SGD with carefully tuned noise scale and clipping.
• Apply a two-stage training protocol: first pre-train on large general datasets (e.g., ImageNet-1K), then fine-tune on downstream datasets (e.g., CIFAR-10, CheXpert, MIMIC-CXR) under DP constraints.
• Use empirical risk minimization with gradient clipping and added Gaussian noise to enforce differential privacy, with privacy accounting via the moments accountant.
• Evaluate model performance across demographic subgroups (e.g., sex, age range) using AUC disparity metrics to assess fairness.
• Systematically vary the privacy budget ε (e.g., ε = 1, 4, 8) to study the impact on accuracy and class-wise performance ordering.
• Conduct ablation studies on data imbalance, including under-sampling specific classes (e.g., cars in CIFAR-10), to isolate the effects of imbalance and DP on fairness.

Experimental results

Research questions

• RQ1 Can differentially private fine-tuning of foundation models achieve accuracy close to non-private state-of-the-art models on image classification tasks, including medical imaging?
• RQ2 Does differential privacy training lead to increased performance disparities across demographic subgroups compared to non-private models?
• RQ3 How does the relative difficulty of classes change under different privacy budgets ε, and does this affect fairness metrics?
• RQ4 To what extent does data imbalance interact with DP training to influence fairness outcomes in private models?
• RQ5 Can the trade-off between privacy and fairness be mitigated through architectural and training choices in vision models?

Key findings

• Private models fine-tuned with DP-SGD achieve test accuracies within 2–4% of non-private state-of-the-art models across four datasets, including medical imaging benchmarks (CheXpert and MIMIC-CXR).
• The most accurate private models on medical imaging tasks show AUC disparities across demographic groups (e.g., sex, age) that are comparable to those of non-private models, indicating no significant fairness degradation.
• At ε = 8, private models achieve overall accuracy within a few percent of non-private models, even under significant distribution shifts between pre-training and downstream data.
• Class-wise accuracy rankings under DP-SGD are not monotonic with ε: for example, deer is the worst-performing class at ε ≤ 4, but dog becomes worse at ε ≥ 4, indicating a shift in relative example difficulty with privacy level.
• In highly imbalanced settings (e.g., 10% under-representation of the car class in CIFAR-10), private models show worse performance on underrepresented classes, but this effect is not inherent to DP-SGD and depends on the degree of imbalance.
• The disparity between private and non-private models in underrepresented classes is not universally worse; at moderate imbalance (25–50%), private models achieve similar per-class accuracy to non-private models at the same overall accuracy.