[Paper Review] Federated and Differentially Private Learning for Electronic Health Records

Stephen Pfohl, Andrew M. Dai+1 more|arXiv (Cornell University)|Nov 13, 2019

Privacy-Preserving Technologies in Data10 references44 citations

TL;DR

The paper compares centralized, federated, and differentially private training for clinical prediction tasks using the eICU-CRD dataset, showing DP-SGD is effective centralized but challenging in federated settings for privacy-preserving learning.

ABSTRACT

The use of collaborative and decentralized machine learning techniques such as federated learning have the potential to enable the development and deployment of clinical risk predictions models in low-resource settings without requiring sensitive data be shared or stored in a central repository. This process necessitates communication of model weights or updates between collaborating entities, but it is unclear to what extent patient privacy is compromised as a result. To gain insight into this question, we study the efficacy of centralized versus federated learning in both private and non-private settings. The clinical prediction tasks we consider are the prediction of prolonged length of stay and in-hospital mortality across thirty one hospitals in the eICU Collaborative Research Database. We find that while it is straightforward to apply differentially private stochastic gradient descent to achieve strong privacy bounds when training in a centralized setting, it is considerably more difficult to do so in the federated setting.

Motivation & Objective

• Motivate collaboration across hospitals for predictive modeling without sharing raw patient data while addressing privacy guarantees.
• Evaluate centralized vs. federated learning in non-private and DP settings for clinical tasks.
• Quantify privacy-utility trade-offs when applying differential privacy in centralized and federated setups.

Proposed method

• Utilize federated averaging and DP-SGD to train logistic regression and one-hidden-layer networks on 31 hospitals from eICU-CRD.
• Construct features from lab orders/results, medications, diagnoses, and patient demographics within 24 hours of admission.
• Compare local, centralized, and federated training, with and without differential privacy, across two tasks: in-hospital mortality and prolonged length of stay.
• Assess privacy via DP parameters (epsilon, delta) and model performance via AUC-ROC with DeLong confidence intervals.

Experimental results

Research questions

• RQ1 Can federated learning provide similar predictive performance to centralized learning on ICU outcome prediction tasks?
• RQ2 How does the addition of differential privacy affect model utility in centralized vs. federated settings?
• RQ3 Is DP-SGD practical in federated learning for clinical data in terms of privacy guarantees and accuracy?

Key findings

• Federated learning often yields AUC-ROC close to centralized learning and frequently improves over local hospital models, though gains are not always statistically significant.
• Differentially private centralized training can achieve strong privacy (epsilon around 1) with modest AUC-ROC degradation (e.g., stay: 0.763 vs 0.73; mortality: 0.876 vs 0.832).
• Federated learning with DP-SGD often performs poorly in both AUC-ROC and privacy cost (epsilon) for mortality prediction compared to non-DP settings.
• DP-SGD hyperparameters optimized on local data do not readily transfer to federated settings, highlighting challenges in privacy-utility trade-offs for distributed clinical data.
• The study emphasizes that ignoring the privacy cost of model selection can yield optimistic privacy estimates.