[Paper Review] A Fully Private Pipeline for Deep Learning on Electronic Health Records

Edward Chou, Thao Nguyen+3 more|arXiv (Cornell University)|Nov 25, 2018

Privacy-Preserving Technologies in Data26 references4 citations

TL;DR

This paper presents a fully private deep learning pipeline for predicting 30-day hospital readmissions from electronic health records (EHRs), combining differential privacy during training and homomorphic encryption during inference. It achieves high performance (AUC = 0.66, recall = 0.60) under strong privacy guarantees ($\epsilon=4$, $\delta=10^{-5}$) while addressing privacy-utility trade-offs through data standardization and optimized activation functions.

ABSTRACT

We introduce an end-to-end private deep learning framework, applied to the task of predicting 30-day readmission from electronic health records. By using differential privacy during training and homomorphic encryption during inference, we demonstrate that our proposed pipeline could maintain high performance while providing robust privacy guarantees against information leak from data transmission or attacks against the model. We also explore several techniques to address the privacy-utility trade-off in deploying neural networks with privacy mechanisms, improving the accuracy of differentially-private training and the computation cost of encrypted operations using ideas from both machine learning and cryptography.

Motivation & Objective

• To address the critical challenge of patient data privacy in healthcare machine learning, where sensitive EHRs are vulnerable to de-anonymization and model inversion attacks.
• To develop an end-to-end privacy-preserving deep learning framework that secures both model training and inference stages.
• To minimize the performance degradation caused by privacy mechanisms like differential privacy and homomorphic encryption in clinical deep learning applications.
• To optimize the privacy-utility trade-off through data standardization and efficient activation function design for homomorphic encryption.
• To demonstrate the feasibility and robustness of a fully private pipeline on a real-world clinical prediction task: 30-day readmission from EHRs.

Proposed method

• Employed differentially private stochastic gradient descent (DP-SGD) with gradient clipping and Gaussian noise injection to ensure $(\epsilon, \delta)$-differential privacy during model training.
• Applied input feature standardization to reduce gradient L2 norms, thereby lowering the clipping threshold and noise requirements, improving training stability and privacy efficiency.
• Used the FV-RNS homomorphic encryption scheme (via Microsoft SEAL) to enable inference on encrypted EHR data without decryption.
• Designed and evaluated parameterized activation functions, particularly approximated Swish with quantized coefficients, to reduce multiplicative operations and computational cost in homomorphic inference.
• Leveraged low-degree polynomial approximations of non-linear activations (e.g., ReLU, Sigmoid) to maintain compatibility with leveled homomorphic encryption.
• Tracked privacy budgets using a privacy accountant to prevent over-usage of the privacy budget and ensure formal privacy guarantees.

Experimental results

Research questions

• RQ1 Can a fully private deep learning pipeline be constructed that ensures strong privacy guarantees during both training and inference on EHR data?
• RQ2 How does input standardization affect the privacy cost and training stability in differentially private deep learning for clinical tasks?
• RQ3 What is the impact of different activation function approximations on the accuracy and computational efficiency of homomorphic inference in neural networks?
• RQ4 What level of differential privacy ($\epsilon$) yields the best balance between utility and privacy for 30-day readmission prediction on EHRs?
• RQ5 Can homomorphic encryption be efficiently applied to real-world clinical deep learning models without prohibitive computational overhead?

Key findings

• Standardization of EHR features significantly reduced gradient L2 norms, lowering the required clipping bound and noise, which improved training stability and reduced privacy cost.
• The model achieved an AUC of 0.66 and recall of 0.60 under $\epsilon=4$, $\delta=10^{-5}$, demonstrating strong performance despite differential privacy.
• The best privacy-utility trade-off was achieved at $\epsilon=4$, where higher noise levels ($\epsilon=1$) led to earlier training termination and lower performance.
• Approximated Swish with coefficient quantization achieved the highest AUC (0.678) and recall (0.645), outperforming ReLU-Sigmoid and square activations, while maintaining low computational overhead.
• Quantized Swish activation reduced runtime by minimizing redundant multiplications, making homomorphic inference more efficient with negligible performance loss.
• The fully private pipeline achieved better AUC (0.63) than prior works on the same dataset (0.61 and 0.23), even under high noise, demonstrating the effectiveness of the proposed privacy-preserving approach.