[Paper Review] Privacy-preserving Machine Learning for Medical Image Classification

Shreyansh Singh, K.K. Shukla|arXiv (Cornell University)|Aug 29, 2021

Privacy-Preserving Technologies in Data27 references4 citations

TL;DR

This paper proposes a privacy-preserving machine learning framework for medical image classification using secure multi-party computation (SMPC) and differential privacy (DP), enabling encrypted inference and training on chest X-rays for pneumonia detection. The VGG16 model achieved 96.41% accuracy with preprocessing and 85.2% with average pooling, while DP reduced accuracy by ~10–13%, demonstrating a trade-off between privacy and performance.

ABSTRACT

With the rising use of Machine Learning (ML) and Deep Learning (DL) in various industries, the medical industry is also not far behind. A very simple yet extremely important use case of ML in this industry is for image classification. This is important for doctors to help them detect certain diseases timely, thereby acting as an aid to reduce chances of human judgement error. However, when using automated systems like these, there is a privacy concern as well. Attackers should not be able to get access to the medical records and images of the patients. It is also required that the model be secure, and that the data that is sent to the model and the predictions that are received both should not be revealed to the model in clear text. In this study, we aim to solve these problems in the context of a medical image classification problem of detection of pneumonia by examining chest x-ray images.

Motivation & Objective

• To address privacy risks in medical image classification where patient data and model predictions must remain confidential.
• To implement secure multi-party computation (SMPC) to enable encrypted model inference without exposing raw data or predictions.
• To integrate differential privacy (DP) into model training to limit membership inference attacks and protect individual patient data.
• To evaluate the trade-off between model accuracy and privacy under different encryption and regularization techniques.
• To develop a scalable, privacy-preserving pipeline for medical image classification applicable beyond pneumonia detection.

Proposed method

• Employed TensorFlow Encrypted (TFE) to implement SMPC-based inference, ensuring model and data remain encrypted throughout the prediction process.
• Used two data preprocessing methods: one with normalization and data augmentation, and another with min-max scaling and additional augmentation to improve model generalization.
• Trained deep neural networks (DNNs) and VGG16 models using SMPC-encrypted data, with model checkpoints and early stopping to optimize validation accuracy.
• Applied differential privacy during training by injecting noise into gradients, with privacy budget (ε) set to control information leakage.
• Evaluated model performance using accuracy, cross-entropy loss, and training dynamics across multiple epochs under both encrypted and DP-trained settings.
• Used Keras-based models with compatible layers, avoiding unsupported layers like Batch Normalization and Dropout in TFE.

Experimental results

Research questions

• RQ1 Can secure multi-party computation enable privacy-preserving inference on medical images without exposing raw data or model outputs?
• RQ2 How does differential privacy affect the accuracy and generalization of deep learning models in medical image classification?
• RQ3 What is the impact of different data preprocessing strategies on model performance in a privacy-preserving setting?
• RQ4 How do model architecture and pooling type (average vs. max) influence accuracy and robustness under encryption and DP?
• RQ5 What are the performance bottlenecks in deploying encrypted deep learning models for real-time medical diagnostics?

Key findings

• The VGG16 model trained with the second preprocessing method achieved the highest accuracy of 96.41%, outperforming other models.
• The DNN-Max model achieved 87.33% accuracy with a cross-entropy loss of 0.29, outperforming DNN-Av (85.2% accuracy, 0.33 loss).
• Training with differential privacy reduced model accuracy by approximately 10–13%, with the largest drop observed in DNN models (13%) and smaller drops in VGG16 models (10–11%).
• The VGG16-2 model trained with DP achieved 84.89% accuracy, confirming the expected trade-off between privacy and performance.
• SMPC-based inference with TF Encrypted introduced significant latency, especially for deeper models like VGG16, making real-time deployment challenging.
• TF Encrypted faced limitations with advanced Keras layers such as Batch Normalization and Dropout, requiring model simplification or custom implementation.