[Paper Review] Defending Medical Image Diagnostics against Privacy Attacks using Generative Methods

William Paul, Yinzhi Cao+2 more|arXiv (Cornell University)|Mar 4, 2021

Privacy-Preserving Technologies in Data35 references4 citations

TL;DR

This paper proposes a novel privacy defense for medical image diagnostics using generative adversarial networks (GANs) to create synthetic retinal images that protect patient identity while preserving diagnostic utility. By replacing real training data with GAN-synthesized data, the method significantly reduces membership inference attack success—achieving state-of-the-art privacy protection with minimal impact on model accuracy, validated on diabetic retinopathy classification.

ABSTRACT

Machine learning (ML) models used in medical imaging diagnostics can be vulnerable to a variety of privacy attacks, including membership inference attacks, that lead to violations of regulations governing the use of medical data and threaten to compromise their effective deployment in the clinic. In contrast to most recent work in privacy-aware ML that has been focused on model alteration and post-processing steps, we propose here a novel and complementary scheme that enhances the security of medical data by controlling the data sharing process. We develop and evaluate a privacy defense protocol based on using a generative adversarial network (GAN) that allows a medical data sourcer (e.g. a hospital) to provide an external agent (a modeler) a proxy dataset synthesized from the original images, so that the resulting diagnostic systems made available to model consumers is rendered resilient to privacy attackers. We validate the proposed method on retinal diagnostics AI used for diabetic retinopathy that bears the risk of possibly leaking private information. To incorporate concerns of both privacy advocates and modelers, we introduce a metric to evaluate privacy and utility performance in combination, and demonstrate, using these novel and classical metrics, that our approach, by itself or in conjunction with other defenses, provides state of the art (SOTA) performance for defending against privacy attacks.

Motivation & Objective

• To address the risk of privacy leaks in medical AI, particularly through membership inference attacks that expose whether a patient’s data was used in training.
• To develop a data-centric privacy defense that operates at the data source level, avoiding reliance on model-level modifications or trust in modelers.
• To introduce a new metric, P1-score, that quantitatively balances privacy and diagnostic utility for informed trade-off decisions between data sourcers and modelers.
• To evaluate the effectiveness of synthetic data generated via GANs in defending against privacy attacks while maintaining high diagnostic performance on retinal imaging tasks.
• To demonstrate that the proposed method outperforms or complements existing defenses in reducing attack accuracy without degrading model utility.

Proposed method

• The method uses a conditional GAN to generate synthetic retinal images from real diabetic retinopathy images, preserving diagnostic features while obscuring individual identities.
• The data sourcer shares only the synthetic dataset with external modelers, thereby preventing direct access to real patient data and mitigating privacy leakage.
• A novel identity equivalence relation ∼I is defined to determine whether two images (real and synthetic) are considered to represent the same individual, based on perceptual similarity or biometric features like retinal vasculature.
• The approach integrates with existing defenses such as MMD and Mixup, enhancing their privacy performance without altering their training procedures.
• The P1-score metric is introduced as a composite measure combining model accuracy (utility) and attack accuracy (privacy risk), with equal weighting to assess trade-offs.
• The method is evaluated on a retinal image dataset for diabetic retinopathy, comparing attack success rates and model performance across varying ratios of real-to-synthetic data.

Experimental results

Research questions

• RQ1 Can GAN-generated synthetic data effectively defend against membership inference attacks in retinal diagnostics while maintaining high diagnostic model utility?
• RQ2 How does the proposed P1-score metric compare to traditional metrics in capturing the privacy-utility trade-off in medical image AI?
• RQ3 To what extent does combining synthetic data with existing defenses (e.g., MMD, Mixup) improve privacy protection compared to standalone methods?
• RQ4 How does the inclusion of real data in the training mix affect attack success, and can synthetic data mitigate this risk?
• RQ5 Can the identity equivalence relation ∼I be effectively defined using perceptual or biometric features to ensure privacy without compromising data utility?

Key findings

• The proposed GAN-based synthetic data defense reduces membership inference attack accuracy to 55.86% (75% real data) and 56.55% (50% real data), significantly below the baseline of 60.11% when using only real data.
• With 25% real data, the method achieves a 59.67% attack accuracy under MMD+Mixup defense, compared to 57.27% for the baseline, indicating strong privacy preservation even with partial real data.
• The P1-score metric successfully captures the privacy-utility trade-off, enabling data sourcers and modelers to negotiate acceptable levels of data sharing based on combined performance.
• The method achieves state-of-the-art (SOTA) performance in defending against privacy attacks, outperforming or complementing existing defenses like MemGuard and MMD+Mixup.
• Even with 100% real data, the baseline attack accuracy reaches 60.11%, but with synthetic data, attack accuracy drops to 55.86% (75% real), demonstrating that synthetic data alone can substantially reduce privacy risk.
• The approach maintains high diagnostic utility, with model accuracy on real test data remaining above 57% across all settings, indicating minimal degradation in performance despite strong privacy protection.