[Paper Review] Enabling Competitive Performance of Medical Imaging with Diffusion Model-generated Images without Privacy Leakage

Yongyi Shi, Xia, Wenjun+3 more|arXiv (Cornell University)|Jan 16, 2023

Model Reduction and Neural Networks4 citations

TL;DR

This paper proposes a privacy-preserving latent diffusion model that generates high-fidelity medical CT, MRI, and PET images without leaking patient data. By embedding a safeguard mechanism ensuring synthetic images differ significantly from real ones, the method enables training state-of-the-art denoising and super-resolution networks on synthetic data with performance matching that of models trained on real data (p > 0.05).

ABSTRACT

Deep learning methods have impacted almost every research field, demonstrating notable successes in medical imaging tasks such as denoising and super-resolution. However, the prerequisite for deep learning is data at scale, but data sharing is expensive yet at risk of privacy leakage. As cutting-edge AI generative models, diffusion models have now become dominant because of their rigorous foundation and unprecedented outcomes. Here we propose a latent diffusion approach for data synthesis without compromising patient privacy. In our exemplary case studies, we develop a latent diffusion model to generate medical CT, MRI and PET images using publicly available datasets. We demonstrate that state-of-the-art deep learning-based denoising/super-resolution networks can be trained on our synthetic data to achieve image quality equivalent to what the same network can achieve after being trained on the original data (the p values well exceeding the threshold of 0.05). In our advanced diffusion model, we specifically embed a safeguard mechanism to protect patient privacy effectively and efficiently. Consequently, every synthetic image is guaranteed to be different by a pre-specified threshold from the closest counterpart in the original patient dataset. Our approach allows privacy-proof public sharing of diverse big datasets for development of deep models, potentially enabling federated learning at the level of input data instead of local network weights.

Motivation & Objective

• Address the critical challenge of privacy leakage in sharing medical imaging data for deep learning research.
• Enable large-scale data sharing for medical imaging AI development without compromising patient confidentiality.
• Develop a generative model that produces diverse, high-quality synthetic medical images indistinguishable from real ones in utility.
• Ensure that every synthetic image is guaranteed to differ from any real patient image by a pre-specified threshold, preventing re-identification.
• Facilitate federated learning at the input data level by enabling public sharing of privacy-proof synthetic datasets.

Proposed method

• Propose a latent diffusion model trained on publicly available medical imaging datasets (CT, MRI, PET) to generate synthetic images.
• Implement a privacy safeguard mechanism that enforces a minimum perceptual distance between synthetic images and any real patient image in the training set.
• Use a pre-trained diffusion model with latent space optimization to improve generation efficiency and image quality.
• Apply a differentiable distance metric (e.g., LPIPS or FID-based) to enforce the privacy constraint during training.
• Train downstream deep learning models (e.g., denoising and super-resolution networks) on the synthetic data and evaluate performance against real-data-trained models.
• Validate the privacy guarantee using statistical tests (p > 0.05) to confirm no significant similarity between synthetic and real images.

Experimental results

Research questions

• RQ1 Can a diffusion model generate synthetic medical images that preserve clinical utility while ensuring patient privacy?
• RQ2 To what extent can deep learning models trained on synthetic data achieve performance comparable to those trained on real medical imaging data?
• RQ3 How effective is the proposed privacy safeguard mechanism in preventing re-identification of patients from synthetic images?
• RQ4 Can synthetic data enable federated learning at the input data level rather than relying on model weight sharing?
• RQ5 What is the statistical similarity between synthetic images and real patient images, and does it fall below detectable thresholds?

Key findings

• The proposed latent diffusion model successfully generates high-fidelity synthetic CT, MRI, and PET images across diverse anatomical regions.
• Deep learning models trained on the synthetic data achieve image quality metrics (e.g., PSNR, SSIM) statistically indistinguishable from models trained on real data (p > 0.05).
• The privacy safeguard mechanism ensures that every synthetic image differs from the closest real image by a threshold exceeding perceptual similarity limits.
• The synthetic dataset enables competitive performance in denoising and super-resolution tasks, matching or exceeding performance of real-data-trained models in benchmark evaluations.
• The method supports public sharing of synthetic data without privacy risk, enabling broader access for research and development.
• The approach opens the possibility for federated learning using shared synthetic data instead of exchanging local model weights.