[Paper Review] Privacy-Preserving and Trustworthy Deep Learning for Medical Imaging

Kiarash Sedghighadikolaei, Attila A. Yavuz|arXiv (Cornell University)|Jun 29, 2024

Artificial Intelligence in Healthcare and EducationMedicine3 citations

TL;DR

This paper proposes a systematic framework for integrating Privacy-Enhancing Technologies (PETs) into Deep Radiomics to ensure privacy and trustworthiness in medical imaging. It classifies PETs, introduces hybrid PET constructions, and provides a taxonomy for their integration across the Deep Radiomics pipeline, with technical insights on challenges and future research directions.

ABSTRACT

The shift towards efficient and automated data analysis through Machine Learning (ML) has notably impacted healthcare systems, particularly Radiomics. Radiomics leverages ML to analyze medical images accurately and efficiently for precision medicine. Current methods rely on Deep Learning (DL) to improve performance and accuracy (Deep Radiomics). Given the sensitivity of medical images, ensuring privacy throughout the Deep Radiomics pipeline-from data generation and collection to model training and inference-is essential, especially when outsourced. Thus, Privacy-Enhancing Technologies (PETs) are crucial tools for Deep Radiomics. Previous studies and systematization efforts have either broadly overviewed PETs and their applications or mainly focused on subsets of PETs for ML algorithms. In Deep Radiomics, where efficiency, accuracy, and privacy are crucial, many PETs, while theoretically applicable, may not be practical without specialized optimizations or hybrid designs. Additionally, not all DL models are suitable for Radiomics. Consequently, there is a need for specialized studies that investigate and systematize the effective and practical integration of PETs into the Deep Radiomics pipeline. This work addresses this research gap by (1) classifying existing PETs, presenting practical hybrid PETS constructions, and a taxonomy illustrating their potential integration with the Deep Radiomics pipeline, with comparative analyses detailing assumptions, architectural suitability, and security, (2) Offering technical insights, describing potential challenges and means of combining PETs into the Deep Radiomics pipeline, including integration strategies, subtilities, and potential challenges, (3) Proposing potential research directions, identifying challenges, and suggesting solutions to enhance the PETs in Deep Radiomics.

Motivation & Objective

• Address the lack of specialized studies on practical integration of Privacy-Enhancing Technologies (PETs) in Deep Radiomics.
• Systematize existing PETs and propose hybrid constructions tailored for the Deep Radiomics pipeline.
• Provide a comparative analysis of PETs based on assumptions, architectural suitability, and security properties.
• Offer technical insights into challenges and integration strategies for combining PETs in medical imaging workflows.
• Identify open challenges and propose future research directions to enhance PET practicality in Deep Radiomics.

Proposed method

• Classify and categorize existing PETs relevant to Deep Radiomics, including homomorphic encryption, secure multi-party computation, and differential privacy.
• Propose practical hybrid PET constructions that combine multiple privacy techniques for improved efficiency and security.
• Develop a taxonomy mapping PETs to stages of the Deep Radiomics pipeline: data collection, model training, inference, and deployment.
• Analyze architectural suitability, threat model assumptions, and security guarantees of each PET type across different pipeline stages.
• Integrate PETs with Convolutional Neural Networks (CNNs) and Stochastic Gradient Descent (SGD) for efficient and privacy-preserving training.
• Evaluate the feasibility of applying Differential Privacy (DP) at gradient, input, and label levels, particularly in Federated Learning and centralized settings.

Figure 1 : Taxonomy, Pros, and Cons of Essential PETs with their Prominent Hybrid Compositions for Deep Radiomics

Experimental results

Research questions

• RQ1 Which PETs are most suitable for different stages of the Deep Radiomics pipeline, and how can they be effectively combined?
• RQ2 How can hybrid PET architectures improve the balance between privacy, accuracy, and computational efficiency in Deep Radiomics?
• RQ3 What are the key challenges in applying PETs—especially Differential Privacy—to non-convex deep learning models like CNNs in medical imaging?
• RQ4 How can PETs be integrated into Federated Learning and centralized training to protect against membership and model extraction attacks?
• RQ5 What are the critical open challenges and future research directions for deploying trustworthy and privacy-preserving deep learning in radiomics?

Key findings

• Differential Privacy (DP) applied to gradients in SGD significantly reduces the success rate of Membership Inference Attacks (MIAs) and Model Extraction Attacks (MEAs).
• Centralized DP (CDP) and Local DP (LDP) offer different trade-offs in privacy and accuracy, with CDP being more effective when the central aggregator is trusted.
• Hybrid PET constructions, such as combining MPC with DP, can enhance security while mitigating noise accumulation in deep learning models.
• Applying DP to input data or model outputs is less effective than gradient-level DP in non-convex models due to lack of closed-form sensitivity analysis.
• Objective perturbation shows promise for PPML but remains challenging in Deep Radiomics due to the non-convex nature of CNN loss functions.
• The integration of PETs into the Deep Radiomics pipeline requires careful architectural design to balance privacy, model accuracy, and computational overhead.

Figure 2 : Integration of PETs with Deep Radiomics Pipeline.