[Paper Review] Privacy Inference Attacks and Defenses in Cloud-based Deep Neural Network: A Survey

Xiaoyu Zhang, Chao Chen+4 more|arXiv (Cornell University)|May 13, 2021

Privacy-Preserving Technologies in Data138 references4 citations

TL;DR

This survey provides a comprehensive analysis of privacy inference attacks and defenses in cloud-based deep neural networks (DNNs), systematically categorizing threats across data manipulation, training, and prediction phases. It introduces the cloud-based ML privacy game framework, reviews state-of-the-art defense techniques—including differential privacy, adversarial training, and cryptographic methods—and identifies key challenges in balancing privacy, utility, and efficiency.

ABSTRACT

Deep Neural Network (DNN), one of the most powerful machine learning algorithms, is increasingly leveraged to overcome the bottleneck of effectively exploring and analyzing massive data to boost advanced scientific development. It is not a surprise that cloud computing providers offer the cloud-based DNN as an out-of-the-box service. Though there are some benefits from the cloud-based DNN, the interaction mechanism among two or multiple entities in the cloud inevitably induces new privacy risks. This survey presents the most recent findings of privacy attacks and defenses appeared in cloud-based neural network services. We systematically and thoroughly review privacy attacks and defenses in the pipeline of cloud-based DNN service, i.e., data manipulation, training, and prediction. In particular, a new theory, called cloud-based ML privacy game, is extracted from the recently published literature to provide a deep understanding of state-of-the-art research. Finally, the challenges and future work are presented to help researchers to continue to push forward the competitions between privacy attackers and defenders.

Motivation & Objective

• To systematically analyze privacy threats in cloud-based deep neural network (DNN) services across the entire pipeline: data manipulation, training, and prediction.
• To categorize and evaluate four major types of privacy inference attacks: membership, property, model inversion, and model extraction attacks.
• To review and synthesize state-of-the-art defense mechanisms, including differential privacy, adversarial training, watermarking, and cryptographic techniques.
• To introduce the cloud-based ML privacy game framework as a theoretical foundation for understanding attacker-defender dynamics in DNN privacy.
• To identify open challenges and future research directions in balancing privacy, model utility, and computational efficiency in secure cloud-based DNNs.

Proposed method

• The paper conducts a systematic literature review of peer-reviewed works published over the past five years on privacy in cloud-based DNNs.
• It classifies privacy attacks into four categories: membership inference, property inference, model inversion, and model extraction, based on attacker goals and access levels.
• Defenses are analyzed along three dimensions: data-level (e.g., differential privacy, data perturbation), model-level (e.g., adversarial training, gradient masking), and system-level (e.g., homomorphic encryption, secure multi-party computation).
• The cloud-based ML privacy game framework is formalized to model the strategic interaction between attackers and defenders in DNN privacy.
• The survey evaluates defense mechanisms based on privacy guarantees, model utility, and computational efficiency, identifying trade-offs in real-world deployment.
• It proposes a research agenda focused on dynamic privacy budget allocation, customized protection for heterogeneous data contributors, and formalizing privacy-by-design principles for DNN training frameworks.

Experimental results

Research questions

• RQ1 What are the primary privacy threats in cloud-based DNN services across the data manipulation, training, and prediction phases?
• RQ2 How do different types of privacy inference attacks—membership, property, model inversion, and model extraction—exploit vulnerabilities in cloud-based DNNs?
• RQ3 What are the most effective defense mechanisms for protecting data, model, and query privacy in cloud-based DNNs, and what are their trade-offs?
• RQ4 How can the cloud-based ML privacy game framework be used to model and analyze the strategic interaction between privacy attackers and defenders?
• RQ5 What are the key open challenges in achieving a sustainable balance between privacy, model utility, and computational efficiency in secure DNN outsourcing?

Key findings

• Membership inference attacks can successfully determine whether a given data sample was part of a model’s training set using only black-box access, posing a significant threat to data privacy.
• Model inversion attacks can reconstruct sensitive training data samples from model outputs, especially when the model is trained on private or sensitive data such as medical records.
• Model extraction attacks allow adversaries to clone a target DNN’s functionality with high accuracy using only query access, undermining model intellectual property.
• Cryptographic defenses like fully homomorphic encryption (FHE) and secure multi-party computation (SMC) provide strong privacy guarantees but incur high computational and communication overhead.
• Differential privacy mechanisms effectively limit information leakage but introduce a trade-off between privacy and model utility, where stronger privacy leads to reduced prediction accuracy.
• Future research must focus on dynamic, personalized privacy budget allocation in collaborative learning settings to meet diverse participant privacy requirements while preserving model utility.