[Paper Review] Security and Privacy Issues in Deep Learning

Ho Bae, Jaehee Jang+5 more|arXiv (Cornell University)|Jul 31, 2018

Adversarial Robustness in Machine Learning236 references58 citations

TL;DR

This paper surveys security and privacy threats to deep learning, categorizing attacks (evasion, poisoning) and privacy breaches, and reviews defense mechanisms including cryptographic and privacy-preserving techniques.

ABSTRACT

To promote secure and private artificial intelligence (SPAI), we review studies on the model security and data privacy of DNNs. Model security allows system to behave as intended without being affected by malicious external influences that can compromise its integrity and efficiency. Security attacks can be divided based on when they occur: if an attack occurs during training, it is known as a poisoning attack, and if it occurs during inference (after training) it is termed an evasion attack. Poisoning attacks compromise the training process by corrupting the data with malicious examples, while evasion attacks use adversarial examples to disrupt entire classification process. Defenses proposed against such attacks include techniques to recognize and remove malicious data, train a model to be insensitive to such data, and mask the model's structure and parameters to render attacks more challenging to implement. Furthermore, the privacy of the data involved in model training is also threatened by attacks such as the model-inversion attack, or by dishonest service providers of AI applications. To maintain data privacy, several solutions that combine existing data-privacy techniques have been proposed, including differential privacy and modern cryptography techniques. In this paper, we describe the notions of some of methods, e.g., homomorphic encryption, and review their advantages and challenges when implemented in deep-learning models.

Motivation & Objective

• Motivate the need for secure and private AI (SPAI) in real-world DL deployments.
• Systematically categorize security threats to DL models during training and inference.
• Review defense strategies against evasion and poisoning attacks.
• Survey privacy threats to DL systems and summarize cryptographic/privacy-preserving defenses.

Proposed method

• Classify attacks by phase (training vs inference) and by access (white-box vs black-box).
• Describe evasion attacks (FGSM, CW, JSMA, BPDA, transfer/black-box attacks) and poisoning attacks (performance degradation, targeted poisoning, backdoor).
• Summarize defense approaches (gradient masking, robustness, detection, certification) against evasion; discuss poisoning defenses (data anomaly detection, pruning, fine-tuning).
• Discuss privacy threats (model inversion, multi-party data) and defenses (differential privacy, homomorphic encryption, secure multiparty computation).
• Highlight practical attack types (universal perturbations, boundary/zeroth-order/one-pixel attacks) and backdoor/trojaning strategies in AI systems.

Experimental results

Research questions

• RQ1 What are the main categories and characteristics of security attacks on DL models during training and inference?
• RQ2 What defense approaches exist for evasion and poisoning attacks, and what are their limitations?
• RQ3 What privacy threats affect DL systems in multi-party or outsourced settings, and what cryptographic/privacy-preserving methods address them?
• RQ4 How can SPAI concepts guide robust and private AI development?

Key findings

• Evasion attacks can be white-box or black-box and include FGSM, iterative FGSM, CW, JSMA, and universal perturbations with varying success and cost.
• Poisoning attacks are categorized into performance degradation, targeted poisoning, and backdoor attacks, with defenses including anomaly detection and robust training strategies.
• Defense techniques against evasion include gradient masking, robustness enhancement, detection, and certification; poisoning defenses rely on data filtering and retraining with clean data.
• Privacy attacks such as model inversion and data leakage motivate use of differential privacy, homomorphic encryption, and secure multiparty computation in DL workflows.