[论文解读] Security and Privacy Issues in Deep Learning

Ho Bae, Jaehee Jang+5 more|arXiv (Cornell University)|Jul 31, 2018

Adversarial Robustness in Machine Learning参考文献 236被引用 58

一句话总结

本文综述深度学习的安全与隐私威胁，对攻击进行分类（规避攻击、污染攻击）和隐私泄露，并评述包括加密技术和隐私保护技术在内的防御机制。

ABSTRACT

To promote secure and private artificial intelligence (SPAI), we review studies on the model security and data privacy of DNNs. Model security allows system to behave as intended without being affected by malicious external influences that can compromise its integrity and efficiency. Security attacks can be divided based on when they occur: if an attack occurs during training, it is known as a poisoning attack, and if it occurs during inference (after training) it is termed an evasion attack. Poisoning attacks compromise the training process by corrupting the data with malicious examples, while evasion attacks use adversarial examples to disrupt entire classification process. Defenses proposed against such attacks include techniques to recognize and remove malicious data, train a model to be insensitive to such data, and mask the model's structure and parameters to render attacks more challenging to implement. Furthermore, the privacy of the data involved in model training is also threatened by attacks such as the model-inversion attack, or by dishonest service providers of AI applications. To maintain data privacy, several solutions that combine existing data-privacy techniques have been proposed, including differential privacy and modern cryptography techniques. In this paper, we describe the notions of some of methods, e.g., homomorphic encryption, and review their advantages and challenges when implemented in deep-learning models.

研究动机与目标

• 在现实世界的 DL 部署中，推动对安全和私有 AI (SPAI) 的需求。
• 系统性地对训练与推理阶段的 DL 模型安全威胁进行分类。
• 评述针对规避攻击和污染攻击的防御策略。
• 综述对 DL 系统的隐私威胁，并总结密码学/隐私保护防御措施。

提出的方法

• 按阶段（训练与推理）及访问方式（白盒与黑盒）对攻击进行分类。
• 描述规避攻击（FGSM、CW、JSMA、BPDA、迁移/黑盒攻击）和污染攻击（性能下降、定向污染、后门）。
• 概述针对规避的防御方法（梯度屏蔽、鲁棒性、检测、认证）；讨论污染防御（数据异常检测、剪枝、微调）。
• 讨论隐私威胁（模型反演、多方数据）及防御（差分隐私、同态加密、安全多方计算）。
• 强调实际攻击类型（通用扰动、边界/零阶/一像素攻击）以及 AI 系统中的后门/木马策略。

实验结果

研究问题

• RQ1 在训练和推理阶段，对 DL 模型的安全攻击有哪些主要类别和特征？
• RQ2 针对规避和污染攻击存在哪些防御方法，以及它们的局限性？
• RQ3 在多方或外包设置下，哪些隐私威胁影响 DL 系统，以及哪些密码学/隐私保护方法可以解决它们？
• RQ4 SPAI 概念如何指导稳健且私有的 AI 发展？

主要发现

• 规避攻击可以是白盒或黑盒，包括 FGSM、迭代 FGSM、CW、JSMA，以及具有不同成功率和成本的通用扰动。
• 污染攻击分为性能下降、定向污染和后门攻击，防御包括异常检测和鲁棒训练策略。
• 针对规避的防御技术包括梯度屏蔽、鲁棒性提升、检测和认证；污染防御依赖于数据筛选和用干净数据重新训练。
• 诸如模型反演和数据泄露等隐私攻击促使在 DL 工作流中使用差分隐私、同态加密和安全多方计算。