[论文解读] Adversarial Attacks and Defences: A Survey

Anirban Chakraborty, Manaar Alam+3 more|arXiv (Cornell University)|Sep 28, 2018

Adversarial Robustness in Machine Learning参考文献 23被引用 455

一句话总结

对深度学习中的对抗攻击及其威胁模型（白盒/黑盒）、攻击面、探索性攻击和中毒攻击以及防御的全面综述，附有实用见解和分类法。

ABSTRACT

Deep learning has emerged as a strong and efficient framework that can be applied to a broad spectrum of complex learning problems which were difficult to solve using the traditional machine learning techniques in the past. In the last few years, deep learning has advanced radically in such a way that it can surpass human-level performance on a number of tasks. As a consequence, deep learning is being extensively used in most of the recent day-to-day applications. However, security of deep learning systems are vulnerable to crafted adversarial examples, which may be imperceptible to the human eye, but can lead the model to misclassify the output. In recent times, different types of adversaries based on their threat model leverage these vulnerabilities to compromise a deep learning system where adversaries have high incentives. Hence, it is extremely important to provide robustness to deep learning algorithms against these adversaries. However, there are only a few strong countermeasures which can be used in all types of attack scenarios to design a robust deep learning system. In this paper, we attempt to provide a detailed discussion on different types of adversarial attacks with various threat models and also elaborate the efficiency and challenges of recent countermeasures against them.

研究动机与目标

• 总结深度神经网络及相关模型的对抗攻击全景。
• 按威胁模型、阶段（训练 vs 测试）和应用对攻击进行分类。
• 讨论各类攻击下的防御及其局限性。
• 提供面向鲁棒ML系统设计的分类体系和实用指导。

提出的方法

• 建立术语与威胁模型的定性分类体系。
• 对训练阶段和测试阶段的攻击面与对抗能力进行分类。
• 系统性地评估探索性、规避和中毒攻击及其相关防御。
• 综述重要攻击及应用，并与关键研究相互引用。

实验结果

研究问题

• RQ1 机器学习系统的主要威胁模型和攻击面是什么？
• RQ2 训练时（中毒）和测试时（规避）场景下攻击有何不同？
• RQ3 存在哪些防御方法，以及它们在不同攻击类别中的局限性？
• RQ4 哪些攻击在现实世界系统和服务中得到验证，包括ML API？
• RQ5 如何在概念上组织对抗威胁以指导鲁棒设计？

主要发现

• 白盒和黑盒攻击模型由对目标模型和训练过程的对手知识决定区分。
• 规避攻击主导测试时威胁，而中毒攻击影响训练数据和模型完整性。
• 探索性攻击在不改变训练集的情况下揭示关于模型和训练数据的信息。
• GANs与生成框架既用作攻击工具，也用作防御机制。
• 防御通常针对特定攻击类别定制，可能降低模型性能或效率。
• 本综述把攻击与防御整理为分类体系，以帮助研究人员和从业者。