[论文解读] Adversarial Machine Learning: An Interpretation Perspective

Ninghao Liu, Mengnan Du+1 more|arXiv (Cornell University)|Apr 23, 2020

Adversarial Robustness in Machine Learning参考文献 72被引用 6

一句话总结

本文提出一种统一的解释视角，以理解机器学习中的对抗鲁棒性，将对抗攻击与防御视为可解释性的自然延伸。通过将解释分类为原始特征和模型组件两类，本文展示了可解释性技术如何增强攻击生成与防御机制，为模型脆弱性与鲁棒性提供了新见解。

ABSTRACT

Recent years have witnessed the significant advances of machine learning in a wide spectrum of applications. However, machine learning models, especially deep neural networks, have been recently found to be vulnerable to carefully-crafted input called adversarial samples. The difference between normal and adversarial samples is almost imperceptible to human. Many work have been proposed to study adversarial attack and defense in different scenarios. An intriguing and crucial aspect among those work is to understand the essential cause of model vulnerability, which requires in-depth exploration of another concept in machine learning models, i.e., interpretability. Interpretable machine learning tries to extract human-understandable terms for the working mechanism of models, which also receives a lot of attention from both academia and industry. Recently, an increasing number of work start to incorporate interpretation into the exploration of adversarial robustness. Furthermore, we observe that many previous work of adversarial attacking, although did not mention it explicitly, can be regarded as natural extension of interpretation. In this paper, we review recent work on adversarial attack and defense, particularly, from the perspective of machine learning interpretation. We categorize interpretation into two types, according to whether it focuses on raw features or model components. For each type of interpretation, we elaborate on how it could be used in attacks, or defense against adversaries. After that, we briefly illustrate other possible correlations between the two domains. Finally, we discuss the challenges and future directions along tackling adversary issues with interpretation.

研究动机与目标

• 通过将可解释性整合到对抗机器学习中，探究模型对对抗样本脆弱性的根本原因。
• 将解释方法分类为基于原始特征与基于模型组件的方法，以实现系统性分析。
• 展示如何利用可解释性技术来改进对抗攻击策略与防御机制。
• 识别并讨论深度学习模型中可解释性与对抗鲁棒性之间新兴的相关性。
• 概述通过可解释性提升对抗鲁棒性的开放挑战与未来研究方向。

提出的方法

• 将解释方法分为两类：(1) 聚焦于原始输入特征的解释，(2) 聚焦于神经元或层等内部模型组件的解释。
• 分析解释技术如何通过识别显著特征或模型敏感组件，指导对抗样本的生成。
• 通过突出决策关键特征或组件，利用解释技术检测并缓解模型脆弱性。
• 将现有对抗攻击方法映射到解释框架中，表明许多攻击通过扰动关键特征或组件隐式执行了解释。
• 通过可解释性分析识别的关键组件进行修改或正则化，利用解释技术设计更具鲁棒性的模型。
• 提出一个概念性框架，将对抗鲁棒性定位为可解释模型设计的衍生属性。

实验结果

研究问题

• RQ1 在对抗机器学习背景下，解释技术如何系统性地分类？
• RQ2 对原始特征的解释在生成有效对抗攻击方面有何作用？
• RQ3 对模型组件的解释如何增强对抗样本的防御机制？
• RQ4 现有对抗攻击方法与解释技术之间存在何种隐含联系？
• RQ5 如何利用可解释性来提升深度神经网络对对抗扰动的鲁棒性？

主要发现

• 聚焦于原始特征的解释技术可揭示模型预测中最具影响力的输入区域，从而实现有针对性的对抗扰动。
• 对神经元或注意力头等模型组件的解释可暴露易受对抗操纵的决策路径。
• 许多现有对抗攻击方法隐式执行了解释，通过识别并利用显著特征或组件，即使未明确提及亦然。
• 可解释性可通过识别并正则化脆弱组件来增强防御，从而提升模型鲁棒性。
• 将可解释性整合到对抗鲁棒性研究中，揭示了模型透明性与韧性之间更深层次、系统性的关联。
• 未来工作应聚焦于开发基于可解释性的防御框架，确保其在多样化模型架构上的鲁棒性与泛化能力。