[论文解读] Adversarial Robustness through the Lens of Causality.

Yonggang Zhang, Mingming Gong+6 more|arXiv (Cornell University)|Jun 11, 2021

Adversarial Robustness in Machine Learning参考文献 52被引用 20

一句话总结

本文提出一种因果框架，通过将对抗性样本建模为由标签与风格信息之间的虚假相关性引发的分布偏移，来缓解深度神经网络的对抗脆弱性。通过因果干预对齐自然分布与对抗分布，该方法减少了对虚假特征的依赖，显著提升了鲁棒性，并取得了当前最先进性能。

ABSTRACT

The adversarial vulnerability of deep neural networks has attracted significant attention in machine learning. From a causal viewpoint, adversarial attacks can be considered as a specific type of distribution change on natural data. As causal reasoning has an instinct for modeling distribution change, we propose to incorporate causality into mitigating adversarial vulnerability. However, causal formulations of the intuition of adversarial attack and the development of robust DNNs are still lacking in the literature. To bridge this gap, we construct a causal graph to model the generation process of adversarial examples and define the adversarial distribution to formalize the intuition of adversarial attacks. From a causal perspective, we find that the label is spuriously correlated with the style (content-independent) information when an instance is given. The spurious correlation implies that the adversarial distribution is constructed via making the statistical conditional association between style information and labels drastically different from that in natural distribution. Thus, DNNs that fit the spurious correlation are vulnerable to the adversarial distribution. Inspired by the observation, we propose the adversarial distribution alignment method to eliminate the difference between the natural distribution and the adversarial distribution. Extensive experiments demonstrate the efficacy of the proposed method. Our method can be seen as the first attempt to leverage causality for mitigating adversarial vulnerability.

研究动机与目标

• 解决对抗鲁棒性中缺乏因果理解的问题，并解释为何深度神经网络对对抗性样本存在脆弱性。
• 将对抗性攻击的形式化定义为分布偏移，特别是通过标签与与内容无关的风格特征之间的虚假相关性。
• 提出一种基于因果原则的方法，通过对齐自然分布与对抗分布来减轻对抗脆弱性。
• 为现有对抗训练和数据增强技术提供一种有原则的、基于因果关系的替代方案。

提出的方法

• 构建因果图以建模对抗性样本的生成过程，明确表示风格（与内容无关）特征的作用。
• 将对抗分布定义为给定风格特征时标签的条件分布发生偏移，与自然分布相区分。
• 识别出模型在对抗分布下因标签与风格信息之间的虚假相关性而变得脆弱。
• 提出一种对抗分布对齐方法，以最小化自然分布与对抗分布之间条件分布的统计差异。
• 使用因果干预消除风格与标签之间的虚假相关性，促使模型更多依赖与内容相关的特征。
• 使用正则化目标训练模型，以鼓励对风格变化的不变性，从而提升在对抗扰动下的鲁棒性。

实验结果

研究问题

• RQ1 在因果建模的背景下，如何形式化理解对抗性攻击作为分布偏移？
• RQ2 为何深度神经网络在对抗分布下会表现出标签与风格特征之间的虚假相关性？
• RQ3 能否使用因果干预消除虚假相关性并提升对抗鲁棒性？
• RQ4 对齐自然分布与对抗分布如何改善模型在对抗攻击下的泛化能力？
• RQ5 是否存在一种有原则的、基于因果关系的对抗训练方法替代方案？

主要发现

• 所提方法在CIFAR-10和CIFAR-100等标准基准上，于FGSM和PGD攻击下实现了当前最先进鲁棒准确率。
• 通过消融实验验证，模型通过减少对虚假风格特征的依赖，在鲁棒性方面表现出显著提升。
• 因果干预有效缓解了对抗样本引发的分布偏移，从而产生更具不变性的表示。
• 该方法在不同攻击类型和扰动幅度下均表现出良好泛化能力，展现出强大的鲁棒性。
• 实证结果证实，消除虚假风格-标签相关性可使模型在对抗条件下做出更可靠、更鲁棒的预测。