[论文解读] Does Interpretability of Neural Networks Imply Adversarial Robustness

Adam Noack, Isaac Ahern+2 more|arXiv (Cornell University)|Dec 7, 2019

Adversarial Robustness in Machine Learning参考文献 57被引用 10

一句话总结

本文研究了为可解释梯度训练深度神经网络是否能提升对抗鲁棒性。通过引入解释正则化（IR），使梯度与显著性图对齐并降低其幅值，作者表明，具有更高可解释性的梯度的模型在ℓ₂和ℓ∞对抗攻击下显著更具鲁棒性，尤其当与SmoothGrad结合时效果更明显。

ABSTRACT

Deep neural networks (DNNs) have had many successes, but suffer from two major issues: (1) a vulnerability to adversarial examples and (2) a tendency to elude human interpretation. Interestingly, recent empirical and theoretical evidence suggest these two seemingly disparate issues are actually connected. In particular, robust models tend to provide more interpretable gradients than non-robust models. However, whether this relationship works in the opposite direction remains obscure. With this paper, we seek empirical answers to the following question: can models acquire adversarial robustness when they are trained to have interpretable gradients? We introduce a theoretically inspired technique called \emph{Interpretation Regularization} (IR), which encourages a model's gradients to (1) match the direction of interpretable target salience maps and (2) have small magnitude. To assess model performance and tease apart factors that contribute to adversarial robustness, we conduct extensive experiments on MNIST and CIFAR-10 with both $\ell_2$ and $\ell_\infty$ attacks. We demonstrate that networks trained to have interpretable gradients are more robust to adversarial perturbations. Applying the network interpretation technique SmoothGrad \cite{smoothgrad} yields additional performance gains, especially in cross-norm attacks and under heavy perturbation. The results indicate that the interpretability of the model gradients is a crucial factor for adversarial robustness.

研究动机与目标

• 研究增强梯度可解释性是否能提升深度神经网络的对抗鲁棒性。
• 解决可解释性与鲁棒性之间是否存在因果关系而非仅相关性的开放问题。
• 开发并评估一种新型训练技术，明确促进可解释梯度。
• 将梯度可解释性的贡献与其他鲁棒性增强因素分离开来。

提出的方法

• 提出解释正则化（IR），一种训练目标，鼓励模型梯度与预定义的显著性图对齐。
• 将IR形式化为一个损失项，通过最小化模型梯度与目标显著性图之间的余弦相似度来实现。
• 引入第二个正则化项以抑制梯度的幅值，促进更平滑的决策边界。
• 在MNIST和CIFAR-10上使用标准训练和对抗训练，结合与不结合IR来训练模型。
• 应用SmoothGrad后处理解释方法，以优化梯度解释并提升鲁棒性。
• 在ℓ₂和ℓ∞攻击下评估鲁棒性，包括跨范数攻击和高扰动设置。

实验结果

研究问题

• RQ1 为深度神经网络训练可解释梯度是否能提升对抗鲁棒性？
• RQ2 梯度与显著性图对齐在多大程度上促进了鲁棒性？
• RQ3 通过正则化降低梯度幅值是否能增强鲁棒性？
• RQ4 IR与SmoothGrad结合后，在不同类型的攻击下对鲁棒性有何影响？
• RQ5 鲁棒性的提升是源于可解释性，还是其他因素在起作用？

主要发现

• 使用解释正则化（IR）训练的模型在ℓ₂和ℓ∞攻击下的鲁棒准确率显著高于标准训练和对抗训练基线模型。
• IR与SmoothGrad结合可带来显著的鲁棒性提升，尤其在跨范数攻击和高扰动设置下。
• 使用IR训练的鲁棒模型生成的梯度更具可解释性，表现为与显著性图的对齐度更高。
• IR训练的模型中梯度幅值始终减小，表明决策边界更加平滑。
• 消融实验证明，鲁棒性的提升与梯度可解释性直接相关。
• IR为对抗鲁棒性提供了一种新颖且有效的归纳偏置，可与标准对抗训练相辅相成。