[论文解读] Interpreting Adversarial Examples by Activation Promotion and Suppression

Kaidi Xu, Sijia Liu+6 more|arXiv (Cornell University)|Apr 3, 2019

Adversarial Robustness in Machine Learning参考文献 33被引用 39

一句话总结

本文从像素、图像和网络视角分析对抗扰动，揭示在激活上的 promotion-suppression effect (PSE)，并将扰动与类别判别区域和语义概念联系起来。它还探索了 interpretability-driven robustness 与 neuron masking 作为防御的可能性。

ABSTRACT

It is widely known that convolutional neural networks (CNNs) are vulnerable to adversarial examples: images with imperceptible perturbations crafted to fool classifiers. However, interpretability of these perturbations is less explored in the literature. This work aims to better understand the roles of adversarial perturbations and provide visual explanations from pixel, image and network perspectives. We show that adversaries have a promotion-suppression effect (PSE) on neurons' activations and can be primarily categorized into three types: i) suppression-dominated perturbations that mainly reduce the classification score of the true label, ii) promotion-dominated perturbations that focus on boosting the confidence of the target label, and iii) balanced perturbations that play a dual role in suppression and promotion. We also provide image-level interpretability of adversarial examples. This links PSE of pixel-level perturbations to class-specific discriminative image regions localized by class activation mapping (Zhou et al. 2016). Further, we examine the adversarial effect through network dissection (Bau et al. 2017), which offers concept-level interpretability of hidden units. We show that there exists a tight connection between the units' sensitivity to adversarial attacks and their interpretability on semantic concepts. Lastly, we provide some new insights from our interpretation to improve the adversarial robustness of networks.

研究动机与目标

• 研究像素级扰动如何从像素和图像视角影响 CNN 预测。
• 用注意力与区域分析解释不同对抗攻击的有效性。
• 研究对抗扰动如何影响 CNN 内部表征和概念检测器。
• 将扰动模式与类别判别的图像区域和语义概念联系起来，以获得鲁棒性洞见。

提出的方法

• 定义一个像素级敏感性度量，聚合扰动区域内真实类别和目标类别 logits 的变化。
• 引入 promotion-suppression ratio (PSR) 将扰动分为 suppression-dominated、promotion-dominated 或 balance-dominated。
• 利用 class activation maps (CAM) 来解释扰动如何与判别性图像区域对齐。
• 基于 CAM 应用 interpretability scores (IS) 以量化扰动与判别性区域的对齐程度。
• 采用 network dissection 评估扰动如何影响概念检测器及其可解释性。
• 提出使用 CAM 的细化策略以约束扰动并研究效果。

实验结果

研究问题

• RQ1 像素级扰动如何在真实标签和目标标签之间促进或抑制 CNN 激活？
• RQ2 如何通过 CAM 和 IS 量化对抗扰动的图像级可解释性？
• RQ3 如 network dissection 所揭示，对抗扰动与内部网络概念之间的关系是什么？
• RQ4 基于解释的细化与 neuron masking 是否能在不显著损害干净样本准确率的情况下提升鲁棒性？
• RQ5 在不同攻击与目标下，扰动模式如何与判别性图像区域相关？

主要发现

• 对抗扰动在神经元激活上呈现 promotion-suppression effect，可归类为 suppression-dominated、promotion-dominated 或 balanced。
• CAM-based analysis shows perturbations align with discriminative regions for true and target labels, enabling image-level interpretability of attacks.
• There is a measurable link between pixel perturbations and semantic concepts via network dissection, with sensitive units often being interpretable concepts.
• Interpretability-driven refinements can yield more effective attacks by targeting high-sensitivity regions, while certain CAM-constrained refinements may require larger perturbations.
• Sensitive units to adversarial perturbations tend to correspond to high-interpretability concept detectors, especially in deeper network layers.