[论文解读] Boosting Gradient for White-Box Adversarial Attacks

Hongying Liu, Zhenyu Zhou+4 more|arXiv (Cornell University)|Oct 21, 2020

Adversarial Robustness in Machine Learning参考文献 27被引用 10

一句话总结

本文提出ADV-ReLU，一种新颖方法，通过在反向传播过程中修正ReLU激活函数中的两个梯度误导问题——'错误阻断'和'过度传输'——来增强基于梯度的白盒对抗攻击。通过基于分数的选择和自适应阈值化方法优化梯度计算，ADV-ReLU降低了ℓ₂-范数扰动，并提升了迁移性，在将攻击迁移至Inception-v4时，成功率达到基线攻击的7.0%更高。

ABSTRACT

Deep neural networks (DNNs) are playing key roles in various artificial intelligence applications such as image classification and object recognition. However, a growing number of studies have shown that there exist adversarial examples in DNNs, which are almost imperceptibly different from original samples, but can greatly change the network output. Existing white-box attack algorithms can generate powerful adversarial examples. Nevertheless, most of the algorithms concentrate on how to iteratively make the best use of gradients to improve adversarial performance. In contrast, in this paper, we focus on the properties of the widely-used ReLU activation function, and discover that there exist two phenomena (i.e., wrong blocking and over transmission) misleading the calculation of gradients in ReLU during the backpropagation. Both issues enlarge the difference between the predicted changes of the loss function from gradient and corresponding actual changes, and mislead the gradients which results in larger perturbations. Therefore, we propose a universal adversarial example generation method, called ADV-ReLU, to enhance the performance of gradient based white-box attack algorithms. During the backpropagation of the network, our approach calculates the gradient of the loss function versus network input, maps the values to scores, and selects a part of them to update the misleading gradients. Comprehensive experimental results on \emph{ImageNet} demonstrate that our ADV-ReLU can be easily integrated into many state-of-the-art gradient-based white-box attack algorithms, as well as transferred to black-box attack attackers, to further decrease perturbations in the ${\ell _2}$-norm.

研究动机与目标

• 为解决ReLU激活函数在深度神经网络反向传播过程中扭曲梯度计算的被低估作用。
• 识别并纠正两种特定的梯度误导现象——'错误阻断'和'过度传输'——以改善对抗攻击质量。
• 开发一种通用、即插即用的框架，无需架构修改即可增强现有基于梯度的白盒攻击算法。
• 提升对抗样本在多种网络架构中的鲁棒性和迁移性，尤其是在黑盒设置下。

提出的方法

• ADV-ReLU通过基于分数映射评估神经元重要性的方法，重新计算ReLU的梯度，从而修改反向传播过程。
• 引入约束常数$c$，以平衡输入值与继承梯度，减少ReLU导数带来的误导。
• 通过排序率$s$控制选择前导激活神经元的比例，以优化精度与噪声之间的权衡。
• 该方法采用两阶段梯度修正：ADV-ReLU-B用于偏差校正，ADV-ReLU-T用于梯度传播优化。
• 该框架设计为通用模块，可轻松集成至现有白盒攻击算法（如I-FGSM和MI-FGSM）中。
• 采用自适应步长$\alpha = 0.001$，以确保在迭代优化过程中实现平滑、梯度引导的扰动路径。

实验结果

研究问题

• RQ1 ReLU激活函数的导数特性在对抗攻击的反向传播过程中如何误导梯度计算？
• RQ2 纠正ReLU中的'错误阻断'和'过度传输'现象能在多大程度上提升对抗样本的质量？
• RQ3 像ADV-ReLU这样的通用梯度修正框架能否有效集成至多种白盒攻击算法中，以减少扰动幅度？
• RQ4 改进后的梯度一致性如何影响对抗样本向黑盒模型的迁移性？

主要发现

• ADV-ReLU在多个模型上均降低了白盒攻击的ℓ₂-范数扰动，其中在Inception-v4上的改进最为显著。
• 与I-FGSM结合使用时，ADV-ReLU使在Inception-v4上的攻击成功率相比基线提升了7.0%，证明了其迁移性的增强。
• 排序率$s$的最优参数范围为[0.01, 0.03]，在ResNet101等残差网络上，更高值会导致性能下降。
• 当约束常数$c > 0.1$时性能趋于稳定，表明更高的阈值可提升梯度一致性并减少误导。
• 较小的步长（如$\alpha = 0.001$）通过实现更平滑、更精确的梯度路径，增强了ADV-ReLU的有效性。
• ADV-ReLU-B与ADV-ReLU-T子算法表现出不同行为：ADV-ReLU-B在残差网络上对参数调优更敏感，而ADV-ReLU-T则在非残差架构中提升了梯度传播效率。