[论文解读] Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness against Adversarial Attack

Adnan Siraj Rakin, Zhezhi He+1 more|arXiv (Cornell University)|Nov 22, 2018

Adversarial Robustness in Machine Learning被引用 11

一句话总结

本文提出参数化噪声注入（PNI），一种可训练的随机性技术，在对抗训练过程中通过可学习的缩放参数在逐层注入高斯噪声，以提升深度神经网络的鲁棒性。PNI在ResNet-20上针对PGD攻击实现了最先进性能，相较于原始对抗训练，干净数据准确率提升1.1%，扰动数据准确率提升6.8%。

ABSTRACT

Recent development in the field of Deep Learning have exposed the underlying vulnerability of Deep Neural Network (DNN) against adversarial examples. In image classification, an adversarial example is a carefully modified image that is visually imperceptible to the original image but can cause DNN model to misclassify it. Training the network with Gaussian noise is an effective technique to perform model regularization, thus improving model robustness against input variation. Inspired by this classical method, we explore to utilize the regularization characteristic of noise injection to improve DNN's robustness against adversarial attack. In this work, we propose Parametric-Noise-Injection (PNI) which involves trainable Gaussian noise injection at each layer on either activation or weights through solving the min-max optimization problem, embedded with adversarial training. These parameters are trained explicitly to achieve improved robustness. To the best of our knowledge, this is the first work that uses trainable noise injection to improve network robustness against adversarial attacks, rather than manually configuring the injected noise level through cross-validation. The extensive results show that our proposed PNI technique effectively improves the robustness against a variety of powerful white-box and black-box attacks such as PGD, C & W, FGSM, transferable attack and ZOO attack. Last but not the least, PNI method improves both clean- and perturbed-data accuracy in comparison to the state-of-the-art defense methods, which outperforms current unbroken PGD defense by 1.1 % and 6.8 % on clean test data and perturbed test data respectively using Resnet-20 architecture.

研究动机与目标

• 解决深度神经网络对对抗样本的脆弱性问题，即微小扰动输入导致误分类。
• 克服现有防御方法依赖固定噪声水平或在PGD和C&W等强攻击下失效的局限性。
• 开发一种可泛化、可训练的正则化方法，在不牺牲干净数据准确率的前提下增强鲁棒性。
• 证明可学习噪声注入可超越人工配置噪声，避免梯度遮蔽陷阱。
• 建立一种对多样化对抗攻击（包括可迁移攻击和黑盒攻击）均有效的防御方法。

提出的方法

• 提出参数化噪声注入（PNI），一种在训练期间应用于激活层或权重层的可学习噪声注入机制。
• 将PNI整合进最小-最大对抗训练框架中，使噪声缩放参数端到端优化。
• 为每层使用可学习的噪声缩放系数，动态调整噪声强度，替代人工超参数调优。
• 通过每层特定的可学习参数，在网络激活或权重中注入高斯噪声，实现自适应鲁棒性学习。
• 使用PGD、FGSM和C&W攻击生成的对抗样本进行模型训练，噪声注入在前向和反向传播中均应用。
• 通过在对抗扰动下联合优化网络权重与噪声参数，确保鲁棒性。

实验结果

研究问题

• RQ1 可学习噪声注入能否提升深度神经网络对各类白盒与黑盒对抗攻击的鲁棒性？
• RQ2 通过端到端优化学习每层的噪声强度，是否优于固定且人工调优的噪声水平？
• RQ3 PNI能否同时提升干净数据与扰动数据的准确率，而无需像许多先前防御方法那样牺牲干净准确率？
• RQ4 PNI带来的鲁棒性增益是源于梯度遮蔽还是真正的对抗鲁棒性？
• RQ5 当攻击强度增加时（如PGD攻击步数增多或扰动范围增大），PNI的表现如何？

主要发现

• 在ResNet-20上，PNI相较于原始对抗训练，干净数据准确率提升1.1%，表明泛化能力增强。
• 在PGD白盒攻击下，PNI相较于最先进且未被攻破的PGD防御方法，扰动数据准确率提升6.8%。
• PNI在多种架构上对FGSM、C&W、ZOO及可迁移攻击均保持强鲁棒性。
• 该方法通过了[23]中所有梯度遮蔽检测，表明其鲁棒性并非源于梯度掩蔽。
• 即使PGD攻击步数增加至100，PNI仍显著优于原始对抗训练，证实其鲁棒性超越随机梯度效应。
• PNI表明，在黑盒攻击场景下，权重级噪声注入比激活级噪声更有效，如表5所示。