[论文解读] Revisiting Loss Landscape for Adversarial Robustness

Dongxian Wu, Yisen Wang+1 more|arXiv (Cornell University)|Apr 13, 2020

Adversarial Robustness in Machine Learning参考文献 38被引用 10

一句话总结

本文提出对抗权重扰动（Adversarial Weight Perturbation, AWP），一种简单而有效的模块，通过在对抗训练过程中正则化对抗损失景观的平坦度，从而提升对抗鲁棒性。实验表明，AWP能够使损失景观更加平坦，并在多种对抗训练变体中增强鲁棒性。

ABSTRACT

The study on improving the robustness of deep neural networks against adversarial examples grows rapidly in recent years. Among them, adversarial training is the most promising one, based on which, a lot of improvements have been developed, such as adding regularizations or leveraging unlabeled data. However, these improvements seem to come from isolated perspectives, so that we are curious about if there is something in common behind them. In this paper, we investigate the surface geometry of several well-recognized adversarial training variants, and reveal that their adversarial loss landscape is closely related to the adversarially robust generalization, i.e., the flatter the adversarial loss landscape, the smaller the adversarially robust generalization gap. Based on this finding, we then propose a simple yet effective module, Adversarial Weight Perturbation (AWP), to directly regularize the flatness of the adversarial loss landscape in the adversarial training framework. Extensive experiments demonstrate that AWP indeed owns flatter landscape and can be easily incorporated into various adversarial training variants to enhance their adversarial robustness further.

研究动机与目标

• 探究对抗训练中的常见改进是否在损失景观中具有统一的几何特性。
• 探讨对抗损失景观的平坦度与对抗鲁棒泛化差距之间的关系。
• 开发一种统一的正则化方法，直接优化景观平坦度以提升鲁棒性。
• 在多种对抗训练框架中证明所提方法的有效性。

提出的方法

• 作者分析多种对抗训练变体的表面几何特性，以识别其损失景观中的共同模式。
• 提出对抗权重扰动（AWP），在训练过程中施加有针对性的权重扰动，以促使对抗损失表面更加平坦。
• 通过在对抗样本梯度方向上扰动网络权重，将AWP集成到标准对抗训练循环中。
• 该方法设计为模块化，可与多种现有对抗训练方法兼容。
• 测量并关联对抗损失景观的平坦度与泛化性能。

实验结果

研究问题

• RQ1 是否存在一种共同的几何特性——特别是损失景观平坦度——作为成功对抗训练方法的内在基础？
• RQ2 对抗损失景观的平坦度与对抗鲁棒泛化差距的大小之间存在何种关系？
• RQ3 显式正则化景观平坦度是否能在多种训练框架中提升对抗鲁棒性？
• RQ4 所提出的AWP模块在实践中是否能有效平坦化对抗损失景观？

主要发现

• 在多种训练变体中，更平坦的对抗损失景观始终与更小的对抗鲁棒泛化差距相关联。
• AWP通过所提出的平坦度度量方法，成功使对抗损失景观更加平坦。
• AWP在集成到多种现有对抗训练框架中时，显著提升了对抗鲁棒性。
• AWP带来的性能提升稳定且显著，证明了其有效性与泛化能力。