[论文解读] Increasing-Margin Adversarial (IMA) Training to Improve Adversarial Robustness of Neural Networks

Linhai Ma, Liang Liang|arXiv (Cornell University)|May 19, 2020

Adversarial Robustness in Machine Learning参考文献 66被引用 8

一句话总结

本文提出了一种新型对抗训练方法——增益边界对抗（IMA）训练，通过均衡状态分析生成最优对抗样本，缓解了深度神经网络中标准准确率与鲁棒性之间的权衡问题。IMA在图像分类与分割任务中实现了最先进水平的鲁棒性，同时对干净数据准确率的下降极小，甚至在某一应用场景中同时提升了准确率与鲁棒性。

ABSTRACT

Deep neural networks (DNNs) are vulnerable to adversarial noises. Adversarial training is a general and effective strategy to improve DNN robustness (i.e., accuracy on noisy data) against adversarial noises. However, DNN models trained by the current existing adversarial training methods may have much lower standard accuracy (i.e., accuracy on clean data), compared to the same models trained by the standard method on clean data, and this phenomenon is known as the trade-off between accuracy and robustness and is considered unavoidable. This issue prevents adversarial training from being used in many application domains, such as medical image analysis, as practitioners do not want to sacrifice standard accuracy too much in exchange for adversarial robustness. Our objective is to lift (i.e., alleviate or even avoid) this trade-off between standard accuracy and adversarial robustness for medical image classification and segmentation. We propose a novel adversarial training method, named Increasing-Margin Adversarial (IMA) Training, which is supported by an equilibrium state analysis about the optimality of adversarial training samples. Our method aims to preserve accuracy while improving robustness by generating optimal adversarial training samples. We evaluate our method and the other eight representative methods on six publicly available image datasets corrupted by noises generated by AutoAttack and white-noise attack. Our method achieves the highest adversarial robustness for image classification and segmentation with the smallest reduction in accuracy on clean data. For one of the applications, our method improves both accuracy and robustness. Our study has demonstrated that our method can lift the trade-off between standard accuracy and adversarial robustness for the image classification and segmentation applications.

研究动机与目标

• 解决深度神经网络中标准准确率与对抗鲁棒性之间长期存在的权衡问题，尤其在医学图像应用中至关重要。
• 克服现有对抗训练方法在提升鲁棒性时显著降低标准准确率的局限性。
• 开发一种在保持高干净数据准确率的同时提升对抗鲁棒性的方法。
• 使对抗训练在医疗影像等安全关键领域中更广泛地应用，其中标准准确率不能被牺牲。
• 证明准确率-鲁棒性权衡并非不可避免，可通过改进对抗样本生成方法加以缓解。

提出的方法

• 提出一种基于对抗样本最优性均衡状态分析的新对抗训练框架——增益边界对抗（IMA）训练。
• 生成能最大化正确与错误类别预测之间边距的对抗样本，从而在不损害干净数据准确率的前提下优化鲁棒性。
• 在训练过程中引入动态边距调整机制，逐步提升模型鲁棒性，同时保持对干净数据的泛化能力。
• 引入一种损失函数，平衡标准交叉熵损失与基于边距的对抗损失，确保干净准确率与鲁棒性均得到提升。
• 利用均衡状态分析的理论洞见，指导生成最具信息量的最优对抗样本，以实现鲁棒性的最大提升。
• 在标准深度学习架构下，端到端地将该方法应用于图像分类与语义分割任务。

实验结果

研究问题

• RQ1 通过改进对抗样本生成，能否缓解深度神经网络中的标准准确率-鲁棒性权衡问题？
• RQ2 所提出的IMA训练方法是否在保持或提升干净数据准确率的同时，实现了比现有方法更高的对抗鲁棒性？
• RQ3 IMA训练在多样化的图像数据集上，能在多大程度上缩小标准准确率与鲁棒准确率之间的性能差距？
• RQ4 IMA训练能否在真实世界应用中同时提升标准准确率与对抗鲁棒性？
• RQ5 对抗样本最优性均衡状态分析如何为设计更有效的对抗训练策略提供指导？

主要发现

• 在六组公开图像数据集上，IMA训练在AutoAttack与白噪声攻击下均实现了最高的对抗鲁棒性，优于八种基线方法。
• 与其他对抗训练方法相比，IMA方法在干净数据上的标准准确率下降最小，有效缓解了标准准确率-鲁棒性权衡问题。
• 在某一应用场景中，IMA训练同时提升了干净数据准确率与对抗鲁棒性，证明该权衡并非固有缺陷。
• 均衡状态分析为生成能最大化鲁棒性增益且最小化准确率损失的最优对抗样本提供了理论基础。
• IMA训练在图像分类与语义分割任务中均保持了强劲性能，展现出广泛的适用性。
• 该方法在多样化的数据集与攻击类型下表现稳健，证实其泛化能力超越特定架构或数据分布。