[论文解读] Robust Models are less Over-Confident

Julia Grabinski, Paul Gavrikov+2 more|arXiv (Cornell University)|Oct 12, 2022

Adversarial Robustness in Machine Learning被引用 6

一句话总结

该论文表明，对抗性训练（鲁棒）模型即使在干净数据上，其预测置信度也显著低于非鲁棒模型，且校准性能更优。关键发现是，鲁棒性训练本身能有效降低过自信，而诸如先进激活函数和下采样层等网络组件的改进可进一步提升校准效果，使鲁棒模型在实际部署中更具可靠性。

ABSTRACT

Despite the success of convolutional neural networks (CNNs) in many academic benchmarks for computer vision tasks, their application in the real-world is still facing fundamental challenges. One of these open problems is the inherent lack of robustness, unveiled by the striking effectiveness of adversarial attacks. Current attack methods are able to manipulate the network's prediction by adding specific but small amounts of noise to the input. In turn, adversarial training (AT) aims to achieve robustness against such attacks and ideally a better model generalization ability by including adversarial samples in the trainingset. However, an in-depth analysis of the resulting robust models beyond adversarial robustness is still pending. In this paper, we empirically analyze a variety of adversarially trained models that achieve high robust accuracies when facing state-of-the-art attacks and we show that AT has an interesting side-effect: it leads to models that are significantly less overconfident with their decisions, even on clean data than non-robust models. Further, our analysis of robust models shows that not only AT but also the model's building blocks (like activation functions and pooling) have a strong influence on the models' prediction confidences. Data & Project website: https://github.com/GeJulia/robustness_confidences_evaluation

研究动机与目标

• 探究对抗性鲁棒性与模型校准（尤其是预测置信度）之间的关系。
• 分析鲁棒模型在干净和对抗性输入上是否比非鲁棒模型更少过自信。
• 评估网络架构组件（如激活函数和下采样层）对模型置信度与校准的影响。
• 探讨鲁棒模型中的预测置信度是否可作为错误预测的指示器。
• 提供鲁棒性与校准的统一分析，倡导未来研究中应同时考虑两者。

提出的方法

• 在CIFAR-10、CIFAR-100和ImageNet上，使用标准训练和对抗性训练方法，分别训练了71对鲁棒与非鲁棒模型。
• 通过在干净样本和对抗样本上测量Softmax概率的输出置信度分布，评估正确与错误预测的置信度。
• 评估架构组件（特别是改进的激活函数，如SiLU，以及下采样层）对置信度校准的影响。
• 使用置信度阈值检测错误预测，评估置信度作为模型不确定性代理的可靠性。
• 比较模型在已知与未见过的对抗攻击下的泛化性能，以评估对特定扰动的过拟合情况。
• 发布了一个全面的模型库和评估代码，以支持可复现性与进一步研究。

实验结果

研究问题

• RQ1 与非鲁棒模型相比，对抗性训练模型在干净数据上的预测是否表现出更低的过自信？
• RQ2 如激活函数和下采样层等架构组件如何影响鲁棒模型的置信度校准？
• RQ3 鲁棒模型中的预测置信度能否可靠地指示错误预测？
• RQ4 鲁棒模型是否能泛化到未见过的对抗攻击，还是对训练期间遇到的特定扰动存在过拟合？
• RQ5 对抗性训练在提升鲁棒性之外，能在多大程度上内在地改善模型校准？

主要发现

• 非鲁棒模型在错误预测上表现出显著的过自信，其在对抗样本上的平均置信度超过95%。
• 鲁棒模型在干净和对抗输入上均表现出更低的置信度，错误预测的平均置信度降至约50–60%，表明其校准性能更优。
• 采用改进架构组件（如SiLU激活函数和专用下采样层）的模型展现出更优的置信度校准，显著降低了错误预测的过自信现象。
• 鲁棒模型中的预测置信度可有效检测错误预测，高置信度输出与正确分类高度相关。
• 尽管校准性能有所提升，鲁棒模型仍对训练期间使用的特定对抗攻击存在过拟合，其在未见过的攻击上性能退化程度与非鲁棒模型相似。
• 分析结果证实，对抗性训练不仅使模型更具鲁棒性，还使其不确定性估计更可靠，表明鲁棒性训练具有有益的附加效应。