[论文解读] The Efficacy of SHIELD under Different Threat Models

Cory Cornelius, Das, Nilaksh+4 more|arXiv (Cornell University)|Feb 1, 2019

Adversarial Robustness in Machine Learning参考文献 19被引用 5

一句话总结

本文在攻击者掌握完整防御系统信息的自适应威胁模型下，评估了基于压缩的防御方法Shield的鲁棒性。结果表明，与从预训练权重微调相比，从头训练模型可使白盒环境下目标PGD攻击的成功率从64.3%降至48.9%，表明权重相关性显著影响模型鲁棒性。

ABSTRACT

In this appraisal paper, we evaluate the efficacy of SHIELD, a compression-based defense framework for countering adversarial attacks on image classification models, which was published at KDD 2018. Here, we consider alternative threat models not studied in the original work, where we assume that an adaptive adversary is aware of the ensemble defense approach, the defensive pre-processing, and the architecture and weights of the models used in the ensemble. We define scenarios with varying levels of threat and empirically analyze the proposed defense by varying the degree of information available to the attacker, spanning from a full white-box attack to the gray-box threat model described in the original work. To evaluate the robustness of the defense against an adaptive attacker, we consider the targeted-attack success rate of the Projected Gradient Descent (PGD) attack, which is a strong gradient-based adversarial attack proposed in adversarial machine learning research. We also experiment with training the SHIELD ensemble from scratch, which is different from re-training using a pre-trained model as done in the original work. We find that the targeted PGD attack has a success rate of 64.3% against the original SHIELD ensemble in the full white box scenario, but this drops to 48.9% if the models used in the ensemble are trained from scratch instead of being retrained. Our experiments further reveal that an ensemble whose models are re-trained indeed have higher correlation in the cosine similarity space, and models that are trained from scratch are less vulnerable to targeted attacks in the white-box and gray-box scenarios.

研究动机与目标

• 评估Shield在攻击者逐步掌握防御系统信息的自适应敌手威胁下的鲁棒性。
• 评估强投影梯度下降（PGD）攻击在原始Shield论文未考虑的多种威胁模型下的有效性。
• 研究训练过程（特别是从预训练模型微调与从头训练）对模型权重相关性及防御鲁棒性的影响。
• 确定在白盒和灰盒场景下，权重相关性较低的集成模型是否对目标对抗攻击更具韧性。

提出的方法

• 在从完全白盒（攻击者知晓所有防御组件）到灰盒（攻击者仅知晓模型架构）的多种威胁模型下，对Shield进行实证评估。
• 使用目标PGD攻击测量不同威胁模型下的成功率，重点关注针对预测概率最低类别的攻击。
• 训练两种Shield集成模型：一种是基于JPEG压缩图像对预训练的ResNet-50 v2模型进行微调，另一种是直接从头开始在相同数据上训练。
• 通过计算模型特征图之间的余弦相似度来度量权重相关性，分析训练方式对集成多样性的影响。
• 通过在新训练的模型上生成对抗样本，并测试其在原始Shield集成上的迁移性，评估对抗样本的可迁移性。
• 比较不同威胁模型和训练方式下模型的准确率下降情况与攻击成功率。

实验结果

研究问题

• RQ1 在攻击者知晓集成架构、预处理方式和模型权重的完全白盒威胁模型下，目标PGD攻击在Shield上的表现如何？
• RQ2 与从预训练权重微调相比，从头训练Shield集成模型是否能提升其在白盒和灰盒环境下的抗目标攻击能力？
• RQ3 集成模型之间的权重相关性在多大程度上影响目标对抗攻击的成功率？
• RQ4 在灰盒场景下，针对新训练模型生成的对抗样本是否能成功欺骗原始Shield集成？
• RQ5 与非自适应威胁模型相比，Shield在自适应威胁模型下的鲁棒性如何，特别是在使用强梯度攻击（如PGD）时？

主要发现

• 在完全白盒威胁模型下，目标PGD攻击对原始Shield集成的攻击成功率高达64.3%，该集成使用的是微调后的模型。
• 当使用从头训练的模型替代微调模型时，目标PGD攻击的成功率降至48.9%，下降了15.4个百分点。
• 原始Shield集成中微调的模型在余弦相似度空间中表现出更高的权重相关性，导致集成对目标攻击更为脆弱。
• 从头训练的模型表现出更低的权重相关性，在白盒和灰盒威胁模型下均对目标攻击具有显著更强的鲁棒性。
• 在灰盒威胁模型下，即使模型准确率下降，攻击者仅知晓模型架构时，目标PGD攻击完全失败，成功率为0%。
• 在灰盒设置下，快速梯度法（FGM）同样无法控制预测结果，尽管其降低了模型准确率，攻击成功率仍为0%。