[论文解读] Adversarially Robust Few-Shot Learning: A Meta-Learning Approach

Micah Goldblum, Liam Fowl+1 more|arXiv (Cornell University)|Oct 2, 2019

Adversarial Robustness in Machine Learning参考文献 33被引用 42

一句话总结

本文提出 Adversarial Querying (AQ)，一种与具体算法无关的元学习方法，用于生成对对抗攻击鲁棒的少样本模型，在标准基准上优于鲁棒迁移学习。

ABSTRACT

Previous work on adversarially robust neural networks for image classification requires large training sets and computationally expensive training procedures. On the other hand, few-shot learning methods are highly vulnerable to adversarial examples. The goal of our work is to produce networks which both perform well at few-shot classification tasks and are simultaneously robust to adversarial examples. We develop an algorithm, called Adversarial Querying (AQ), for producing adversarially robust meta-learners, and we thoroughly investigate the causes for adversarial vulnerability. Moreover, our method achieves far superior robust performance on few-shot image classification tasks, such as Mini-ImageNet and CIFAR-FS, than robust transfer learning.

研究动机与目标

• 在少样本学习情境中说明对鲁棒性的需求。
• 开发一个元学习框架，为少样本任务产生对抗鲁棒的特征提取器。
• 证明 adversarial querying (AQ) 相对于现有防御措施在鲁棒性和清洁精度方面具有更优表现。
• 展示在固定骨干网络、仅对最后一层进行微调能够提高鲁棒性。
• 在多种元学习骨架和数据集上提供算法无关的证据。

提出的方法

• 为元学习制定极小极大目标：min_theta E_{S,(x,y)} [ max_{||δ||_p<ε} L(F_{A(θ,S)}, x+δ, y) ].
• 提出 adversarial querying (AQ)：在支持数据上进行微调与对查询数据进行对抗性扰动以最大化损失之间交替，并通过 A 反向传播。
• 在 AQ 过程中仅对查询数据进行攻击，以在鲁棒性提升与计算效率之间取得平衡。
• 在 CIFAR-FS 和 Mini-ImageNet 上对 ProtoNet、R2-D2 和 MetaOptNet 进行 AQ 测试。
• 将 AQ 与对抗性训练的迁移学习及其他防御方法进行比较，以展示鲁棒性提升。
• 考察骨干网络与头部选择对鲁棒性的影响，并显示固定骨干网络并仅对最后一层再训练可提升性能。

实验结果

研究问题

• RQ1 在少样本设置中，adversarial querying 是否能够产生对强对抗攻击鲁棒的元学习模型？
• RQ2 AQ 是否在不同元学习骨架中具有算法无关性，鲁棒性是否更依赖头部还是骨干？
• RQ3 AQ 相较于对抗性训练和预处理防御在少样本鲁棒性方面有何差异？
• RQ4 仅对最后一层微调对自然精度和鲁棒精度有何影响？
• RQ5 通过 AQ 学得的鲁棒特征提取器在迁移到鲁棒性方面是否比鲁棒训练的迁移学习更具鲁棒性？

主要发现

• AQ-based meta-learners achieve higher robust accuracy and competitive natural accuracy than robust transfer learning on Mini-ImageNet and CIFAR-FS.
• Naturally trained meta-learners are highly vulnerable to 20-step PGD attacks, while AQ significantly improves robustness across multiple backbones.
• Attacking only query data (not support data) during AQ provides robust gains with lower computational cost than perturbing support data.
• Re-training only the final layer during fine-tuning yields better natural and robust accuracy than updating the entire network for AQ.
• The R2-D2 head contributes markedly to robustness, more so than the embedding, across AQ models.
• AQ outperforms adversarially trained transfer learning in robustness across tested datasets and backbones.