[論文レビュー] On Evaluating Adversarial Robustness

Nicholas Carlini, Anish Athalye+7 more|arXiv (Cornell University)|Feb 18, 2019

Adversarial Robustness in Machine Learning参考文献 73被引用数 580

ひとこと要約

This paper provides methodological guidance for evaluating defenses against adversarial examples, emphasizing threat models, adaptive attacks, reproducibility, and a structured evaluation checklist to avoid common pitfalls.

ABSTRACT

Correctly evaluating defenses against adversarial examples has proven to be extremely difficult. Despite the significant amount of recent work attempting to design defenses that withstand adaptive attacks, few have succeeded; most papers that propose defenses are quickly shown to be incorrect. We believe a large contributing factor is the difficulty of performing security evaluations. In this paper, we discuss the methodological foundations, review commonly accepted best practices, and suggest new methods for evaluating defenses to adversarial examples. We hope that both researchers developing defenses as well as readers and reviewers who wish to understand the completeness of an evaluation consider our advice in order to avoid common pitfalls.

研究の動機と目的

• Motivate why evaluating defenses against adversarial examples is critical for security and robustness.
• Define principled methodologies for defense evaluations grounded in realistic threat models.
• Offer a comprehensive, actionable checklist to avoid common evaluation pitfalls.

提案手法

• Define threat models including adversary goals, capabilities, and knowledge to guide evaluation.
• Advocate for adaptive adversaries and end-to-end defense testing under the stated threat model.
• Recommend reproducible research practices, including releasing code and pre-trained models.
• Provide a structured evaluation checklist with common severe flaws and pitfalls to audit defenses.
• Suggest prioritizing strong, adaptive, and varied attacks to genuinely test defenses.

実験結果

リサーチクエスチョン

• RQ1 What constitutes a rigorous threat model for adversarial robustness evaluations?
• RQ2 How should defenses be tested against adaptive adversaries to ensure robustness claims hold under realistic conditions?
• RQ3 What best practices and reproducibility standards should accompany defense evaluations?
• RQ4 What common evaluation flaws most frequently undermine robustness claims, and how can they be avoided?

主な発見

• Adaptive attacks that are tailored to the defense are essential to validate robustness claims.
• White-box evaluations should assume complete defender knowledge; secrecy undermines falsifiability.
• Releasing source code and pre-trained models greatly improves the reliability of evaluations.
• A structured, living evaluation checklist helps identify and prevent common flaws in defense assessments.
• Evaluations should report both clean accuracy and robustness under attack, including diverse attack strategies and hyperparameters.