[Paper Review] RobustBench: a standardized adversarial robustness benchmark

Francesco Croce, Maksym Andriushchenko+6 more|arXiv (Cornell University)|Oct 19, 2020

Adversarial Robustness in Machine Learning135 references116 citations

TL;DR

RobustBench establishes standardized, reproducible adversarial robustness evaluation using AutoAttack, a leader-board, and a Model Zoo to compare defenses and analyze robustness across distributions and related factors.

ABSTRACT

As a research community, we are still lacking a systematic understanding of the progress on adversarial robustness which often makes it hard to identify the most promising ideas in training robust models. A key challenge in benchmarking robustness is that its evaluation is often error-prone leading to robustness overestimation. Our goal is to establish a standardized benchmark of adversarial robustness, which as accurately as possible reflects the robustness of the considered models within a reasonable computational budget. To this end, we start by considering the image classification task and introduce restrictions (possibly loosened in the future) on the allowed models. We evaluate adversarial robustness with AutoAttack, an ensemble of white- and black-box attacks, which was recently shown in a large-scale study to improve almost all robustness evaluations compared to the original publications. To prevent overadaptation of new defenses to AutoAttack, we welcome external evaluations based on adaptive attacks, especially where AutoAttack flags a potential overestimation of robustness. Our leaderboard, hosted at https://robustbench.github.io/, contains evaluations of 120+ models and aims at reflecting the current state of the art in image classification on a set of well-defined tasks in $\ell_\infty$- and $\ell_2$-threat models and on common corruptions, with possible extensions in the future. Additionally, we open-source the library https://github.com/RobustBench/robustbench that provides unified access to 80+ robust models to facilitate their downstream applications. Finally, based on the collected models, we analyze the impact of robustness on the performance on distribution shifts, calibration, out-of-distribution detection, fairness, privacy leakage, smoothness, and transferability.

Motivation & Objective

• Define a standardized, reliable evaluation protocol for adversarial robustness under common threat models.
• Provide an up-to-date, public leaderboard to track progress in robust image classification.
• Open-source a Model Zoo of robust models to facilitate downstream use and fair comparisons.
• Assess how robustness interacts with distribution shifts, calibration, OOD detection, fairness, and privacy leakage.

Proposed method

• Use AutoAttack as the current standard evaluation for l_infinity and l_2 threat models on CIFAR-10, CIFAR-100, and ImageNet.
• Impose restrictions on submitted models to ensure reliable evaluation (non-zero input gradients, deterministic forward pass, no forward-time optimization loops).
• Provide external adaptive evaluations to flag potential robustness overestimation and encourage further testing.
• Maintain a public leaderboard (robustbench.github.io) with 120+ model evaluations and a Model Zoo with 80+ robust models.
• Open-source a unified library to benchmark models and enable easy downstream usage of robust models.

Experimental results

Research questions

• RQ1 What constitutes a reliable, standardized evaluation of adversarial robustness across common threat models?
• RQ2 How does robustness under l_infinity and l_2 perturbations relate to calibration, distribution shifts, and other properties like OOD detection and privacy leakage?
• RQ3 Can a publicly maintained leaderboard and model zoo accelerate progress and fair comparisons in adversarial robustness research?

Key findings

• Many previously reported robust accuracies are overestimated when evaluated with suboptimal attacks; standardized AutoAttack provides tighter upper bounds.
• Robust models tend to be underconfident and require calibration (temperature scaling improves ECE substantially, but gaps remain).
• Robust training can degrade OOD detection quality and fairness across classes, though effects vary by method and threat model.
• Extra training data helps mitigate robustness-accuracy trade-offs, but robustness remains associated with some performance degradation on clean accuracy.
• Adversarial examples transfer preferentially among robust-robust models and robust-to-robust, but less so from robust to non-robust models, with model smoothness correlating with robustness.