[Paper Review] Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack

Francesco Croce, Matthias Hein|arXiv (Cornell University)|Jul 3, 2019

Adversarial Robustness in Machine Learning29 references121 citations

TL;DR

Introduces FAB-attack, a white-box method that efficiently finds minimally distorted adversarial examples in l1, l2, and linf norms, with scaling-invariant properties and strong robustness against gradient masking.

ABSTRACT

The evaluation of robustness against adversarial manipulation of neural networks-based classifiers is mainly tested with empirical attacks as methods for the exact computation, even when available, do not scale to large networks. We propose in this paper a new white-box adversarial attack wrt the $l_p$-norms for $p \in \{1,2,\infty\}$ aiming at finding the minimal perturbation necessary to change the class of a given input. It has an intuitive geometric meaning, yields quickly high quality results, minimizes the size of the perturbation (so that it returns the robust accuracy at every threshold with a single run). It performs better or similar to state-of-the-art attacks which are partially specialized to one $l_p$-norm, and is robust to the phenomenon of gradient masking.

Motivation & Objective

• Motivate robust evaluation of classifier robustness against adversarial perturbations with minimal perturbations.
• Develop a white-box attack that yields minimal l_p-norm perturbations across p in {1,2,∞}.
• Provide an almost off-the-shelf, scalable attack that avoids extensive hyperparameter tuning.
• Ensure the attack is robust to gradient masking and scaling issues common in other attacks.

Proposed method

• Define minimal adversarial perturbations with respect to l_p norms and box constraints.
• Use projections onto the intersection of a decision hyperplane and a box (C) to compute exact p-norm projections (proj_p).
• Iteratively bias steps toward the original input by convex combination of projections (x(i) and x_orig) on the target hyperplane (pi_s) as per equation (10).
• Compute adaptive step bias alpha from relative distances to the decision hyperplane (Equation 9).
• Incorporate an extrapolation step (eta) and a final search to tighten perturbation size (Equations 12-13).
• Optionally apply random restarts to explore multiple starting points and improve results.

Experimental results

Research questions

• RQ1 Can a single, scalable white-box attack achieve minimal adversarial perturbations across l1, l2, and linf norms without per-dataset tuning?
• RQ2 Does projecting onto hyperplanes with box constraints and biasing steps toward the original input yield smaller perturbations while maintaining attack success across different networks and defenses?
• RQ3 Is the FAB-attack robust to gradient masking and scaling of the classifier?
• RQ4 What is the empirical performance of FAB compared to state-of-the-art attacks across MNIST, CIFAR-10, and Restricted ImageNet under multiple norms?

Key findings

• FAB-attack often outperforms or matches state-of-the-art attacks across l1, l2, and linf norms on MNIST and CIFAR-10.
• FAB-attack remains effective under gradient masking and is invariant to scaling of the classifier, unlike PGD.
• With modest restarts, FAB-attack achieves high-quality adversarial examples while using fewer hyperparameters and no step-size tuning.
• On Restricted ImageNet, FAB-attack yields best results for l1 and competitive results for linf and l2 compared to strong baselines.
• FAB-attack often requires fewer iterations with competitive or better robustness metrics than competitors, and a low-budget variant (FAB-10) remains strong.
• Across norms, FAB generally achieves the best or near-best average robust accuracy and small average/max difference to the best attack.