[Paper Review] Adversarial Training and Robustness for Multiple Perturbations

Florian Tramèr, Dan Boneh|arXiv (Cornell University)|Apr 30, 2019

Adversarial Robustness in Machine LearningComputer Science39 references84 citations

TL;DR

Analyzes robustness trade-offs across multiple perturbation types and proposes multi-perturbation adversarial training and new attacks; shows that robust performance against several perturbations cannot be achieved as well as single-perturbation robustness, with gradient-masking observed on MNIST.

ABSTRACT

Defenses against adversarial examples, such as adversarial training, are typically tailored to a single perturbation type (e.g., small $\ell_\infty$-noise). For other perturbations, these defenses offer no guarantees and, at times, even increase the model's vulnerability. Our aim is to understand the reasons underlying this robustness trade-off, and to train models that are simultaneously robust to multiple perturbation types. We prove that a trade-off in robustness to different types of $\ell_p$-bounded and spatial perturbations must exist in a natural and simple statistical setting. We corroborate our formal analysis by demonstrating similar robustness trade-offs on MNIST and CIFAR10. Building upon new multi-perturbation adversarial training schemes, and a novel efficient attack for finding $\ell_1$-bounded adversarial examples, we show that no model trained against multiple attacks achieves robustness competitive with that of models trained on each attack individually. In particular, we uncover a pernicious gradient-masking phenomenon on MNIST, which causes adversarial training with first-order $\ell_\infty, \ell_1$ and $\ell_2$ adversaries to achieve merely $50\%$ accuracy. Our results question the viability and computational scalability of extending adversarial robustness, and adversarial training, to multiple perturbation types.

Motivation & Objective

• Understand why robustness to one perturbation type often reduces robustness to others (MEPs).
• Develop training schemes to achieve simultaneous robustness to multiple perturbation types.
• Design efficient attacks (including for l1) to evaluate multi-perturbation defenses.
• Demonstrate trade-offs on MNIST and CIFAR-10 and analyze gradient-masking effects.

Proposed method

• Define adversarial risk under multiple perturbation sets S1,...,Sn and two natural metrics: Avg and Max adversarial risk.
• Prove theoretical trade-offs (MEPs) between l-infinity, l1, l2, and spatial perturbations.
• Propose multi-perturbation adversarial training strategies (Max and Avg) that use adversarial examples from multiple perturbation types.
• Introduce Sparse L1 Descent (SLIDE), an efficient l1-attack suitable for adversarial training.
• Develop and evaluate affine perturbation analysis to understand composite perturbations.
• Empirically evaluate on MNIST and CIFAR-10 using MNIST CNN and CIFAR-10 Wide-ResNet.

Experimental results

Research questions

• RQ1 Can a model be simultaneously robust to multiple perturbation types (e.g., l-infinity, l1, l2, and spatial perturbations)?
• RQ2 What are the theoretical limits of multi-perturbation robustness in natural statistical models?
• RQ3 Do multi-perturbation training strategies (Max/Avg) improve robustness across perturbation types, and at what cost?
• RQ4 How do affine combinations of perturbations affect robustness compared to unions of perturbations?
• RQ5 Are current adversarial training methods affected by gradient-masking when extended to multiple perturbations?

Key findings

• Robustness to multiple perturbations incurs accuracy costs (typically 5-10 percentage points) compared to single-perturbation training.
• In MNIST, l1, l2, and l-infinity robustness can exhibit gradient-masking, reducing effectiveness of first-order attacks.
• Models trained on multiple perturbations (Avg/Max strategies) improve multi-perturbation robustness but do not reach optimal multi-perturbation performance (OPT) and show trade-offs.
• Affine combinations of perturbations can be stronger than either perturbation alone, and robustness to a union of perturbations may be insufficient against affine adversaries.
• SLIDE attack provides an efficient l1 adversary competitive with stronger attacks, enabling practical multi-perturbation training.
• On CIFAR-10, Adv_avg and Adv_max improve multi-perturbation robustness but still fall short of optimal composite-perturbation robustness.