[Paper Review] Ensemble Adversarial Training: Attacks and Defenses

Florian Tramèr, Alexey Kurakin+4 more|arXiv (Cornell University)|May 19, 2017

Adversarial Robustness in Machine Learning48 references1,107 citations

TL;DR

The paper analyzes why single-step adversarial training fails due to gradient masking and introduces Ensemble Adversarial Training to improve black-box robustness by training with adversarial examples from static pre-trained models.

ABSTRACT

Adversarial examples are perturbed inputs designed to fool machine learning models. Adversarial training injects such examples into training data to increase robustness. To scale this technique to large datasets, perturbations are crafted using fast single-step methods that maximize a linear approximation of the model's loss. We show that this form of adversarial training converges to a degenerate global minimum, wherein small curvature artifacts near the data points obfuscate a linear approximation of the loss. The model thus learns to generate weak perturbations, rather than defend against strong ones. As a result, we find that adversarial training remains vulnerable to black-box attacks, where we transfer perturbations computed on undefended models, as well as to a powerful novel single-step attack that escapes the non-smooth vicinity of the input data via a small random step. We further introduce Ensemble Adversarial Training, a technique that augments training data with perturbations transferred from other models. On ImageNet, Ensemble Adversarial Training yields models with strong robustness to black-box attacks. In particular, our most robust model won the first round of the NIPS 2017 competition on Defenses against Adversarial Attacks. However, subsequent work found that more elaborate black-box attacks could significantly enhance transferability and reduce the accuracy of our models.

Motivation & Objective

• Explain why single-step adversarial training converges to degenerate minima and is vulnerable to black-box attacks.
• Propose Ensemble Adversarial Training to diversify adversarial perturbations seen during training.
• Demonstrate robustness gains on ImageNet and analyze transferability of attacks across models.

Proposed method

• Formulate adversarial training with bounded l_infinity perturbations.
• Demonstrate gradient-masking/degenerate minima under single-step attacks.
• Introduce R+FGSM: a random-perturbation pre-step to single-step attacks.
• Propose Ensemble Adversarial Training by incorporating adversarial examples from static pre-trained models.
• Evaluate on ImageNet with Inception v3 and Inception ResNet v2 against various white-box and black-box attacks.
• Discuss convergence and trade-offs in white-box vs black-box robustness.

Experimental results

Research questions

• RQ1 Can single-step adversarial training create a degenerate minimum that masks the true loss landscape?
• RQ2 Does transferring adversarial perturbations from static models improve robustness to black-box attacks?
• RQ3 How does Ensemble Adversarial Training affect robustness to various attack types across large-scale datasets?

Key findings

• Single-step adversarial training exhibits gradient-masking, reducing the effectiveness of linear approximations to the loss near data points.
• Adversarial training with single-step methods increases white-box robustness but degrades black-box robustness due to transferability.
• A new R+FGSM attack (random start plus FGSM) strengthens single-step attacks across models.
• Ensemble Adversarial Training (training with perturbations from static pre-trained models) improves robustness to black-box attacks on ImageNet.
• Ensemble models show reduced transferability of adversarial perturbations but white-box robustness can be compromised.
• The best ensemble model (IRv2_adv-ens) achieved top performance in NIPS 2017 defense competition and demonstrated notable robustness to black-box attacks at the time.