[Paper Review] Adversarial Machine Learning at Scale

Alexey Kurakin, Ian Goodfellow+1 more|arXiv (Cornell University)|Nov 4, 2016

Adversarial Robustness in Machine Learning375 citations

TL;DR

This paper demonstrates scalable adversarial training on ImageNet using Inception v3, showing improved robustness to one-step adversarial attacks, discusses transferability, model capacity effects, and a label leaking phenomenon.

ABSTRACT

Adversarial examples are malicious inputs designed to fool machine learning models. They often transfer from one model to another, allowing attackers to mount black box attacks without knowledge of the target model's parameters. Adversarial training is the process of explicitly training a model on adversarial examples, in order to make it more robust to attack or to reduce its test error on clean inputs. So far, adversarial training has primarily been applied to small problems. In this research, we apply adversarial training to ImageNet. Our contributions include: (1) recommendations for how to succesfully scale adversarial training to large models and datasets, (2) the observation that adversarial training confers robustness to single-step attack methods, (3) the finding that multi-step attack methods are somewhat less transferable than single-step attack methods, so single-step attacks are the best for mounting black-box attacks, and (4) resolution of a "label leaking" effect that causes adversarially trained models to perform better on adversarial examples than on clean examples, because the adversarial example construction process uses the true label and the model can learn to exploit regularities in the construction process.

Motivation & Objective

• Demonstrate scalable adversarial training for large-scale models and datasets (ImageNet) using batch normalization and mixed adversarial/clean minibatches.
• Assess robustness of trained models to different adversarial attack methods, especially one-step vs multi-step attacks.
• Investigate how model capacity and training choices influence robustness to adversarial perturbations.
• Identify transferability of adversarial examples across models and implications for black-box attacks.
• Expose and analyze the label leaking effect in adversarial training scenarios.

Proposed method

• Review and compare multiple adversarial example generation methods (one-step and iterative).
• Propose adversarial training algorithm that injects adversarial examples into each minibatch with a controllable loss weighting parameter lambda.
• Use randomized epsilon per example to avoid overfitting to a fixed perturbation size.
• Employ batch normalization and a mixed minibatch containing both clean and adversarial examples for stable large-scale training.
• Evaluate with Inception v3 on ImageNet, using RMSProp and synchronized distributed training across 50 machines.

Experimental results

Research questions

• RQ1 How can adversarial training be scaled to large models and datasets like ImageNet?
• RQ2 Does adversarial training with one-step attacks provide robustness to other one-step and some multi-step attacks?
• RQ3 How does model capacity affect adversarial robustness, with and without adversarial training?
• RQ4 What is the transferability of adversarial examples between models, and how does attack type influence it?
• RQ5 Is there a label leaking phenomenon in adversarial training, and how should attacks be constructed for robust evaluation?

Key findings

• Adversarial training with one-step methods increases robustness to those one-step attacks, achieving up to about 74% top-1 accuracy on adversarial examples while losing roughly 0.8% on clean accuracy.
• Increasing model capacity (deeper/increased width) improves robustness when paired with adversarial training.
• Iterative adversarial examples remain largely resistant to robustness gained from one-step adversarial training, indicating limited cross-protection against multi-step attacks.
• Transferability is higher for FGSM-style adversarial examples, while iterative one-step methods transfer less, suggesting a potential security-benefit for black-box attacks.
• A label leaking effect was observed when true labels were used in one-step adversarial construction, leading to higher accuracy on adversarial examples than on clean ones; the effect disappears when not using the true label or when using iterative methods.