[Paper Review] Understanding and Improving Fast Adversarial Training

Maksym Andriushchenko, Nicolas Flammarion|arXiv (Cornell University)|Jul 6, 2020

Adversarial Robustness in Machine Learning47 references45 citations

TL;DR

This paper analyzes why FGSM-based fast adversarial training suffers from catastrophic overfitting and introduces GradAlign, a regularizer that improves robustness and reduces the gap to PGD-based training.

ABSTRACT

A recent line of work focused on making adversarial training computationally efficient for deep learning models. In particular, Wong et al. (2020) showed that $\ell_\infty$-adversarial training with fast gradient sign method (FGSM) can fail due to a phenomenon called "catastrophic overfitting", when the model quickly loses its robustness over a single epoch of training. We show that adding a random step to FGSM, as proposed in Wong et al. (2020), does not prevent catastrophic overfitting, and that randomness is not important per se -- its main role being simply to reduce the magnitude of the perturbation. Moreover, we show that catastrophic overfitting is not inherent to deep and overparametrized networks, but can occur in a single-layer convolutional network with a few filters. In an extreme case, even a single filter can make the network highly non-linear locally, which is the main reason why FGSM training fails. Based on this observation, we propose a new regularization method, GradAlign, that prevents catastrophic overfitting by explicitly maximizing the gradient alignment inside the perturbation set and improves the quality of the FGSM solution. As a result, GradAlign allows to successfully apply FGSM training also for larger $\ell_\infty$-perturbations and reduce the gap to multi-step adversarial training. The code of our experiments is available at https://github.com/tml-epfl/understanding-fast-adv-training.

Motivation & Objective

• Investigate when and why fast adversarial training with FGSM yields robust models or catastrophically overfits.
• Analyze the role of randomness in FGSM-based training and its actual effect on perturbation magnitude.
• Relate catastrophic overfitting to gradient alignment and local linearity in neural networks.
• Propose GradAlign to explicitly maximize gradient alignment within the perturbation set.
• Evaluate GradAlign against other fast and multi-step adversarial training methods across datasets.

Proposed method

• Formulate adversarial training under the l_infty threat model and contrast FGSM, FGSM with random start, and PGD-based approaches.
• Introduce a gradient alignment regularizer GradAlign that minimizes 1 - cos(angle of gradients) between x and x+eta.
• Analyze gradient alignment in single-layer CNNs to illustrate how a single filter can induce nonlinearity and overfitting.
• Provide theoretical bounds showing how random starts reduce expected perturbation length, linking this to linear approximation quality.
• Empirically compare FGSM, FGSM-RS, FGSM+GradAlign, AT for Free, PGD-2, and PGD-10 across CIFAR-10, SVHN, and ImageNet.
• Document training details and evaluation setup using PGD-50-10 robustness as the primary metric.

Experimental results

Research questions

• RQ1 Under what conditions does FGSM-based adversarial training avoid catastrophic overfitting?
• RQ2 Does randomness in FGSM (FGSM-RS) primarily reduce perturbation magnitude, or is there another mechanism at play?
• RQ3 How does gradient alignment within the perturbation set relate to robustness and catastrophic overfitting?
• RQ4 Can a regularizer that maximizes gradient alignment (GradAlign) prevent catastrophic overfitting and improve fast adversarial training without extensive inner maximization?
• RQ5 How do the proposed methods compare to PGD-based adversarial training on standard benchmarks like CIFAR-10, SVHN, and ImageNet?

Key findings

• FGSM and related fast adversarial training methods can exhibit catastrophic overfitting; GradAlign prevents this and closes the robustness gap to PGD-10.
• FGSM-RS does not inherently solve catastrophic overfitting; reducing FGSM step size can achieve similar robustness without randomness.
• A random start decreases expected perturbation length, which improves linear approximation quality and explains partial benefits of FGSM-RS.
• Catastrophic overfitting correlates with a drop in gradient alignment and a misalignment between FGSM and PGD directions.
• GradAlign increases gradient alignment between x and x+eta, enabling FGSM training to achieve robustness for larger l_infty radii, approaching PGD-10 performance.
• GradAlign also improves robustness when combined with PGD-2 and is scalable to ImageNet, though with training slowdown due to double backpropagation.