[Paper Review] Attacks Which Do Not Kill Training Make Adversarial Learning Stronger

Jingfeng Zhang, Xilie Xu+5 more|arXiv (Cornell University)|Feb 26, 2020

Adversarial Robustness in Machine Learning50 references84 citations

TL;DR

Proposes Friendly Adversarial Training (FAT), which uses friendly adversarial data (least adversarial within misclassified samples) via early-stopped PGD to improve robustness without sacrificing natural accuracy. Provides theoretical upper bound and empirical evidence that robustness can be achieved without harming standard generalization.

ABSTRACT

Adversarial training based on the minimax formulation is necessary for obtaining adversarial robustness of trained models. However, it is conservative or even pessimistic so that it sometimes hurts the natural generalization. In this paper, we raise a fundamental question---do we have to trade off natural generalization for adversarial robustness? We argue that adversarial training is to employ confident adversarial data for updating the current model. We propose a novel approach of friendly adversarial training (FAT): rather than employing most adversarial data maximizing the loss, we search for least adversarial (i.e., friendly adversarial) data minimizing the loss, among the adversarial data that are confidently misclassified. Our novel formulation is easy to implement by just stopping the most adversarial data searching algorithms such as PGD (projected gradient descent) early, which we call early-stopped PGD. Theoretically, FAT is justified by an upper bound of the adversarial risk. Empirically, early-stopped PGD allows us to answer the earlier question negatively---adversarial robustness can indeed be achieved without compromising the natural generalization.

Motivation & Objective

• Question the necessity of trading off natural generalization for adversarial robustness in standard adversarial training.
• Introduce friendly adversarial training (FAT) that uses least adversarial, confidently misclassified data to update models.
• Provide theoretical justification via an upper bound on adversarial risk for FAT.
• Show that early-stopped PGD can implement FAT efficiently and improve both standard and robust accuracy.
• Demonstrate that FAT enables larger perturbation budgets (epsilon) during training while maintaining or improving performance.

Proposed method

• Formulate FAT as minimizing loss over friendly adversarial data, defined by misclassified adversarial samples with a margin constraint rho.
• Replace the inner maximization in standard adversarial training with a constrained minimization over adversarial samples that satisfy a confidence margin and minimize the loss.
• Develop PGD-K-τ, an early-stopped PGD algorithm that selects misclassified samples with small loss (or correctly classified samples with large loss) controlled by τ, generalizing conventional PGD-K.
• Prove a tight upper bound on adversarial risk that incorporates both standard and robust terms and uses a margin parameter ρ.
• Provide a practical FAT algorithm that can adapt existing defenses (e.g., TRADES, MART) into FAT variants.
• Empirically validate FAT on CIFAR-10 and SVHN with ResNet-18, Small CNN, and Wide ResNet architectures, comparing standard and robust accuracies under various attacks.

Experimental results

Research questions

• RQ1 Can adversarial robustness be achieved without sacrificing natural generalization in adversarial training?
• RQ2 Does using friendly adversarial data (within misclassified samples) improve training stability and generalization?
• RQ3 How does early-stopped PGD (PGD-K-τ) influence the training dynamics and robustness of the model?
• RQ4 What theoretical guarantees can justify FAT as an upper-bound approach to adversarial risk?
• RQ5 Is FAT compatible with existing adversarial training methods (TRADES, MART) to yield improved performance?

Key findings

• FAT improves standard (natural) test accuracy while maintaining competitive robust accuracy across attacks.
• Early-stopped PGD (PGD-K-τ) alleviates cross-over mixture of adversarial data and enables progressive strengthening of robustness during training.
• FAT enables larger training perturbation budgets ε_train without harming generalization, unlike conventional adversarial training.
• Theoretical upper bound on adversarial risk shows FAT can reduce risk by combining misclassified adversarial data with confidence margins.
• FAT variants can be derived from existing methods (e.g., FAT for TRADES, FAT for MART), offering practical paths for deployment.
• Empirical results indicate robustness can be enhanced with controlled τ values (e.g., τ in {0,1,2,3}), balancing standard and robust performance.