[Paper Review] Adversarial Training Can Hurt Generalization

Aditi Raghunathan, Sang Michael Xie+3 more|arXiv (Cornell University)|Jun 14, 2019

Adversarial Robustness in Machine Learning21 references144 citations

TL;DR

The paper shows that adversarial training can harm standard generalization even when the optimal predictor is good for both standard and robust accuracy, via a convex construction, and demonstrates that robust self-training with unlabeled data largely alleviates this tradeoff.

ABSTRACT

While adversarial training can improve robust accuracy (against an adversary), it sometimes hurts standard accuracy (when there is no adversary). Previous work has studied this tradeoff between standard and robust accuracy, but only in the setting where no predictor performs well on both objectives in the infinite data limit. In this paper, we show that even when the optimal predictor with infinite data performs well on both objectives, a tradeoff can still manifest itself with finite data. Furthermore, since our construction is based on a convex learning problem, we rule out optimization concerns, thus laying bare a fundamental tension between robustness and generalization. Finally, we show that robust self-training mostly eliminates this tradeoff by leveraging unlabeled data.

Motivation & Objective

• Demonstrate a finite-data tradeoff between standard and robust accuracy under adversarial training in a convex setting.
• Isolate optimization from statistics to show a fundamental tension between robustness and generalization.
• Show that robust self-training with unlabeled data can eliminate the tradeoff observed with finite data.

Proposed method

• Construct a convex learning problem with a staircase-like optimal predictor that is robust and accurate.
• Define invariance sets B(x) and analyze standard vs robustness-focused objective with squared loss.
• Prove consistency of standard and robust estimators under target invariance (infinite data).
• Use a finite-sample construction to show robust training incurs higher test error than standard training for small samples.
• Subsample CIFAR-10 to observe how the accuracy gap shrinks with more data, mirroring the construction.
• Demonstrate robust self-training on unlabeled data mitigates the tradeoff.

Experimental results

Research questions

• RQ1 Can adversarial training harm standard generalization even when the Bayes predictor is robust and accurate?
• RQ2 Does a finite-sample regime reveal a tradeoff between standard and robust objectives independent of optimization issues?
• RQ3 Can unlabeled data via robust self-training close or remove the standard-robust performance gap?
• RQ4 Under what conditions does robustness act as a regularizer versus a source of overfitting in small data scenarios?

Key findings

• A convex construction shows adversarial training can worsen standard generalization at finite samples even when the optimal predictor is good for both objectives.
• The gap between robust and standard test errors decreases as the training sample size increases, suggesting more data mitigates the tradeoff.
• Robust self-training with additional unlabeled data largely eliminates the tradeoff, achieving comparable performance to standard training while remaining robust.
• On CIFAR-10 and MNIST-like settings, the observed tradeoff aligns with the staircase construction, and larger data or unlabeled data reduces or removes it.
• The effect of robustness depends on the complexity of the robust predictor; when the robust predictor is simple (e.g., slope m=0), adversarial training can improve standard generalization.