[Paper Review] The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization

Yifei Min, Lin Chen+1 more|arXiv (Cornell University)|Feb 25, 2020

Adversarial Robustness in Machine LearningComputer Science43 references30 citations

TL;DR

The paper shows that increasing training data can hurt, help, or cause a double-descent in generalization for adversarially robust models, depending on adversary strength and loss function.

ABSTRACT

Adversarial training has shown its ability in producing models that are robust to perturbations on the input data, but usually at the expense of decrease in the standard accuracy. To mitigate this issue, it is commonly believed that more training data will eventually help such adversarially robust models generalize better on the benign/unperturbed test data. In this paper, however, we challenge this conventional belief and show that more training data can hurt the generalization of adversarially robust models in the classification problems. We first investigate the Gaussian mixture classification with a linear loss and identify three regimes based on the strength of the adversary. In the weak adversary regime, more data improves the generalization of adversarially robust models. In the medium adversary regime, with more training data, the generalization loss exhibits a double descent curve, which implies the existence of an intermediate stage where more training data hurts the generalization. In the strong adversary regime, more data almost immediately causes the generalization error to increase. Then we move to the analysis of a two-dimensional classification problem with a 0-1 loss. We prove that more data always hurts the generalization performance of adversarially trained models with large perturbations. To complement our theoretical results, we conduct empirical studies on Gaussian mixture classification, support vector machines (SVMs), and linear regression.

Motivation & Objective

• Motivate and analyze how adversarial training impacts generalization on unperturbed test data.
• Identify regimes (weak, medium, strong adversary) where more data either helps, hurts, or yields double descent in generalization.
• Characterize differences across loss functions (linear and 0-1) and model settings (Gaussian mixtures, Manhattan model).
• Provide theoretical results complemented by empirical studies across multiple models (SVMs, linear regression, Gaussian mixtures).

Proposed method

• Define robust classifier with adversarial perturbations in an infinity-norm ball and analyze generalization error L_n.
• Study Gaussian mixture classification with linear loss to establish three adversary regimes and phase transitions (Theorem 1).
• Introduce the Manhattan model to analyze 0-1 loss and prove weak/strong regime behavior (Theorem 4).
• Perform empirical experiments on Gaussian mixtures with 0-1 loss, SVMs, and linear regression to observe data-size effects.
• Use corollaries to illustrate special-case behavior (e.g., equal means and variances).

Experimental results

Research questions

• RQ1 Under what conditions does more training data improve generalization for adversarially robust models?
• RQ2 Can adversarial robustness lead to double-descent phenomena as data size grows?
• RQ3 How do different loss functions (linear vs 0-1) and problem setups affect the data-size generalization behavior under adversarial training?
• RQ4 Do empirical trends observed in Gaussian mixtures extend to SVMs and linear regression?
• RQ5 What thresholds in adversary strength delineate weak, medium, and strong regimes?

Key findings

• Three adversary regimes exist: weak adversary, where more data always improves generalization; medium adversary, where a double-descent pattern can occur; strong adversary, where more data can worsen generalization.
• In the Gaussian mixture with linear loss, there exist thresholds δ1, δ2 and regimes with monotone improvement, double descent, or monotone deterioration as data grows (Theorem 1).
• Under a 0-1 loss in the Manhattan model, robust classification can have zero generalization error for small perturbations (ε<2μ) and increasing generalization error with data for larger perturbations (Theorem 4).
• Empirical results with Gaussian mixtures, SVMs, and linear regression show trends consistent with the theoretical weak/strong regime behavior, including instances of increasing test loss with more data under stronger adversaries.
• The findings suggest that adversarial training may not always benefit unperturbed generalization and that current frameworks may require new ideas to achieve dual robustness and generalization.