[Paper Review] Adversarial Training Should Be Cast as a Non-Zero-Sum Game

Alexander Robey, Fabian Latorre+3 more|arXiv (Cornell University)|Jun 19, 2023

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper proposes a non-zero-sum bilevel formulation for adversarial training, where the defender minimizes an upper bound on classification error and the adversary maximizes a continuous reformulation of the 0-1 loss. The resulting BETA-AT algorithm achieves state-of-the-art robustness on CIFAR-10 without heuristic techniques and eliminates robust overfitting by avoiding surrogate-based relaxations that invalidate robustness guarantees.

ABSTRACT

One prominent approach toward resolving the adversarial vulnerability of deep neural networks is the two-player zero-sum paradigm of adversarial training, in which predictors are trained against adversarially chosen perturbations of data. Despite the promise of this approach, algorithms based on this paradigm have not engendered sufficient levels of robustness and suffer from pathological behavior like robust overfitting. To understand this shortcoming, we first show that the commonly used surrogate-based relaxation used in adversarial training algorithms voids all guarantees on the robustness of trained classifiers. The identification of this pitfall informs a novel non-zero-sum bilevel formulation of adversarial training, wherein each player optimizes a different objective function. Our formulation yields a simple algorithmic framework that matches and in some cases outperforms state-of-the-art attacks, attains comparable levels of robustness to standard adversarial training algorithms, and does not suffer from robust overfitting.

Motivation & Objective

• To identify the fundamental flaw in standard surrogate-based adversarial training, which voids robustness guarantees due to improper relaxation of the 0-1 loss.
• To reformulate adversarial training as a bilevel non-zero-sum game where defender and adversary optimize distinct objectives, preserving robustness guarantees.
• To develop a heuristic-free adversarial training algorithm (BETA-AT) that matches or exceeds state-of-the-art robustness on CIFAR-10.
• To demonstrate that robust overfitting is an artifact of surrogate-based zero-sum formulations, not a fundamental limitation of robust generalization.
• To show that a properly formulated adversarial objective enables accurate robustness estimation without complex attack heuristics like multiple restarts or adaptive learning rates.

Proposed method

• Derive a bilevel optimization formulation where the defender minimizes an upper bound on the 0-1 classification error, and the adversary maximizes a continuous relaxation of the same error.
• Use a continuous reformulation of the 0-1 loss as the adversary's objective, ensuring that maximizing it corresponds to finding true adversarial examples.
• Propose BETA (BEst Adversarial Attack), a new attack algorithm based on the adversary’s objective, which estimates robustness accurately without heuristics.
• Develop BETA-AT, a training algorithm based on the bilevel formulation that avoids surrogate-based relaxations and does not require ad hoc techniques like multiple restarts or adaptive learning rates.
• Implement the algorithm using standard first-order optimization, enabling practical deployment without complex hyperparameter tuning.
• Use early stopping for model selection, showing that BETA-AT maintains strong robustness without heuristic stopping conditions.

Experimental results

Research questions

• RQ1 Does the use of surrogate losses in standard adversarial training invalidate robustness guarantees by weakening the adversary?
• RQ2 Can a bilevel non-zero-sum formulation of adversarial training preserve robustness guarantees while remaining tractable?
• RQ3 Does eliminating surrogate-based relaxations in adversarial training eliminate robust overfitting as an artifact of flawed optimization?
• RQ4 Can a properly formulated adversarial objective achieve robustness comparable to AutoAttack without complex heuristics?
• RQ5 Is the proposed bilevel framework generalizable beyond standard adversarial training to other distribution shift settings?

Key findings

• The surrogate-based relaxation commonly used in adversarial training voids all guarantees on robustness, leading to weak adversaries and ineffective training.
• The proposed bilevel non-zero-sum formulation ensures that maximizing the adversary’s objective corresponds to finding the true optimal adversarial examples, preserving robustness guarantees.
• BETA-AT achieves test robustness on CIFAR-10 that matches or exceeds state-of-the-art methods like AutoAttack, despite not using heuristics such as multiple restarts or adaptive learning rates.
• The algorithm eliminates robust overfitting entirely, suggesting it is a consequence of improper surrogate relaxation, not a fundamental trade-off between accuracy and robustness.
• BETA provides robustness estimates nearly identical to AutoAttack, demonstrating that accurate robustness evaluation does not require complex attack heuristics when the objective is correctly aligned.
• The bilevel formulation is generalizable and provides a principled foundation for future work in robust machine learning, including convergence and sample complexity analysis.