[Paper Review] Towards A Unified Min-Max Framework for Adversarial Exploration and Robustness

Jingkang Wang, Tianyun Zhang+5 more|arXiv (Cornell University)|Sep 25, 2019

Adversarial Robustness in Machine Learning36 references21 citations

TL;DR

This paper proposes a unified min-max optimization framework for adversarial exploration and robustness, extending beyond standard adversarial training to handle diverse problems like ensemble attacks, universal perturbations, and generalized robustness. By learning adaptive domain weights, the method improves performance over averaging strategies and provides interpretable insights into attack and defense difficulty.

ABSTRACT

The worst-case training principle that minimizes the maximal adversarial loss, also known as adversarial training (AT), has shown to be a state-of-the-art approach for enhancing adversarial robustness against norm-ball bounded input perturbations. Nonetheless, min-max optimization beyond the purpose of AT has not been rigorously explored in the research of adversarial attack and defense. In particular, given a set of risk sources (domains), minimizing the maximal loss induced from the domain set can be reformulated as a general min-max problem that is different from AT. Examples of this general formulation include attacking model ensembles, devising universal perturbation under multiple inputs or data transformations, and generalized AT over different types of attack models. We show that these problems can be solved under a unified and theoretically principled min-max optimization framework. We also show that the self-adjusted domain weights learned from our method provides a means to explain the difficulty level of attack and defense over multiple domains. Extensive experiments show that our approach leads to substantial performance improvement over the conventional averaging strategy.

Motivation & Objective

• To address the lack of a principled, unified framework for min-max optimization beyond standard adversarial training.
• To generalize min-max formulations for diverse adversarial tasks such as attacking model ensembles and generating universal perturbations.
• To develop a method that learns self-adjusted domain weights to explain attack and defense difficulty across multiple domains.
• To improve performance over conventional averaging strategies in multi-domain adversarial settings.

Proposed method

• Formulate adversarial robustness and exploration as a general min-max optimization problem over a set of risk sources (domains).
• Introduce a unified framework that subsumes standard adversarial training, ensemble attacks, and universal perturbation generation under one optimization structure.
• Learn domain-specific weights end-to-end via optimization, allowing the model to self-adjust based on the difficulty of each domain.
• Use a min-max objective where the inner maximization finds the worst-case perturbation across domains, and the outer minimization improves model robustness.
• Apply the framework to diverse tasks, including multi-input, multi-transformation, and multi-attack model robustness.
• Demonstrate that learned domain weights correlate with intrinsic attack and defense difficulty, enabling interpretability.

Experimental results

Research questions

• RQ1 Can a single min-max framework unify diverse adversarial tasks such as robustness training, ensemble attacks, and universal perturbation generation?
• RQ2 How do self-adjusted domain weights learned in the framework reflect the relative difficulty of attack and defense across different domains?
• RQ3 Does the proposed framework outperform conventional averaging strategies in multi-domain adversarial settings?
• RQ4 To what extent can the learned domain weights provide interpretable insights into the difficulty of adversarial examples?
• RQ5 Can the framework be applied effectively to generalized adversarial training over heterogeneous attack models?

Key findings

• The proposed unified min-max framework achieves substantial performance gains over conventional averaging strategies in multi-domain adversarial tasks.
• Self-adjusted domain weights learned by the method correlate with the intrinsic difficulty of attack and defense, offering interpretability into adversarial robustness.
• The framework generalizes to various adversarial scenarios, including ensemble attacks and universal perturbations, with consistent improvements.
• The method enables a principled approach to robustness that extends beyond standard adversarial training.
• Empirical results show that the framework outperforms existing methods in both robustness and exploration across diverse domain configurations.