[Paper Review] Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methods

Maher Nouiehed, Maziar Sanjabi+3 more|arXiv (Cornell University)|Feb 21, 2019

Stochastic Gradient Optimization Techniques43 references37 citations

TL;DR

The paper develops multi-step gradient descent-ascent algorithms to find ε-first-order Nash equilibria in non-convex min-max games, with optimal ε^{-2} rates under PL conditions and ε^{-3.5} rates for non-convex concave cases, plus empirical validation on Fashion-MNIST.

ABSTRACT

Recent applications that arise in machine learning have surged significant interest in solving min-max saddle point games. This problem has been extensively studied in the convex-concave regime for which a global equilibrium solution can be computed efficiently. In this paper, we study the problem in the non-convex regime and show that an \varepsilon--first order stationary point of the game can be computed when one of the player's objective can be optimized to global optimality efficiently. In particular, we first consider the case where the objective of one of the players satisfies the Polyak-Łojasiewicz (PL) condition. For such a game, we show that a simple multi-step gradient descent-ascent algorithm finds an \varepsilon--first order stationary point of the problem in \widetilde{\mathcal{O}}(\varepsilon^{-2}) iterations. Then we show that our framework can also be applied to the case where the objective of the "max-player" is concave. In this case, we propose a multi-step gradient descent-ascent algorithm that finds an \varepsilon--first order stationary point of the game in \widetilde{\cal O}(\varepsilon^{-3.5}) iterations, which is the best known rate in the literature. We applied our algorithm to a fair classification problem of Fashion-MNIST dataset and observed that the proposed algorithm results in smoother training and better generalization.

Motivation & Objective

• Motivate solving min-max saddle point problems in non-convex regimes encountered in ML (e.g., GANs, robust/adversarial learning).
• Define and target first-order Nash equilibria (FNE) as a tractable goal when general Nash equilibria may not exist.
• Develop algorithms with convergence guarantees to ε-FNE under PL and non-convex concave settings.
• Provide empirical validation showing smoother training and improved generalization in a fair classification task.

Proposed method

• Formulate the problem as a two-player zero-sum min-max game and define ε-FNE using first-order conditions (Definitions 2.1 and 2.3).
• Assume smoothness with Lipschitz gradients and consider two settings: PL-condition in one player's objective and concavity in the other.
• For PL-games, propose a Multi-step Gradient Descent-Ascent (GDA) that alternates inner maximization steps with outer descent, grounded in Danskin-like gradient evaluation (Lemma A.5).
• Prove convergence: under Assumptions 2.5 and 3.3, the algorithm achieves an ε-FNE in O(ε^{-2}) iterations with corresponding gradient evaluations (Theorem 3.4 and Corollary 3.5).
• For non-convex concave games, introduce a regularized inner maximization and a two-step framework (Algorithm 2) using Accelerated Projected Gradient Ascent for α and Frank-Wolfe/PGD for θ to obtain an ε-FNE in Õ(ε^{-3.5}) evaluations (Theorem 4.2, Corollary 4.3).

Experimental results

Research questions

• RQ1 Can an ε-first-order Nash equilibrium be efficiently computed in general non-convex non-concave min-max games?
• RQ2 What are the iteration/computation complexities to obtain ε-FNE under PL conditions and under non-convex concave assumptions?
• RQ3 Do iterative first-order methods provide practical benefits (stability, generalization) in ML tasks like fair classification and robust training?
• RQ4 How does regularization of the inner problem affect convergence guarantees and empirical performance in non-convex concave min-max problems?

Key findings

• In PL-games, a multi-step GDA algorithm achieves an ε-FNE with O(ε^{-2}) gradient evaluations in θ and O(ε^{-2} log(ε^{-1})) in α, up to logarithmic factors.
• In non-convex concave games, a two-step framework achieves an ε-FNE with Õ(ε^{-3.5}) total gradient evaluations (overall Õ(ε^{-3.5}) when accounting for inner/outer steps).
• The inner maximization can be differentiated via a gradient of the value function despite non-uniqueness of inner optima (via Lemma A.5 and related results).
• Empirical applications include a fair classification task on Fashion-MNIST showing smoother training and improved generalization under the proposed framework.
• Regularized inner maximization in the non-convex concave setting aids convergence and improves worst-case performance in stochastic training, as demonstrated in experiments.