[Paper Review] What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization?

Chi Jin, Praneeth Netrapalli+1 more|arXiv (Cornell University)|Feb 2, 2019

Stochastic Gradient Optimization TechniquesComputer Science33 references74 citations

TL;DR

The paper defines local minimax as a local surrogate for global minimax in sequential minimax games, analyzes its properties and existence, and links it to the limit behavior of gradient descent ascent (GDA) dynamics.

ABSTRACT

Minimax optimization has found extensive applications in modern machine learning, in settings such as generative adversarial networks (GANs), adversarial training and multi-agent reinforcement learning. As most of these applications involve continuous nonconvex-nonconcave formulations, a very basic question arises---"what is a proper definition of local optima?" Most previous work answers this question using classical notions of equilibria from simultaneous games, where the min-player and the max-player act simultaneously. In contrast, most applications in machine learning, including GANs and adversarial training, correspond to sequential games, where the order of which player acts first is crucial (since minimax is in general not equal to maximin due to the nonconvex-nonconcave nature of the problems). The main contribution of this paper is to propose a proper mathematical definition of local optimality for this sequential setting---local minimax, as well as to present its properties and existence results. Finally, we establish a strong connection to a basic local search algorithm---gradient descent ascent (GDA): under mild conditions, all stable limit points of GDA are exactly local minimax points up to some degenerate points.

Motivation & Objective

• Clarify differences between global/local optimality notions in minimax settings, especially for sequential games.
• Introduce and formalize local minimax as a local surrogate for global minimax points.
• Establish first- and second-order conditions for local minimax points.
• Prove existence/non-existence results for local minimax points under various regularity assumptions.
• Connect local minimax to the asymptotic behavior and stability of gradient descent ascent (GDA) dynamics.

Proposed method

• Propose a formal definition of local minimax for two-player sequential minimax problems.
• Derive first-order and second-order necessary/sufficient conditions for local minimax points.
• Analyze existence results and provide conditions under which local minimax points exist (e.g., strong concavity in y).
• Examine the relationship between local minimax points and stable limit points of gamma-GDA (gradient descent ascent with step-size ratio).
• Use limit-flow analysis of GDA to relate stable fixed points to local minimax points.
• Discuss special cases and provide a framework for max-oracle scenarios (Appendix 4).

Experimental results

Research questions

• RQ1 What is an appropriate local optimality concept for sequential minimax problems in nonconvex-nonconcave settings?
• RQ2 Under what conditions do local minimax points exist and how can they be characterized?
• RQ3 How do local minimax points relate to the asymptotic behavior and stability of gradient descent ascent (GDA)?
• RQ4 When does global minimax imply local minimax, and what regularity guarantees exist for existence?
• RQ5 What is the impact of an efficient max-oracle on finding minimax points?

Key findings

• Local minimax is defined as a local surrogate for global minimax points in sequential two-player games.
• Local minimax points satisfy first-order conditions: ∇_x f = 0 and ∇_y f = 0.
• Second-order conditions reflect the sequential game order: ∇^2_y_y f ≤ 0 and a Schur-complement condition on x holds for sufficiency.
• Local Nash equilibria imply local minimax points, but local minimax points exist even when local Nash equilibria may not.
• Under certain regularity (e.g., f is twice differentiable and f(·,·) is strongly concave in y near local maxima), the global minimax point is also a local minimax point.
• The asymptotic behavior of gamma-GDA (with step-size ratio γ) is linked to local minimax points; stable limit points of gamma-GDA align with local minimax points up to degenerate cases.