[논문 리뷰] What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization?

Chi Jin, Praneeth Netrapalli외 1명|arXiv (Cornell University)|2019. 02. 02.

Stochastic Gradient Optimization Techniques참고 문헌 33인용 수 74

한 줄 요약

본 논문은 순차적 minimax 게임에서 전역 minimax의 로컬 대리로서 로컬 미니맥스를 정의하고, 그 특성 및 존재성을 분석하며, 이를 gradient descent ascent (GDA) 역학의 극한 거동과 연결한다.

ABSTRACT

Minimax optimization has found extensive applications in modern machine learning, in settings such as generative adversarial networks (GANs), adversarial training and multi-agent reinforcement learning. As most of these applications involve continuous nonconvex-nonconcave formulations, a very basic question arises---"what is a proper definition of local optima?" Most previous work answers this question using classical notions of equilibria from simultaneous games, where the min-player and the max-player act simultaneously. In contrast, most applications in machine learning, including GANs and adversarial training, correspond to sequential games, where the order of which player acts first is crucial (since minimax is in general not equal to maximin due to the nonconvex-nonconcave nature of the problems). The main contribution of this paper is to propose a proper mathematical definition of local optimality for this sequential setting---local minimax, as well as to present its properties and existence results. Finally, we establish a strong connection to a basic local search algorithm---gradient descent ascent (GDA): under mild conditions, all stable limit points of GDA are exactly local minimax points up to some degenerate points.

연구 동기 및 목표

• Clarify differences between global/local optimality notions in minimax settings, especially for sequential games.
• Introduce and formalize local minimax as a local surrogate for global minimax points.
• Establish first- and second-order conditions for local minimax points.
• Prove existence/non-existence results for local minimax points under various regularity assumptions.
• Connect local minimax to the asymptotic behavior and stability of gradient descent ascent (GDA) dynamics.

제안 방법

• Propose a formal definition of local minimax for two-player sequential minimax problems.
• Derive first-order and second-order necessary/sufficient conditions for local minimax points.
• Analyze existence results and provide conditions under which local minimax points exist (e.g., strong concavity in y).
• Examine the relationship between local minimax points and stable limit points of gamma-GDA (gradient descent ascent with step-size ratio).
• Use limit-flow analysis of GDA to relate stable fixed points to local minimax points.
• Discuss special cases and provide a framework for max-oracle scenarios (Appendix 4).

실험 결과

연구 질문

• RQ1 What is an appropriate local optimality concept for sequential minimax problems in nonconvex-nonconcave settings?
• RQ2 Under what conditions do local minimax points exist and how can they be characterized?
• RQ3 How do local minimax points relate to the asymptotic behavior and stability of gradient descent ascent (GDA)?
• RQ4 When does global minimax imply local minimax, and what regularity guarantees exist for existence?
• RQ5 What is the impact of an efficient max-oracle on finding minimax points?

주요 결과

• Local minimax is defined as a local surrogate for global minimax points in sequential two-player games.
• Local minimax points satisfy first-order conditions: ∇_x f = 0 and ∇_y f = 0.
• Second-order conditions reflect the sequential game order: ∇^2_y_y f ≤ 0 and a Schur-complement condition on x holds for sufficiency.
• Local Nash equilibria imply local minimax points, but local minimax points exist even when local Nash equilibria may not.
• Under certain regularity (e.g., f is twice differentiable and f(·,·) is strongly concave in y near local maxima), the global minimax point is also a local minimax point.
• The asymptotic behavior of gamma-GDA (with step-size ratio γ) is linked to local minimax points; stable limit points of gamma-GDA align with local minimax points up to degenerate cases.