[論文レビュー] What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization?

Chi Jin, Praneeth Netrapalli+1 more|arXiv (Cornell University)|Feb 2, 2019

Stochastic Gradient Optimization Techniques参考文献 33被引用数 74

ひとこと要約

論文は局所ミンマックスを逐次的ミンマックスゲームに対するグローバルミンマックスの局所的な代理として定義し、その性質と存在を分析し、勾配降下-上昇 (GDA) ダイナミクスの極限挙動へと結びつける。

ABSTRACT

Minimax optimization has found extensive applications in modern machine learning, in settings such as generative adversarial networks (GANs), adversarial training and multi-agent reinforcement learning. As most of these applications involve continuous nonconvex-nonconcave formulations, a very basic question arises---"what is a proper definition of local optima?" Most previous work answers this question using classical notions of equilibria from simultaneous games, where the min-player and the max-player act simultaneously. In contrast, most applications in machine learning, including GANs and adversarial training, correspond to sequential games, where the order of which player acts first is crucial (since minimax is in general not equal to maximin due to the nonconvex-nonconcave nature of the problems). The main contribution of this paper is to propose a proper mathematical definition of local optimality for this sequential setting---local minimax, as well as to present its properties and existence results. Finally, we establish a strong connection to a basic local search algorithm---gradient descent ascent (GDA): under mild conditions, all stable limit points of GDA are exactly local minimax points up to some degenerate points.

研究の動機と目的

• Clarify differences between global/local optimality notions in minimax settings, especially for sequential games.
• Introduce and formalize local minimax as a local surrogate for global minimax points.
• Establish first- and second-order conditions for local minimax points.
• Prove existence/non-existence results for local minimax points under various regularity assumptions.
• Connect local minimax to the asymptotic behavior and stability of gradient descent ascent (GDA) dynamics.

提案手法

• Propose a formal definition of local minimax for two-player sequential minimax problems.
• Derive first-order and second-order necessary/sufficient conditions for local minimax points.
• Analyze existence results and provide conditions under which local minimax points exist (e.g., strong concavity in y).
• Examine the relationship between local minimax points and stable limit points of gamma-GDA (gradient descent ascent with step-size ratio).
• Use limit-flow analysis of GDA to relate stable fixed points to local minimax points.
• Discuss special cases and provide a framework for max-oracle scenarios (Appendix 4).

実験結果

リサーチクエスチョン

• RQ1 What is an appropriate local optimality concept for sequential minimax problems in nonconvex-nonconcave settings?
• RQ2 Under what conditions do local minimax points exist and how can they be characterized?
• RQ3 How do local minimax points relate to the asymptotic behavior and stability of gradient descent ascent (GDA)?
• RQ4 When does global minimax imply local minimax, and what regularity guarantees exist for existence?
• RQ5 What is the impact of an efficient max-oracle on finding minimax points?

主な発見

• 局所ミンマックスは、逐次的な二人プレイヤーゲームにおけるグローバルミンマックス点の局所的な代理として定義される。
• 局所ミンマックス点は一階条件を満たす: ∇_x f = 0 および ∇_y f = 0。
• 二階条件は逐次ゲームの順序を反映: ∇^2_y_y f ≤ 0 および x に関する Schur 補数条件が十分性を満たす。
• 局所ナッシュ均衡は局所ミンマックス点を意味するが、局所ミンマックス点は存在するが局所ナッシュ均衡が存在しない場合もある。
• ある正則性（例: f が二回連 differentiable で、f(·,·) が局所 maxima 付近で y に対して強く凹である場合）、グローバルミンマックス点は局所ミンマックス点でもある。
• gamma-GDA（ステップサイズ比 γ を用いる）の漸近挙動は局所ミンマックス点と関連する。gamma-GDA の安定な極限点は、退化的な場合を除き、局所ミンマックス点と一致する。