[Paper Review] Global Convergence and Variance-Reduced Optimization for a Class of Nonconvex-Nonconcave Minimax Problems

Junchi Yang, Negar Kiyavash+1 more|arXiv (Cornell University)|Feb 22, 2020

Stochastic Gradient Optimization Techniques31 references33 citations

TL;DR

The paper introduces a two-sided Polyak-Łojasiewicz (PL) condition under which alternating gradient descent ascent (AGDA) and stochastic AGDA converge globally for nonconvex-nonconcave minimax problems, and proposes a variance-reduced AGDA (VR-AGDA) with provably faster rates in finite-sum settings.

ABSTRACT

Nonconvex minimax problems appear frequently in emerging machine learning applications, such as generative adversarial networks and adversarial learning. Simple algorithms such as the gradient descent ascent (GDA) are the common practice for solving these nonconvex games and receive lots of empirical success. Yet, it is known that these vanilla GDA algorithms with constant step size can potentially diverge even in the convex setting. In this work, we show that for a subclass of nonconvex-nonconcave objectives satisfying a so-called two-sided Polyak-Łojasiewicz inequality, the alternating gradient descent ascent (AGDA) algorithm converges globally at a linear rate and the stochastic AGDA achieves a sublinear rate. We further develop a variance reduced algorithm that attains a provably faster rate than AGDA when the problem has the finite-sum structure.

Motivation & Objective

• Motivate global convergence in nonconvex-nonconcave minimax problems without convex-concave assumptions.
• Identify a practical condition (two-sided PL) that guarantees global convergence of AGDA and Stoc-AGDA.
• Develop and analyze a variance-reduced AGDA (VR-AGDA) for finite-sum minimax problems with improved rates.

Proposed method

• Generalizes Polyak-Łojasiewicz (PL) to two-sided PL condition for minimax objectives with independent PL constants for x and y.
• Proposes AGDA and Stoc-AGDA with alternating updates and analyzes convergence under two-sided PL.
• Introduces a potential function combining optimality gaps to prove convergence rates.
• Develops VR-AGDA by integrating SVRG-style variance reduction with alternating updates; proves linear convergence under two-sided PL.
• Provides complexity results showing VR-AGDA improvements over AGDA in finite-sum settings.

Experimental results

Research questions

• RQ1 Can AGDA and Stoc-AGDA achieve global convergence for nonconvex-nonconcave minimax problems under a two-sided PL condition?
• RQ2 Does a variance-reduced variant (VR-AGDA) improve convergence rates for finite-sum minimax problems without requiring convex-concave structure?

Key findings

• Under two-sided PL, AGDA achieves global linear convergence to a saddle point with appropriate step sizes.
• Stoc-AGDA converges to a saddle point sublinearly (O(1/t)) with diminishing stepsizes, factoring in stochastic variance.
• VR-AGDA attains faster rates with total complexities O((n+κ^9) log(1/ε)) when n≥κ^9 and O(n^{2/3} κ^3 log(1/ε)) when n≤κ^9, improving over AGDA.
• Three equivalent optimality criteria (saddle point, global minimax, stationary point) hold under two-sided PL.
• Empirical results on robust least squares and LQR imitation learning demonstrate VR-AGDA's superior performance, especially at high condition numbers.