[Paper Review] On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems

Tianyi Lin, Chi Jin+1 more|arXiv (Cornell University)|Jun 2, 2019

Sparse and Compressive Sensing TechniquesEngineering53 references118 citations

TL;DR

The paper analyzes two-timescale gradient descent ascent (GDA) and stochastic GDA (SGDA) for nonconvex-concave minimax problems, proving nonasymptotic convergence to an epsilon-stationary point of Phi(x)=max_y f(x,y) with explicit gradient and stochastic gradient complexities in both nonconvex-strongly-concave and nonconvex-concave settings.

ABSTRACT

We consider nonconvex-concave minimax problems, $\min_{\mathbf{x}} \max_{\mathbf{y} \in \mathcal{Y}} f(\mathbf{x}, \mathbf{y})$, where $f$ is nonconvex in $\mathbf{x}$ but concave in $\mathbf{y}$ and $\mathcal{Y}$ is a convex and bounded set. One of the most popular algorithms for solving this problem is the celebrated gradient descent ascent (GDA) algorithm, which has been widely used in machine learning, control theory and economics. Despite the extensive convergence results for the convex-concave setting, GDA with equal stepsize can converge to limit cycles or even diverge in a general setting. In this paper, we present the complexity results on two-time-scale GDA for solving nonconvex-concave minimax problems, showing that the algorithm can find a stationary point of the function $Φ(\cdot) := \max_{\mathbf{y} \in \mathcal{Y}} f(\cdot, \mathbf{y})$ efficiently. To the best our knowledge, this is the first nonasymptotic analysis for two-time-scale GDA in this setting, shedding light on its superior practical performance in training generative adversarial networks (GANs) and other real applications.

Motivation & Objective

• Motivate and study nonconvex-concave minimax problems of the form min_x max_y f(x,y) with f nonconvex in x and concave in y.
• Provide nonasymptotic convergence guarantees for two-timescale GDA and SGDA.
• Characterize algorithmic complexity in terms of gradient evaluations and stochastic gradient evaluations under different problem structures.
• Offer insights into practical performance in GANs and robust learning settings.

Proposed method

• Study two-timescale GDA with eta_x << eta_y to stabilize convergence in nonconvex-(strongly)-concave problems.
• Prove that the algorithm finds an epsilon-stationary point of Phi(x)=max_y f(x,y) with explicit gradient complexity O(kappa^2 epsilon^-2) in the nonconvex-strongly-concave setting.
• Prove that SGDA achieves gradient and stochastic gradient complexities O(kappa^3 epsilon^-4) in the same setting with batch size M=Theta(max{1,kappa sigma^2 epsilon^-2}).
• Show that in the nonconvex-concave setting, the methods achieve O(epsilon^-6) gradient evaluations and O(epsilon^-8) stochastic gradient evaluations, using Moreau envelope-based stationarity notions.
• Discuss how the analysis handles slowly changing concave objectives and relates Moreau envelope stationarity to subgradient notions.

Experimental results

Research questions

• RQ1 Can two-timescale GDA/SGDA provably converge to an epsilon-stationary point in nonconvex-concave minimax problems?
• RQ2 What are the nonasymptotic gradient and stochastic gradient complexities of two-timescale GDA/SGDA under nonconvex-strongly-concave and nonconvex-concave settings?
• RQ3 How do step-size choices (eta_x, eta_y) and their ratio affect convergence and complexity?
• RQ4 How does the Moreau envelope-based stationarity notion relate to traditional subgradient-based stationarity in this context?

Key findings

• In nonconvex-strongly-concave problems, two-timescale GDA achieves O(kappa^2 epsilon^-2) gradient evaluations to reach an epsilon-stationary point of Phi, and SGDA achieves O(kappa^3 epsilon^-4) stochastic gradient evaluations.
• In nonconvex-concave problems, two-timescale GDA requires O(epsilon^-6) gradient evaluations and SGDA requires O(epsilon^-8) stochastic gradient evaluations to reach the corresponding stationarity notion.
• The ratio of step-sizes eta_y/eta_x must be Theta(kappa^2), reflecting the non-symmetric nature of min-max problems and ensuring stability.
• The analysis introduces a novel technique for analyzing concave optimization with a slowly changing objective, enabling the nonasymptotic guarantees for single-loop GDA/SGDA.
• The results hold for both deterministic (gradient) and stochastic (SGDA) settings, with explicit dependence on problem conditioning and diameter of the feasible y-set.