[Paper Review] A Tight and Unified Analysis of Gradient-Based Methods for a Whole Spectrum of Games

Waïss Azizian, Ioannis Mitliagkas+2 more|arXiv (Cornell University)|Jun 13, 2019

Advanced Optimization Algorithms Research42 references4 citations

TL;DR

This paper presents a unified spectral analysis of gradient-based methods—particularly extragradient (EG), optimistic gradient (OG), and consensus optimization (CO)—for differentiable games, covering the full spectrum from bilinear to strongly monotone settings. It establishes tighter global convergence rates by revealing that these methods adapt their convergence mechanism based on problem structure, and proves EG achieves the optimal rate among multi-step extrapolation methods.

ABSTRACT

We consider differentiable games where the goal is to find a Nash equilibrium. The machine learning community has recently started using variants of the gradient method (GD). Prime examples are extragradient (EG), the optimistic gradient method (OG) and consensus optimization (CO), which enjoy linear convergence in cases like bilinear games, where the standard GD fails. The full benefits of theses relatively new methods are not known as there is no unified analysis for both strongly monotone and bilinear games. We provide new analyses of the EG's local and global convergence properties and use is to get a tighter global convergence rate for OG and CO. Our analysis covers the whole range of settings between bilinear and strongly monotone games. It reveals that these methods converge via different mechanisms at these extremes; in between, it exploits the most favorable mechanism for the given problem. We then prove that EG achieves the optimal rate for a wide class of algorithms with any number of extrapolations. Our tight analysis of EG's convergence rate in games shows that, unlike in convex minimization, EG may be much faster than GD.

Motivation & Objective

• To provide a single, precise analysis of extragradient (EG) that unifies local and global convergence across bilinear and strongly monotone games.
• To theoretically compare EG, GD, and multi-step extrapolation methods via upper and lower bounds on convergence rates.
• To extend spectral analysis to derive tighter global convergence guarantees for OG and CO, revealing adaptive convergence mechanisms depending on problem structure.
• To prove that EG achieves the optimal convergence rate among a broad class of multi-step extrapolation algorithms with any number of extrapolations.

Proposed method

• Performs spectral analysis of EG to derive a local convergence rate valid across the entire spectrum from bilinear to strongly monotone games.
• Leverages and extends the framework of Arjevani et al. (2016) to derive lower bounds for 1-Stationary Canonical Linear Iterative methods with arbitrary extrapolation steps.
• Uses Tseng's error bound and a novel global rate analysis to improve convergence guarantees for EG, OG, and CO by incorporating both strong monotonicity and singular value bounds.
• Applies Lemmas 1 and 16 to analyze the spectral radius of the iteration map at equilibrium, linking convergence to the eigenstructure of the Jacobian.
• Derives a global convergence rate for EG that unifies behavior across settings and extends to OG and CO via the same framework.
• Empirically evaluates the improvement of the new global rate by measuring the ratio $\eta\mu / (\eta\mu + \frac{7}{16}\eta^2\gamma^2)$ across random matrix games with varying dimensions.

Experimental results

Research questions

• RQ1 How can a single analysis unify the convergence behavior of EG across bilinear and strongly monotone games?
• RQ2 What is the optimal convergence rate achievable by multi-step extrapolation methods, and does EG achieve it?
• RQ3 How do OG and CO converge in intermediate settings between bilinear and strongly monotone games, and what mechanisms underlie their performance?
• RQ4 To what extent does incorporating singular value information ($\gamma$) improve global convergence rates beyond strong monotonicity ($\mu$) alone?
• RQ5 Can the spectral analysis of EG be extended to provide tighter global convergence guarantees for OG and CO?

Key findings

• The proposed global convergence rate for EG is $\frac{1}{4}\left(\frac{\mu}{L} + \frac{\gamma^2}{16L^2}\right)$, which is tighter than prior results and unifies behavior across settings.
• For OG, the global convergence rate is $\frac{1}{4}\left(\frac{\mu}{L} + \frac{\gamma^2}{32L^2}\right)$, showing improved performance over previous bounds.
• For CO, the global convergence rate is $\frac{\mu^2}{2L_H^2} + \frac{\gamma^2}{2L_H^2}$, which incorporates both strong monotonicity and singular value information.
• The spectral analysis confirms that EG's convergence rate is tight and that it significantly outperforms GD in non-convex settings, especially in bilinear games.
• The lower bounds prove that increasing the number of extrapolation steps or using different step sizes does not yield significant improvements, confirming EG's optimality among this class.
• Empirical evaluation shows that the new rate improves upon the standard rate by up to a factor of 2 when $\eta^2\gamma^2 \gg \eta\mu$, especially when player dimensions are balanced.