[Paper Review] Convergence of Gradient Methods on Bilinear Zero-Sum Games

Guojun Zhang, Yaoliang Yu|arXiv (Cornell University)|Aug 15, 2019

Advanced Bandit Algorithms Research44 references4 citations

TL;DR

This paper provides a systematic convergence analysis of gradient methods on bilinear zero-sum games, comparing simultaneous (Jacobi-style) and alternating (Gauss–Seidel-style) updates. It establishes exact convergence conditions, derives optimal parameter settings, and proves that alternating updates converge faster and over a larger parameter region than simultaneous updates, extending the Stein–Rosenberg theorem to bilinear games with provable linear convergence rates.

ABSTRACT

Min-max formulations have attracted great attention in the ML community due to the rise of deep generative models and adversarial methods, while understanding the dynamics of gradient algorithms for solving such formulations has remained a grand challenge. As a first step, we restrict to bilinear zero-sum games and give a systematic analysis of popular gradient updates, for both simultaneous and alternating versions. We provide exact conditions for their convergence and find the optimal parameter setup and convergence rates. In particular, our results offer formal evidence that alternating updates converge "better" than simultaneous ones.

Motivation & Objective

• To understand the convergence behavior of gradient-based algorithms in bilinear zero-sum games, a foundational setting for GANs and adversarial training.
• To compare the convergence properties of simultaneous (Jacobi-style) and alternating (Gauss–Seidel-style) gradient updates.
• To derive exact convergence conditions and optimal step sizes or hyperparameters for popular algorithms like Extra Gradient, Optimistic Gradient Descent, and momentum.
• To generalize existing convergence results and establish a formal analogy to the Stein–Rosenberg theorem in the context of bilinear games.

Proposed method

• Analyzes the dynamics of gradient-type algorithms using linear system theory, modeling updates as linear iterations with matrix splitting.
• Derives convergence conditions by studying the spectral radius of the iteration matrix, ensuring it is less than one.
• Applies the Stein–Rosenberg theorem framework to bilinear games, proving that Gauss–Seidel updates converge over a strictly larger parameter region than Jacobi updates.
• Derives optimal step sizes and hyperparameters (e.g., β) that minimize the convergence rate, expressed in terms of the singular values σ₁ and σₙ of the matrix E.
• Uses singular value decomposition (SVD) to reduce the singular case to the non-singular case, preserving convergence analysis.
• Employs a distance measure Δₜ² to track convergence to the saddle point set, reducing the problem to convergence in the non-singular subspace.

Experimental results

Research questions

• RQ1 Under what exact conditions do simultaneous and alternating gradient updates converge on bilinear zero-sum games?
• RQ2 How do the convergence regions of Jacobi-style and Gauss–Seidel-style updates compare, and is the former strictly contained in the latter?
• RQ3 What are the optimal step sizes and hyperparameters that yield the fastest linear convergence rates for algorithms like Extra Gradient and Optimistic Gradient Descent?
• RQ4 Can a formal analog of the Stein–Rosenberg theorem be established for bilinear games, proving that alternating updates converge faster than simultaneous ones?
• RQ5 How does the convergence rate scale with the condition number κ = σ₁/σₙ in these settings?

Key findings

• Alternating (Gauss–Seidel) updates converge over a strictly larger parameter region than simultaneous (Jacobi) updates, and always converge when Jacobi does, confirming a formal extension of the Stein–Rosenberg theorem to bilinear games.
• For the Extra Gradient method, convergence is guaranteed when β₁ + β₂ + α² < 2/σ₁², and optimal parameters yield a convergence rate exponent of ∼1 − 2/κ².
• Optimistic Gradient Descent with Gauss–Seidel updates achieves a convergence rate exponent of ∼1 − 1/κ², which is faster than the ∼1 − 1/(6κ²) rate of its Jacobi counterpart.
• Momentum-based methods with alternating updates converge at a rate of ∼1 − 2/(9κ²), outperforming the Jacobi version and showing improved stability.
• In the singular case, convergence is determined by the projection onto the null spaces of E and Eᵀ, and the analysis reduces to the non-singular case via SVD, preserving all convergence results.