[論文レビュー] Last-iterate convergence rates for min-max optimization

Jacob Abernethy, Kevin A. Lai+1 more|arXiv (Cornell University)|Jun 5, 2019

Advanced Optimization Algorithms Research参考文献 35被引用数 41

ひとこと要約

この論文は、凸凹ミニマックス問題における新しい十分双線形条件の下で、Hamiltonian Gradient Descent (HGD)アルゴリズムの非漸近的な最終反復線形収束速度を証明し、Consensus Optimization (CO) および確率的HGDに対する類似の結果を示す。

ABSTRACT

While classic work in convex-concave min-max optimization relies on average-iterate convergence results, the emergence of nonconvex applications such as training Generative Adversarial Networks has led to renewed interest in last-iterate convergence guarantees. Proving last-iterate convergence is challenging because many natural algorithms, such as Simultaneous Gradient Descent/Ascent, provably diverge or cycle even in simple convex-concave min-max settings, and previous work on global last-iterate convergence rates has been limited to the bilinear and convex-strongly concave settings. In this work, we show that the Hamiltonian Gradient Descent (HGD) algorithm achieves linear convergence in a variety of more general settings, including convex-concave problems that satisfy a "sufficiently bilinear" condition. We also prove similar convergence rates for the Consensus Optimization (CO) algorithm of [MNG17] for some parameter settings of CO.

研究の動機と目的

• Motivate and establish last-iterate convergence guarantees for min-max problems beyond bilinear and strongly convex-strongly concave settings.
• Introduce and analyze Hamiltonian Gradient Descent (HGD) as gradient descent on the Hamiltonian to find saddle points.
• Derive global linear convergence rates under weaker assumptions than prior work, including a novel sufficiently bilinear condition.
• Connect HGD to Consensus Optimization (CO) and show comparable rates under suitable parameters.
• Extend results to stochastic HGD and show corresponding O(1/√k) rates.

提案手法

• Define the Hamiltonian H(x) = 1/2 ||ξ(x)||^2 with ξ(x) = (∂g/∂x1, -∂g/∂x2).
• Update x^(k+1) = x^(k) - η ∇H(x^(k)), requiring Hessian-vector products via ∇H = ξ^T J.
• Prove that H(x) satisfies the Polyak-Łojasiewicz (PL) condition under various assumptions, enabling linear convergence of gradient descent on H.
• Introduce a novel “sufficiently bilinear” condition (eq. 3) involving cross-derivatives and second-order terms that ensures linear convergence in convex-concave settings without strong convexity.
• Show that if HGD converges under the PL condition with parameter α, then ||ξ(x^(k))|| decays geometrically with rate (1 - α/L_H)^(k/2).
• Provide extensions to stochastic HGD (O(1/√k) rates) and to Consensus Optimization (CO) under suitable parameter choices.

実験結果

リサーチクエスチョン

• RQ1 Can last-iterate convergence be guaranteed globally for min-max problems beyond bilinear and strongly convex-strongly concave cases?
• RQ2 Under what conditions does Hamiltonian Gradient Descent achieve linear, non-asymptotic convergence for convex-concave min-max objectives?
• RQ3 What is the role of a sufficiently bilinear cross-derivative structure in ensuring fast convergence?
• RQ4 How do stochastic variants of HGD and related algorithms like Consensus Optimization perform in these settings?

主な発見

• HGD achieves global linear last-iterate convergence in several settings beyond strong convexity/linearity, including convex-concave problems under a sufficiently bilinear condition.
• A PL condition for the Hamiltonian is established via bounds on JJ^T, enabling linear convergence guarantees.
• A concrete rate expression shows ||ξ(x^(k))|| decays geometrically with rate depending on problem constants (e.g., γ, L, μ, ρ, Γ) under the sufficient bilinear condition.
• For the nonconvex-nonconcave and related nonconvex-linear cases, the paper derives explicit PL parameters (α) and shows linear decay of the gradient norm of the Hamiltonian.
• Stochastic HGD inherits an O(1/√k) convergence rate under the PL framework, using standard stochastic gradient arguments.
• Consensus Optimization (CO) can achieve the same linear rates as HGD in the same settings when the CO update parameter γ is chosen sufficiently large.