[Paper Review] Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile

Panayotis Mertikopoulos, Bruno Lecouat+4 more|arXiv (Cornell University)|Jul 7, 2018

Advanced Numerical Methods in Computational MathematicsEngineering3 references111 citations

TL;DR

The paper analyzes mirror descent for non-monotone saddle-point problems defined via coherence, introduces optimistic mirror descent (OMD) with an extra-gradient step, proves convergence results, and validates gains empirically on GANs and other models.

ABSTRACT

Owing to their connection with generative adversarial networks (GANs), saddle-point problems have recently attracted considerable interest in machine learning and beyond. By necessity, most theoretical guarantees revolve around convex-concave (or even linear) problems; however, making theoretical inroads towards efficient GAN training depends crucially on moving beyond this classic framework. To make piecemeal progress along these lines, we analyze the behavior of mirror descent (MD) in a class of non-monotone problems whose solutions coincide with those of a naturally associated variational inequality - a property which we call coherence. We first show that ordinary, "vanilla" MD converges under a strict version of this condition, but not otherwise; in particular, it may fail to converge even in bilinear models with a unique solution. We then show that this deficiency is mitigated by optimism: by taking an "extra-gradient" step, optimistic mirror descent (OMD) converges in all coherent problems. Our analysis generalizes and extends the results of Daskalakis et al. (2018) for optimistic gradient descent (OGD) in bilinear problems, and makes concrete headway for establishing convergence beyond convex-concave games. We also provide stochastic analogues of these results, and we validate our analysis by numerical experiments in a wide array of GAN models (including Gaussian mixture models, as well as the CelebA and CIFAR-10 datasets).

Motivation & Objective

• Motivate and analyze the limitations of vanilla mirror descent in non-monotone saddle-point problems where coherence with a variational inequality holds.
• Introduce optimistic mirror descent (OMD) with an extra-gradient step to stabilize and ensure convergence.
• Establish convergence guarantees for OMD in strictly coherent and stochastic settings.
• Provide stochastic analogues and validate the theory through GAN-related experiments on multiple datasets.

Proposed method

• Model saddle-point problems via a differentiable objective f with variables x=(x1,x2).
• Define the gradient vector g(x)=(∇x1 f(x1,x2), -∇x2 f(x1,x2)) and study coherence with the associated VI.
• Use a distance-generating function h to define Bregman divergences and prox-mappings for mirror descent (MD).
• Show that vanilla MD may diverge or cycle in null-coherent cases, even with vanishing stepsizes.
• Introduce optimistic mirror descent (OMD) by adding an extra-gradient step: compute an intermediate x+ and then update using g(x+).
• Prove convergence results: monotone convergence of D(x*,Xn) under coherence for OMD; almost sure convergence in stochastic strictly coherent settings; corollaries for bilinear and convex-concave cases.

Experimental results

Research questions

• RQ1 When does vanilla mirror descent converge in non-monotone, coherent saddle-point problems?
• RQ2 Can an extra-gradient (optimistic) step stabilize MD to ensure convergence in coherent and strictly coherent settings?
• RQ3 What are the convergence guarantees for OMD in stochastic saddle-point problems?
• RQ4 Do the theoretical gains translate to practical improvements in GAN training and other non-convex bilinear or multi-modal settings?
• RQ5 How do coherence properties (strict vs null) influence the behavior of MD and OMD?

Key findings

• Vanilla MD may fail to converge or cycle in null-coherent (e.g., bilinear) problems, even with unique solutions.
• OMD with an extra-gradient step guarantees convergence in all coherent problems, including null-coherent ones, and yields monotone decrease of the Bregman distance to a solution.
• In strictly coherent problems, OMD converges to a saddle-point almost surely in stochastic settings and the Bregman distance to a solution decreases monotonically.
• For bilinear problems, OMD ensures monotone convergence, unlike vanilla MD which may diverge.
• Experiments show that adding an extra-gradient step to Adam or RMSProp reduces cycling and oscillations in GANs, and improves Inception scores and Fréchet distances on CelebA and CIFAR-10.