[Paper Review] Reducing Noise in GAN Training with Variance Reduced Extragradient

Tatjana Chavdarova, Gauthier Gidel+2 more|arXiv (Cornell University)|Apr 18, 2019

Generative Adversarial Networks and Image Synthesis41 references32 citations

TL;DR

The paper introduces SVRE, a stochastic variance-reduced extragradient algorithm, to mitigate gradient noise in GAN training and achieve better convergence, with strong empirical gains on MNIST, CIFAR-10, SVHN, and ImageNet variants.

ABSTRACT

We study the effect of the stochastic gradient noise on the training of generative adversarial networks (GANs) and show that it can prevent the convergence of standard game optimization methods, while the batch version converges. We address this issue with a novel stochastic variance-reduced extragradient (SVRE) optimization algorithm, which for a large class of games improves upon the previous convergence rates proposed in the literature. We observe empirically that SVRE performs similarly to a batch method on MNIST while being computationally cheaper, and that SVRE yields more stable GAN training on standard datasets.

Motivation & Objective

• Highlight how stochastic gradient noise affects GAN training and convergence in game optimization.
• Propose and formulate SVRE to reduce gradient variance in GANs.
• Provide theoretical convergence guarantees for SVRE under strong monotonicity and cocoercivity assumptions.
• Evaluate SVRE empirically on standard datasets to assess stability, convergence speed, and sample quality.
• Compare SVRE with batch extragradient and standard stochastic methods in GAN setups.

Proposed method

• Formulate GAN training as a two-player differentiable game between generator and discriminator.
• Introduce stochastic variance reduced extragradient (SVRE) by combining SVRG gradient estimates with the extragradient (EG) framework.
• Use a snapshot omega^S and corresponding full gradient mu^S to construct unbiased gradient estimators d_i^G and d_i^D per player.
• Incorporate non-uniform sampling pi_i guided by cocoercivity-derived constants and implement a geometric/Monte Carlo-like epoch sampling (q-memorization) for SVRE.
• Provide a convergence theorem (Theorem 2) showing expected squared error decay under Assumption 1 (strong monotonicity, cocoercivity, regularity) with appropriate step sizes.
• Present a motivating simple stochastic bilinear game illustrating noise-induced divergence for SEG and showing SVRE's resilience.

Experimental results

Research questions

• RQ1 How does stochastic gradient noise impact convergence in GAN training framed as a two-player game?
• RQ2 Can variance reduction techniques be effectively extended to game optimization to improve stability and convergence?
• RQ3 What are the theoretical convergence guarantees for SVRE in terms of strong monotonicity, cocoercivity, and regularity?
• RQ4 Do SVRE-based GANs achieve competitive or superior performance and stability on standard datasets compared to batch extragradient and vanilla stochastic methods?

Key findings

• Stochastic gradients can cause divergence in extragradient methods for GAN-style games, even when batch methods converge.
• SVRE reduces gradient variance in games by leveraging SVRG-style gradient estimates within the extragradient framework.
• Theoretical result: SVRE converges under local strong monotonicity and cocoercivity, with a rate depending on mu, ell, and regularity constants (Theorem 2).
• Empirically, SVRE performs similarly to batch methods on MNIST while being computationally cheaper, and yields more stable GAN training on CIFAR-10 and SVHN with deep architectures.
• On CIFAR-10 and SVHN using deep architectures, SVRE achieved better FID scores compared to SG-A and SE-A baselines (CIFAR-10: 16.77 for WS-SVRE vs 21.70 SG-A, 18.65 SE-A, 23.56 SVRE; SVHN: 4.88 WS-SVRE vs 5.66 SG-A, 5.14 SE-A, 4.81 SVRE).
• SVRE variants with warm-starting (WS-SVRE) can continue improving performance even after baseline instability, without divergence.