[論文レビュー] GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium

Martin Heusel, Hubert Ramsauer+3 more|arXiv (Cornell University)|Jun 26, 2017

Generative Adversarial Networks and Image Synthesis被引用数 4,490

ひとこと要約

二時尺度更新ルール（TTUR）をGANsに導入し、別々の識別子と生成器の学習率を用いて収束を証明し、Adamを摩擦付きの重球として分析し、評価のためのFréchet Inception Distance（FID）を提案する。TTURは複数のデータセットでGANの性能を向上させる。

ABSTRACT

Generative Adversarial Networks (GANs) excel at creating realistic images with complex models for which maximum likelihood is infeasible. However, the convergence of GAN training has still not been proved. We propose a two time-scale update rule (TTUR) for training GANs with stochastic gradient descent on arbitrary GAN loss functions. TTUR has an individual learning rate for both the discriminator and the generator. Using the theory of stochastic approximation, we prove that the TTUR converges under mild assumptions to a stationary local Nash equilibrium. The convergence carries over to the popular Adam optimization, for which we prove that it follows the dynamics of a heavy ball with friction and thus prefers flat minima in the objective landscape. For the evaluation of the performance of GANs at image generation, we introduce the "Fréchet Inception Distance" (FID) which captures the similarity of generated images to real ones better than the Inception Score. In experiments, TTUR improves learning for DCGANs and Improved Wasserstein GANs (WGAN-GP) outperforming conventional GAN training on CelebA, CIFAR-10, SVHN, LSUN Bedrooms, and the One Billion Word Benchmark.

研究の動機と目的

• Motivate and address the lack of convergence guarantees in GAN training.
• Propose a two time-scale update rule (TTUR) with separate learning rates for discriminator and generator.
• Provide theoretical convergence results to stationary local Nash equilibria under TTUR and analyze Adam dynamics as heavy ball with friction.
• Introduce Fréchet Inception Distance (FID) as a robust GAN evaluation metric.
• Empirically validate TTUR on multiple GAN variants and datasets.

提案手法

• Define TTUR with learning rates b(n) for the discriminator and a(n) for the generator and derive stochastic approximation updates.
• Prove convergence of TTUR to a stationary local Nash equilibrium under mild assumptions using ODE and stochastic approximation theory.
• Describe Adam as a heavy ball with friction (HBF) and connect it to TTUR convergence.
• Show that Adam with TTUR preserves convergence by linking its dynamics to HBF ODEs.
• Introduce Fréchet Inception Distance (FID) as a means to evaluate how close generated data are to real data using mean and covariance in Inception coding space.
• Experiment with DCGAN, WGAN-GP (image and language data), and One Billion Word Benchmark to compare TTUR against standard one-time-scale training.

実験結果

リサーチクエスチョン

• RQ1 Can GANs trained with TTUR converge to a stationary local Nash equilibrium under stochastic gradient updates?
• RQ2 How do separate learning rates for the discriminator and generator affect convergence and performance compared to single-time-scale training?
• RQ3 Is Adam compatible with TTUR in terms of convergence, and how does its dynamics influence minima quality?
• RQ4 Does FID provide a more reliable evaluation of GAN quality than Inception Score across varied perturbations and datasets?
• RQ5 Do TTUR-trained GANs outperform conventional training on image and language benchmarks?

主な発見

• TTUR converges to a stationary local Nash equilibrium under mild assumptions.
• Adam can be interpreted as heavy ball with friction, and with TTUR it follows dynamics that preserve convergence.
• TTUR combined with Adam yields convergence and tends to prefer flat minima.
• TTUR consistently improves performance over standard single-time-scale GAN training across CelebA, CIFAR-10, SVHN, LSUN Bedrooms, and One Billion Word benchmarks.
• FID correlates better with data disturbances and human judgment than Inception Score, demonstrating robust GAN evaluation.
• TTUR reduces training oscillations and variance, leading to more stable learning than original methods.