[Paper Review] Improved Techniques for Training GANs

Tim Salimans, Ian Goodfellow+4 more|arXiv (Cornell University)|Jun 10, 2016

Generative Adversarial Networks and Image Synthesis1,383 citations

TL;DR

The paper introduces new architectural features and training procedures to stabilize GAN training, improve semi-supervised learning, and boost the visual quality of generated images, achieving state-of-the-art results on MNIST, CIFAR-10, and SVHN. It also demonstrates high-resolution ImageNet samples and proposes an Inception-based evaluation metric.

ABSTRACT

We present a variety of new architectural features and training procedures that we apply to the generative adversarial networks (GANs) framework. We focus on two applications of GANs: semi-supervised learning, and the generation of images that humans find visually realistic. Unlike most work on generative models, our primary goal is not to train a model that assigns high likelihood to test data, nor do we require the model to be able to learn well without using any labels. Using our new techniques, we achieve state-of-the-art results in semi-supervised classification on MNIST, CIFAR-10 and SVHN. The generated images are of high quality as confirmed by a visual Turing test: our model generates MNIST samples that humans cannot distinguish from real data, and CIFAR-10 samples that yield a human error rate of 21.3%. We also present ImageNet samples with unprecedented resolution and show that our methods enable the model to learn recognizable features of ImageNet classes.

Motivation & Objective

• Improve convergence and stability of GAN training.
• Enhance semi-supervised learning performance using GANs.
• Generate high-quality, perceptually realistic images across multiple datasets.
• Provide practical techniques that can be adopted to train GANs effectively.

Proposed method

• Introduce feature matching as a generator objective to match statistics of real data in an intermediate discriminator layer.
• Implement minibatch discrimination to prevent generator mode collapse by allowing the discriminator to compare samples within a minibatch.
• Use historical averaging to stabilize learning by penalizing deviation from a running average of parameters.
• Apply one-sided label smoothing to stabilize the discriminator while avoiding issues when data is scarce.
• Introduce virtual batch normalization to decouple batch statistics from individual sample outputs in the generator.
• Propose an Inception score as an automatic, human-correlated measure of image quality and diversity.

Experimental results

Research questions

• RQ1 Can stabilizing techniques (feature matching, minibatch discrimination, historical averaging, label smoothing, virtual batch normalization) improve GAN convergence and sample quality?
• RQ2 Do these techniques enable effective semi-supervised learning on MNIST, CIFAR-10, and SVHN, and improve ImageNet-scale generation?
• RQ3 Is the proposed Inception score a reliable proxy for human judgment of image quality and diversity?
• RQ4 How do the new techniques interact with existing GAN architectures to produce recognizable features in large-scale datasets?

Key findings

• Achieved state-of-the-art semi-supervised classification results on MNIST, CIFAR-10, and SVHN.
• Generated MNIST samples indistinguishable from real data by humans in some setups; CIFAR-10 samples yielded a human error rate of 21.3% in visual Turing tests.
• Produced ImageNet-like samples with unprecedented resolution and learned recognizable features of ImageNet classes.
• Inception score correlates with human judgments and provides a scalable automatic evaluation metric for generated images.
• Ablation studies show minibatch features and label-related techniques are critical for sample quality and diversity.
• Ensembles of multiple models further improve semi-supervised classification performance.