[Paper Review] Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect

Wei Xiang, Boqing Gong+3 more|arXiv (Cornell University)|Mar 5, 2018

Generative Adversarial Networks and Image Synthesis141 citations

TL;DR

The paper introduces CT-GAN, a Lipschitz-consistency regularization for WGANs that augments the gradient penalty with a consistency term on real data manifolds, improving image fidelity and enabling strong semi-supervised learning results.

ABSTRACT

Despite being impactful on a variety of problems and applications, the generative adversarial nets (GANs) are remarkably difficult to train. This issue is formally analyzed by \\cite{arjovsky2017towards}, who also propose an alternative direction to avoid the caveats in the minmax two-player training of GANs. The corresponding algorithm, called Wasserstein GAN (WGAN), hinges on the 1-Lipschitz continuity of the discriminator. In this paper, we propose a novel approach to enforcing the Lipschitz continuity in the training procedure of WGANs. Our approach seamlessly connects WGAN with one of the recent semi-supervised learning methods. As a result, it gives rise to not only better photo-realistic samples than the previous methods but also state-of-the-art semi-supervised learning results. In particular, our approach gives rise to the inception score of more than 5.0 with only 1,000 CIFAR-10 images and is the first that exceeds the accuracy of 90% on the CIFAR-10 dataset using only 4,000 labeled images, to the best of our knowledge.

Motivation & Objective

• Motivate and address training instability in Wasserstein GANs (WGANs) by enforcing Lipschitz continuity more effectively.
• Propose a consistency term that enforces Lipschitz continuity over the real-data manifold in addition to gradient penalties.
• Demonstrate improved photo-realistic sample generation and strong semi-supervised learning performance on CIFAR-10 and MNIST.
• Show data efficiency and reduced overfitting in low-data regimes.
• Provide a framework that integrates seamlessly with semi-supervised GAN-based learning.

Proposed method

• Augment the improved WGAN objective with a soft consistency term CT derived from Lipschitz continuity to penalize violations of the Lipschitz bound.
• Introduce perturbations of real data points via dropout-induced virtual points in the discriminator to estimate a local Lipschitz constraint around real data.
• Add a gradient-penalty term GP on interpolated samples between real and generated data as in GP-WGAN, and combine it with CT in the overall loss.
• Formulate the discriminator objective L = E_z[D(G(z))] − E_x[D(x)] + λ1 GP|ẑ + λ2 CT|x′,x″, where CT enforces consistency between perturbed discriminator outputs on nearby real-data neighborhoods.
• Connect the approach with semi-supervised learning by adapting the discriminator to K+1 outputs and incorporating a temporal ensembling-style consistency term CT in the SSL objective.
• Provide training details including hyperparameters (e.g., λ1 = 10, λ2 = 2) and M′ settings (0 to 0.2) used in experiments.

Experimental results

Research questions

• RQ1 Does enforcing Lipschitz continuity over the real data manifold via a consistency term improve WGAN training stability and sample quality?
• RQ2 Can the proposed CT term complement gradient penalties to yield better semi-supervised learning performance with limited labeled data?
• RQ3 How does CT-GAN perform on standard benchmarks (MNIST, CIFAR-10) in both unsupervised and semi-supervised settings compared to prior GAN-based methods?
• RQ4 Does the approach reduce overfitting and maintain data efficiency in low-data regimes?

Key findings

• CT-GAN yields more photo-realistic samples than GP-WGAN on CIFAR-10 and MNIST.
• The method shows reduced overfitting and continued improvement on test data where GP-WGAN saturates.
• CT-GAN achieves state-of-the-art inception scores on CIFAR-10 for both unsupervised and semi-supervised settings (e.g., inception score surpassing prior GAN-based results).
• In semi-supervised CIFAR-10 with only 4,000 labeled images, CT-GAN achieves 9.98% test error, outperforming several competing GAN-based SSL methods.
• On MNIST, CT-GAN attains competitive semi-supervised test errors (0.89% ± 0.13) relative to other methods.
• Qualitative results show cleaner and more coherent samples compared to GP-WGAN across network architectures.