[Paper Review] Triple Generative Adversarial Nets

Chongxuan Li, Kun Xu+2 more|arXiv (Cornell University)|Mar 7, 2017

Generative Adversarial Networks and Image Synthesis25 references230 citations

TL;DR

Triple-GAN introduces three players (generator, discriminator, and classifier) to jointly model p(x,y) in semi-supervised learning, achieving state-of-the-art classification while enabling class-conditional generation and latent space interpolation.

ABSTRACT

Generative Adversarial Nets (GANs) have shown promise in image generation and semi-supervised learning (SSL). However, existing GANs in SSL have two problems: (1) the generator and the discriminator (i.e. the classifier) may not be optimal at the same time; and (2) the generator cannot control the semantics of the generated samples. The problems essentially arise from the two-player formulation, where a single discriminator shares incompatible roles of identifying fake samples and predicting labels and it only estimates the data without considering the labels. To address the problems, we present triple generative adversarial net (Triple-GAN), which consists of three players---a generator, a discriminator and a classifier. The generator and the classifier characterize the conditional distributions between images and labels, and the discriminator solely focuses on identifying fake image-label pairs. We design compatible utilities to ensure that the distributions characterized by the classifier and the generator both converge to the data distribution. Our results on various datasets demonstrate that Triple-GAN as a unified model can simultaneously (1) achieve the state-of-the-art classification results among deep generative models, and (2) disentangle the classes and styles of the input and transfer smoothly in the data space via interpolation in the latent space class-conditionally.

Motivation & Objective

• Address two main SSL GAN problems: (1) generator and discriminator may not be optimal simultaneously, (2) generator cannot control generated sample semantics.
• Propose a three-player framework with a generator, a classifier, and a discriminator to model joint distributions p(x,y).
• Ensure that both the classifier and generator distributions converge to the data distribution through compatible utilities.

Proposed method

• Three networks: a classifier C approximating p(y|x), a class-conditional generator G approximating p(x|y), and a discriminator D distinguishing real vs. fake data-label pairs.
• A minimax game with adversarial losses: U(C,G,D) = E_{(x,y)~p(x,y)}[log D(x,y)] + α E_{(x,y)~p_c(x,y)}[log(1−D(x,y))] + (1−α)E_{(x,y)~p_g(x,y)}[log(1−D(G(y,z),y))], (α ∈ (0,1)).
• A supervised regularization term R_L = E_{(x,y)~p(x,y)}[−log p_c(y|x)] to drive unique equilibrium.
• Addition of pseudo discriminative loss R_P = E_{p_g}[−log p_c(y|x)] to leverage generated pairs for C.
• Training procedure includes updating D, C, and G with minibatch SGD and unbiased estimators, plus practical SSL regularizations.

Experimental results

Research questions

• RQ1 Can Triple-GAN jointly optimize a classifier and a class-conditional generator so that both converge to the true data distribution?
• RQ2 Does introducing a three-player game with a single discriminator prevent the incompatibilities seen in two-player SSL GANs?
• RQ3 Can the model learn disentangled class and style factors and enable meaningful class-conditional latent-space interpolation under limited labels?
• RQ4 What is the impact of pseudo discriminative loss on semi-supervised classification and sample generation?

Key findings

• Triple-GAN achieves state-of-the-art semi-supervised classification results among deep generative models on MNIST, SVHN, and CIFAR-10 with limited labels.
• The model can disentangle classes and styles, and enable smooth class-conditioned interpolation in latent space.
• The equilibrium of the proposed game is when p(x,y) = p_g(x,y) = p_c(x,y), ensuring both C and G converge to the data distribution under the designed utilities.
• A pseudo discriminative loss and other SSL regularizations improve classification accuracy and sample quality.
• Empirical results show Triple-GAN outperforms several SSL GAN baselines (e.g., Improved-GAN) across datasets, with notable gains on SVHN and CIFAR-10.