[Paper Review] Triangle Generative Adversarial Networks

Zhe Gan, Li‐Qun Chen+6 more|arXiv (Cornell University)|Sep 19, 2017

Generative Adversarial Networks and Image Synthesis27 references78 citations

TL;DR

Δ-GAN introduces two generators and two discriminators to perform semi-supervised joint distribution matching across two domains, enabling bidirectional mappings with limited paired data.

ABSTRACT

A Triangle Generative Adversarial Network ($Δ$-GAN) is developed for semi-supervised cross-domain joint distribution matching, where the training data consists of samples from each domain, and supervision of domain correspondence is provided by only a few paired samples. $Δ$-GAN consists of four neural networks, two generators and two discriminators. The generators are designed to learn the two-way conditional distributions between the two domains, while the discriminators implicitly define a ternary discriminative function, which is trained to distinguish real data pairs and two kinds of fake data pairs. The generators and discriminators are trained together using adversarial learning. Under mild assumptions, in theory the joint distributions characterized by the two generators concentrate to the data distribution. In experiments, three different kinds of domain pairs are considered, image-label, image-image and image-attribute pairs. Experiments on semi-supervised image classification, image-to-image translation and attribute-based image generation demonstrate the superiority of the proposed approach.

Motivation & Objective

• Motivate and address semi-supervised joint distribution matching between two related domains.
• Propose a four-network GAN (two generators, two discriminators) to learn bidirectional conditional distributions.
• Provide a theoretical guarantee that the learned joint distributions concentrate to the data distribution under mild assumptions.
• Demonstrate versatility across domain pairs such as image-label, image-image, and image-attribute with applications to classification, translation, and editing.

Proposed method

• Two generators learn bidirectional conditional distributions between domains: p_x(x|y) and p_y(y|x).
• Two discriminators jointly implement a ternary discriminative function to distinguish real pairs from two kinds of fake pairs.
• Adversarial training jointly optimizes the generators and the discriminators to align p, p_x, and p_y.
• Equivalence to Jensen-Shannon divergence among three joint distributions in the optimal-discriminator setting.
• Semi-supervised formulation combines conditional GAN objective with BiGAN/ALI objective to utilize paired and unpaired data.
• The model can sample from conditional distributions without requiring closed-form densities.

Experimental results

Research questions

• RQ1 Can Δ-GAN achieve accurate semi-supervised joint distribution matching between two domains with only a few paired samples?
• RQ2 Do the bidirectional generators converge so that p_x and p_y reproduce the true conditional distributions p(x|y) and p(y|x)?
• RQ3 How does Δ-GAN compare to related methods (e.g., Triple GAN) in various domain-pair tasks such as image-label, image-image, and image-attribute?
• RQ4 Is Δ-GAN effective for a range of applications including semi-supervised classification, image-to-image translation, and attribute-based image generation?

Key findings

• Δ-GAN can match joint distributions across domain pairs and enable bidirectional mappings.
• The framework demonstrates semi-supervised capabilities on tasks like image classification, translation, and attribute-conditioned generation.
• Compared to related methods, Δ-GAN shows advantages in aligning three joint distributions and avoiding asymmetrical objectives.
• Experiments across CIFAR-10, MNIST-like translation tasks, CelebA and COCO demonstrate competitive or superior performance to alternatives.