[Paper Review] cGANs with Projection Discriminator

Takeru Miyato, Masanori Koyama|arXiv (Cornell University)|Feb 15, 2018

Image Processing Techniques and ApplicationsEngineering23 references411 citations

TL;DR

The paper introduces a projection-based discriminator for conditional GANs (cGANs), replacing simple concatenation with an inner-product interaction between embedded labels and features, achieving state-of-the-art results on ImageNet and gains in super-resolution tasks.

ABSTRACT

We propose a novel, projection based way to incorporate the conditional information into the discriminator of GANs that respects the role of the conditional information in the underlining probabilistic model. This approach is in contrast with most frameworks of conditional GANs used in application today, which use the conditional information by concatenating the (embedded) conditional vector to the feature vectors. With this modification, we were able to significantly improve the quality of the class conditional image generation on ILSVRC2012 (ImageNet) 1000-class image dataset from the current state-of-the-art result, and we achieved this with a single pair of a discriminator and a generator. We were also able to extend the application to super-resolution and succeeded in producing highly discriminative super-resolution images. This new structure also enabled high quality category transformation based on parametric functional transformation of conditional batch normalization layers in the generator.

Motivation & Objective

• Motivate a discriminator design that respects probabilistic structure of conditional information.
• Propose a projection-based interaction between conditional labels and feature representations.
• Demonstrate improved quality for class-conditioned image generation on ImageNet and for image super-resolution.
• Showcase capabilities like category morphing and compatibility with conditional batch normalization.

Proposed method

• Derive the projection discriminator form f(x,y;θ)=y^T V φ(x;θΦ) + ψ(φ(x;θΦ)) from probabilistic log-likelihood ratios.
• Replace simple concatenation of y with x or features by an inner-product interaction with an embedding matrix V.
• Use a ResNet-based discriminator and generator with spectral normalization and hinge loss for training.
• Apply conditional batch normalization in the generator to enable category morphing.
• Evaluate on ImageNet (1000-class) for class-conditional generation and on a super-resolution task, comparing against concatenation and AC-GANs.

Experimental results

Research questions

• RQ1 Does a projection-based discriminator that uses inner-product conditioning improve conditional image generation quality compared to concatenation?
• RQ2 Can the projection approach extend effectively to super-resolution and enable category morphing within the generator?
• RQ3 How does the projection discriminator perform relative to AC-GANs and concatenation across large-scale, multi-class datasets?
• RQ4 What is the impact on diversity and mode coverage (as measured by intra-class FID) when using projection vs concatenation?

Key findings

• Projection-based discriminators yield higher Inception Scores than concatenation and AC-GANs on ImageNet (AC-GANs: 28.5 ± .20; concat: 21.1 ± .35; projection: 29.7 ± .61; projection at 850K iterations: 36.8 ± .44).
• Projection achieves lower intra-class FID than both AC-GANs and concatenation (AC-GANs: 260.0; concat: 141.2; projection: 103.1; projection 850K: 92.4).
• On CIFAR-10/100, the projection method also outperformed alternative conditioning approaches (details in Appendix A).
• For super-resolution, projection yields higher Inception Accuracy (35.2) and MS-SSIM (0.878) than bicubic, bilinear, and concatenation baselines; 10-seed ensembles further improve inception accuracy to 36.4.
• Projection enables category morphing via interpolation of conditional batch normalization parameters, producing meaningful intermediate classes.
• Compared to AC-GANs, the projection model avoids mode-collapse and maintains diversity across generated samples.