[Paper Review] ContraGAN: Contrastive Learning for Conditional Image Generation

Minguk Kang, Jaesik Park|arXiv (Cornell University)|Jun 23, 2020

Generative Adversarial Networks and Image SynthesisComputer Science69 references77 citations

TL;DR

ContraGAN introduces a conditional contrastive loss (2C loss) that leverages data-to-data and data-to-class relations in a batch, yielding stronger conditional image generation and reduced discriminator overfitting, with notable FID improvements on Tiny ImageNet and ImageNet.

ABSTRACT

Conditional image generation is the task of generating diverse images using class label information. Although many conditional Generative Adversarial Networks (GAN) have shown realistic results, such methods consider pairwise relations between the embedding of an image and the embedding of the corresponding label (data-to-class relations) as the conditioning losses. In this paper, we propose ContraGAN that considers relations between multiple image embeddings in the same batch (data-to-data relations) as well as the data-to-class relations by using a conditional contrastive loss. The discriminator of ContraGAN discriminates the authenticity of given samples and minimizes a contrastive objective to learn the relations between training images. Simultaneously, the generator tries to generate realistic images that deceive the authenticity and have a low contrastive loss. The experimental results show that ContraGAN outperforms state-of-the-art-models by 7.3% and 7.7% on Tiny ImageNet and ImageNet datasets, respectively. Besides, we experimentally demonstrate that contrastive learning helps to relieve the overfitting of the discriminator. For a fair comparison, we re-implement twelve state-of-the-art GANs using the PyTorch library. The software package is available at https://github.com/POSTECH-CVLab/PyTorch-StudioGAN.

Motivation & Objective

• Motivate conditioning in GANs beyond data-to-class losses by incorporating data-to-data relations.
• Propose a novel 2C (two-way) contrastive loss that jointly uses image embeddings and class embeddings.
• Integrate 2C loss into a GAN framework (ContraGAN) to improve realism and diversity while stabilizing training.
• Evaluate ContraGAN on CIFAR10, Tiny ImageNet, and ImageNet against state-of-the-art conditional GANs and large-scale GANs.

Proposed method

• Introduce a conditional contrastive loss (2C loss) that uses both data-to-class and data-to-data relations within a minibatch.
• Define an encoder S and projection head h to map images to embeddings on a unit hypersphere.
• Replace or augment standard conditioning losses with 2C loss that pulls same-class image embeddings together and pushes others apart, while also aligning embeddings with their class embeddings e(y).
• Formulate l(xi, yi; t) as a contrastive objective that includes a term for same-class negatives, enabling data-to-data and data-to-class supervision.
• Train a discriminator with an auxiliary 2C loss on real samples and incorporate a 2C loss on generated samples to guide the generator.
• Demonstrate that 2C loss reduces discriminator overfitting and improves sample quality across backbone architectures (DCGAN, ResGAN, BigGAN) and datasets.

Experimental results

Research questions

• RQ1 Can a contrastive objective that exploits data-to-data relations within a batch improve conditional image generation beyond traditional data-to-class conditioning?
• RQ2 Does the proposed 2C loss stabilize GAN training and reduce discriminator overfitting on large-scale conditional generation tasks?
• RQ3 What are the empirical gains in generation quality (FID) when integrating 2C loss on CIFAR10, Tiny ImageNet, and ImageNet across common backbones?

Key findings

• ContraGAN improves state-of-the-art results by 7.3% on Tiny ImageNet and 7.7% on ImageNet in terms of FID.
• On CIFAR10, ContraGAN achieves a 1.3% lower FID compared to the art model baseline in Table 3.
• ContraGAN can reduce discriminator overfitting and shows more stable training compared to ProjGAN under similar conditions.
• In ablations, 2C loss significantly reduces FID versus baselines, and benefits from larger batch sizes and consistency regularization when present.
• ContraGAN’s 2C loss explicitly leverages data-to-data relations without requiring hard negative mining or extensive augmentations.