[Paper Review] Image Augmentations for GAN Training

Zhengli Zhao, Zizhao Zhang+3 more|arXiv (Cornell University)|Jun 4, 2020

Advanced Neural Network Applications47 references116 citations

TL;DR

This paper systematically studies image augmentations for GAN training and shows that augmenting both real and generated images, especially with consistency and contrastive regularizations, yields substantial gains in generation quality and state-of-the-art results on CIFAR-10.

ABSTRACT

Data augmentations have been widely studied to improve the accuracy and robustness of classifiers. However, the potential of image augmentation in improving GAN models for image synthesis has not been thoroughly investigated in previous studies. In this work, we systematically study the effectiveness of various existing augmentation techniques for GAN training in a variety of settings. We provide insights and guidelines on how to augment images for both vanilla GANs and GANs with regularizations, improving the fidelity of the generated images substantially. Surprisingly, we find that vanilla GANs attain generation quality on par with recent state-of-the-art results if we use augmentations on both real and generated images. When this GAN training is combined with other augmentation-based regularization techniques, such as contrastive loss and consistency regularization, the augmentations further improve the quality of generated images. We provide new state-of-the-art results for conditional generation on CIFAR-10 with both consistency loss and contrastive loss as additional regularizations.

Motivation & Objective

• Assess the efficacy of a broad set of image augmentation operations in GAN training.
• Determine whether augmenting only real images or both real and generated images affects GAN performance.
• Evaluate the impact of augmentation-driven regularizations such as consistency regularization and contrastive loss on GANs.
• Identify augmentation types and strengths that most improve GAN generation quality across architectures.

Proposed method

• Evaluate 10 basic and 3 advanced image augmentations on CIFAR-10 using SNDCGAN (unconditional) and BigGAN (conditional).
• Compare augmenting only real images versus augmenting both real and generated images before feeding to the discriminator.
• Apply hinge loss and report Fréchet Inception Distance (FID) with multiple random seeds for robustness.
• Incorporate consistency regularization (CR) and balanced CR (BCR) on augmented data to assess additive gains.
• Introduce contrastive loss (Cntr) on augmented copies of real and fake images to examine complementary regularization effects.
• Combine Cntr with BCR to achieve state-of-the-art FID for CIFAR-10 conditional generation.

Experimental results

Research questions

• RQ1 Do augmentations improve GAN performance when applied only to real images versus both real and generated images?
• RQ2 Which augmentation types (spatial vs visual) most improve GAN generation quality?
• RQ3 How do consistency regularization and contrastive loss interact with augmentation strategies in GANs?
• RQ4 Can augmentations combined with regularizations achieve state-of-the-art results on CIFAR-10?

Key findings

• Augmenting only real images is ineffective for vanilla GAN training and can worsen FID.
• Augmenting both real and generated images consistently improves GAN performance across architectures.
• Spatial augmentations (e.g., translation, zoom) outperform purely visual augmentations in improving FID.
• Consistency regularization on augmented data provides substantial gains, often outperforming basic CR variants.
• Combining contrastive loss with consistency regularization and augmentation achieves new state-of-the-art FID on CIFAR-10 conditional generation.
• For CIFAR-10, augmentation plus regularizations can match or exceed previous CR-GAN methods without augmentation.