[Paper Review] Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data

Liming Jiang, Bo Dai+2 more|arXiv (Cornell University)|Nov 12, 2021

Generative Adversarial Networks and Image Synthesis49 references28 citations

TL;DR

APA uses the generator to adaptively augment limited real data with its own fake images, deceiving the discriminator to curb overfitting and improve GAN training quality without external augmentations.

ABSTRACT

Generative adversarial networks (GANs) typically require ample data for training in order to synthesize high-fidelity images. Recent studies have shown that training GANs with limited data remains formidable due to discriminator overfitting, the underlying cause that impedes the generator's convergence. This paper introduces a novel strategy called Adaptive Pseudo Augmentation (APA) to encourage healthy competition between the generator and the discriminator. As an alternative method to existing approaches that rely on standard data augmentations or model regularization, APA alleviates overfitting by employing the generator itself to augment the real data distribution with generated images, which deceives the discriminator adaptively. Extensive experiments demonstrate the effectiveness of APA in improving synthesis quality in the low-data regime. We provide a theoretical analysis to examine the convergence and rationality of our new training strategy. APA is simple and effective. It can be added seamlessly to powerful contemporary GANs, such as StyleGAN2, with negligible computational cost.

Motivation & Objective

• Address the challenge of training GANs with limited data due to discriminator overfitting.
• Introduce Adaptive Pseudo Augmentation (APA) that uses generated images to deceive the discriminator adaptively.
• Provide theoretical analysis linking APA to JS divergence minimization and convergence.
• Demonstrate APA's effectiveness and compatibility across contemporary GANs like StyleGAN2 with negligible computational overhead.

Proposed method

• Auxiliary pseudo augmentation where fake samples from the generator are moderately injected into the real data as pseudo-real examples.
• Deception probability p controls the strength of augmentation and is adaptively updated based on an overfitting heuristic lambda derived from discriminator logits.
• Three lambda variants are proposed: lambda_r, lambda_f, lambda_rf, with lambda_r used in main experiments.
• APA leads to a modified minimax objective that links to minimizing a JS divergence between smoothed real data and generated data distributions.
• Provable convergence properties: with optimal G and D, APA preserves the global minimum at p_g = p_data, implying convergence of the generator under sufficient capacity.

Experimental results

Research questions

• RQ1 Can Adaptive Pseudo Augmentation (APA) improve GAN performance in the low-data regime without external augmentations?
• RQ2 Does APA preserve convergence properties and relate to JS divergence minimization under limited data?
• RQ3 How does APA interact with existing augmentation-based or regularization approaches (e.g., ADA, LC-Reg) in practice?
• RQ4 What is the empirical impact of APA across diverse datasets and data-scarce settings?

Key findings

• APA consistently improves synthesis quality (lower FID and higher IS) across AFHQ-Cat-5k, FFHQ-5k, Anime-5k, and CUB-12k compared to StyleGAN2 baselines.
• With as little as 1k–7k data, APA achieves significant FID/IS gains, sometimes approaching full-data StyleGAN2 performance.
• APA training incurs negligible computational overhead relative to ADA-based approaches.
• APA is complementary to standard data augmentations like ADA and can further boost performance when used together.
• Ablation studies show robustness to lambda variant choices and that adaptive deception (p) outperforms fixed or two-sided label strategies in most cases.
• Theoretically, APA transforms the objective toward minimizing JS divergence between smoothed real data and generated data, supporting convergence claims.