[Paper Review] Dropout-GAN: Learning from a Dynamic Ensemble of Discriminators

Gonçalo Mordido, Haojin Yang+1 more|arXiv (Cornell University)|Jul 30, 2018

Generative Adversarial Networks and Image Synthesis36 references48 citations

TL;DR

Dropout-GAN trains a single generator against a dynamically changing ensemble of discriminators by dropping out discriminator feedback, improving sample diversity and mitigating mode collapse across multiple datasets.

ABSTRACT

We propose to incorporate adversarial dropout in generative multi-adversarial networks, by omitting or dropping out, the feedback of each discriminator in the framework with some probability at the end of each batch. Our approach forces the single generator not to constrain its output to satisfy a single discriminator, but, instead, to satisfy a dynamic ensemble of discriminators. We show that this leads to a more generalized generator, promoting variety in the generated samples and avoiding the common mode collapse problem commonly experienced with generative adversarial networks (GANs). We further provide evidence that the proposed framework, named Dropout-GAN, promotes sample diversity both within and across epochs, eliminating mode collapse and stabilizing training.

Motivation & Objective

• Motivate and address mode collapse in GANs by leveraging multiple discriminators.
• Introduce adversarial dropout to create a dynamic ensemble that changes per batch.
• Demonstrate that varied feedback leads to more diverse generated samples and improved stability.
• Show extensibility of adversarial dropout across multiple GAN variants and datasets.

Proposed method

• Extend the GAN framework to a set of K discriminators that provide feedback to a single generator.
• Apply dropout to the feedback from each discriminator with probability d, so the generator updates use only a random subset of discriminators.
• If all discriminators are dropped, update the generator using a randomly chosen single discriminator to preserve learning signal.
• Maintain each discriminator independently trained with its own gradient updates.
• Evaluate the approach with FID, Inception Score, and qualitative analysis across MNIST, CIFAR-10, and CelebA, varying K and d.

Experimental results

Research questions

• RQ1 Does adversarial dropout over a dynamic ensemble of discriminators reduce mode collapse in GANs?
• RQ2 How do the number of discriminators (K) and dropout rate (d) affect training stability and sample diversity?
• RQ3 Is Dropout-GAN helpful as a general augmentation to existing GAN variants (e.g., LSGAN, DRAGAN, modGAN) across datasets of varying sizes?

Key findings

• Adversarial dropout improves sample diversity and reduces mode collapse across MNIST, CIFAR-10, and CelebA.
• Mid-range dropout rates (d ≈ 0.2–0.5) with larger discriminator ensembles yield the best Fréchet Inception Distance (FID) on several datasets.
• Using multiple discriminators generally outperforms a static single discriminator in terms of FID and stability.
• Dropout-GAN improves several GAN variants (LSGAN, DRAGAN, modGAN) on multiple datasets according to reported metrics.
• The approach yields higher Inception Scores on CIFAR-10, STL-10, and ImageNet when using adversarial dropout with two discriminators.