[Paper Review] Versatile Auxiliary Classifier with Generative Adversarial Network (VAC+GAN)

Shabab Bazrafkan, Hossein Javidnia+1 more|arXiv (Cornell University)|May 1, 2018

Generative Adversarial Networks and Image Synthesis8 references3 citations

TL;DR

This paper proposes VAC+GAN, a versatile framework that enhances conditional GANs by adding a parallel classifier to back-propagate class-specific loss through the generator, enabling precise control over generated sample classes. Unlike prior methods, it works with any GAN architecture without altering loss functions, significantly improving class-specific generation quality and diversity, as validated on gender-specific face generation with superior metrics.

ABSTRACT

One of the most interesting challenges in Artificial Intelligence is to train conditional generators which are able to provide labeled adversarial samples drawn from a specific distribution. In this work, a new framework is presented to train a deep conditional generator by placing a classifier in parallel with the discriminator and back propagate the classification error through the generator network. The method is versatile and is applicable to any variations of Generative Adversarial Network (GAN) implementation, and also gives superior results compared to similar methods.

Motivation & Objective

• To address the limitation of existing conditional GANs in generating distinct, class-specific samples across diverse GAN architectures.
• To overcome the instability and convergence issues in methods like ACGAN and CGAN when merging discriminator and classifier losses, especially when outputs differ in dimensionality.
• To develop a unified, plug-and-play solution compatible with any converging GAN framework, regardless of discriminator or generator structure.
• To mathematically prove the effectiveness of the proposed classifier-backpropagation mechanism in improving class separation and diversity in generated samples.
• To empirically validate the method on gender-specific face generation, demonstrating superior performance over baseline conditional GANs.

Proposed method

• A parallel classifier network is added to the generator's output, decoupling the classification loss from the discriminator's loss function.
• The classifier's binary cross-entropy loss is back-propagated through the generator, enabling class-specific optimization without modifying the original GAN's training objective.
• The method is mathematically proven to converge to an optimal classifier that maximizes class separation, with the optimal classifier being $ C_{G,D}^{*} = \frac{p_{X_1}(\mathbf{x})}{p_{X_1}(\mathbf{x}) + p_{X_2}(\mathbf{x})} $.
• The framework is applied to BEGAN, where the generator loss combines the BEGAN objective with the classifier's BCE loss, and the discriminator loss uses the standard BEGAN loss.
• The classifier is trained with Nesterov momentum, while the generator and discriminator use ADAM optimizer with specific hyperparameters (lr=0.0001, β₁=0.5, β₂=0.999).
• The latent space is partitioned into two disjoint subspaces (e.g., for male/female faces), with the first dimension used to condition the class.

Experimental results

Research questions

• RQ1 Can a classifier be decoupled from the discriminator’s loss in GANs to improve training stability and class-specific generation without modifying the original GAN architecture?
• RQ2 Does back-propagating classification error through the generator enhance the Jensen-Shannon divergence between generated classes, leading to better class separation?
• RQ3 Can the proposed method be universally applied to any GAN variant, regardless of the discriminator’s output type (e.g., scalar vs. image output)?
• RQ4 How does the VAC+GAN framework compare to CGAN and ACGAN in generating diverse, class-specific samples on a multi-class generation task?
• RQ5 To what extent does the method improve inter-class dissimilarity and intra-class diversity in generated samples?

Key findings

• The VAC+GAN framework successfully generates gender-specific face samples with high fidelity and distinct class characteristics, unlike the baseline CBEGAN method which fails to maintain class specificity.
• The proposed method achieves lower UQI and SSIM scores for intra-class comparisons (blue and purple bars in Figure 5), indicating reduced similarity among samples of the same class.
• Inter-class similarity (yellow bars in Figure 5) is significantly lower in VAC+GAN than in CBEGAN, confirming that samples from different classes are more distinguishable.
• The proposed method produces higher MSE, RMSE, and MAE values (Figure 6), demonstrating greater variation in generated images both within and across classes.
• The mathematical proof shows that the classifier loss is maximized when class distributions are balanced, ensuring optimal class discrimination.
• The method is proven to be universally applicable to any GAN framework, including those with non-scalar discriminator outputs, due to the decoupled classifier training.