[Paper Review] Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models

Aditya Grover, Manik Dhar+1 more|arXiv (Cornell University)|May 24, 2017

Generative Adversarial Networks and Image Synthesis73 citations

TL;DR

Flow-GAN enables exact likelihood evaluation for GAN-style generators by using an invertible normalizing-flow generator, enabling both MLE and adversarial training and a hybrid objective to balance sample quality with held-out likelihood.

ABSTRACT

Adversarial learning of probabilistic models has recently emerged as a promising alternative to maximum likelihood. Implicit models such as generative adversarial networks (GAN) often generate better samples compared to explicit models trained by maximum likelihood. Yet, GANs sidestep the characterization of an explicit density which makes quantitative evaluations challenging. To bridge this gap, we propose Flow-GANs, a generative adversarial network for which we can perform exact likelihood evaluation, thus supporting both adversarial and maximum likelihood training. When trained adversarially, Flow-GANs generate high-quality samples but attain extremely poor log-likelihood scores, inferior even to a mixture model memorizing the training data; the opposite is true when trained by maximum likelihood. Results on MNIST and CIFAR-10 demonstrate that hybrid training can attain high held-out likelihoods while retaining visual fidelity in the generated samples.

Motivation & Objective

• Motivate combining maximum likelihood estimation (MLE) with adversarial learning for generative models.
• Introduce Flow-GAN, a GAN with an invertible (normalizing flow) generator that allows exact likelihood evaluation.
• Empirically compare MLE, adversarial learning, and a hybrid objective on MNIST and CIFAR-10.
• Analyze limitations of AIS/KDE for likelihood estimation and relate results to generator Jacobian conditioning.

Proposed method

• Use a normalizing flow as the GAN generator to permit tractable likelihood computation via the change-of-variables formula.
• Enable exact evaluation of p_theta(x) and exact posterior inference over latent variables due to invertibility.
• Train Flow-GANs under both maximum likelihood and adversarial objectives (e.g., Wasserstein GAN).
• Propose a hybrid objective that combines the adversarial objective with a log-likelihood term weighted by lambda.
• Investigate the Jacobian of the generator to explain poor likelihoods under adversarial training and to motivate the hybrid approach.
• Evaluate on MNIST (NICE-based flow) and CIFAR-10 (Real-NVP based flow) using standard sample-quality metrics (MODE/Inception).

Experimental results

Research questions

• RQ1 How do maximum likelihood and adversarial training compare in Flow-GANs in terms of held-out log-likelihood and sample quality?
• RQ2 Can a hybrid objective achieve good both likelihood and sample fidelity on standard image datasets?
• RQ3 What explains the low likelihoods observed under adversarial training when exact likelihoods are computable?
• RQ4 Do simple baselines (e.g., Gaussian mixtures memorizing training data) challenge the notion that adversarially trained generators yield better likelihoods?

Key findings

• Adversarial training yields high sample quality but substantially worse held-out log-likelihoods than MLE.
• MLE-based Flow-GANs achieve much better log-likelihoods but produce less visually sharp samples than adversarially trained counterparts.
• A simple Gaussian mixture baseline can outperform adversarially trained Flow-GANs on both sample quality and held-out likelihood in CIFAR-10 (and MNIST trends hold).
• A hybrid objective combining adversarial training with a likelihood term can balance the two goals and, in MNIST, improve both likelihood and sample quality.
• Analysis of the generator Jacobian reveals that adversarial training leads to ill-conditioned Jacobians and small effective support, explaining low likelihoods; the hybrid objective mitigates this.