[Paper Review] Adversarially Learned Inference

Vincent Dumoulin, Ishmael Belghazi+5 more|arXiv (Cornell University)|Jun 2, 2016

Adversarial Robustness in Machine Learning38 references692 citations

TL;DR

ALI jointly trains an encoder and a decoder in an adversarial framework to learn mutually coherent inference and generation networks, enabling competitive semi-supervised performance while preserving high-fidelity samples.

ABSTRACT

We introduce the adversarially learned inference (ALI) model, which jointly learns a generation network and an inference network using an adversarial process. The generation network maps samples from stochastic latent variables to the data space while the inference network maps training examples in data space to the space of latent variables. An adversarial game is cast between these two networks and a discriminative network is trained to distinguish between joint latent/data-space samples from the generative network and joint samples from the inference network. We illustrate the ability of the model to learn mutually coherent inference and generation networks through the inspections of model samples and reconstructions and confirm the usefulness of the learned representations by obtaining a performance competitive with state-of-the-art on the semi-supervised SVHN and CIFAR10 tasks.

Motivation & Objective

• Motivate and develop a deep generative framework that jointly learns generation and inference networks.
• Bridge GANs and VAEs by enabling efficient, expressive inference within adversarial training.
• Show that learned latent representations are useful for semi-supervised tasks on SVHN and CIFAR-10.

Proposed method

• Define encoder q(x,z) = q(x) q(z|x) and decoder p(x,z) = p(z) p(x|z) and train a discriminator to distinguish joint samples from these two distributions.
• Use an adversarial objective to minimize the Jensen-Shannon divergence between q(x,z) and p(x,z) under an optimal discriminator.
• Employ reparameterization tricks to propagate gradients through stochastic encoders/decoders.
• Provide a conditional generation extension to match p(y) and q(x,y), enabling conditioning on attributes (e.g., CelebA).
• Compare joint inference in ALI to alternatives like InfoGAN and post-hoc learned inference.

Experimental results

Research questions

• RQ1 Can jointly learned inference and generation in an adversarial setting produce coherent encoder and decoder mappings?
• RQ2 Do the learned latent representations support competitive semi-supervised learning on standard benchmarks?
• RQ3 How does ALI compare to alternative inference strategies (e.g., InfoGAN, post-hoc inference) in terms of mode coverage and representation quality?
• RQ4 What is the effect of conditioning on auxiliary information on the learned model behavior?

Key findings

• ALI produces high-fidelity samples comparable to GANs while enabling an inference mechanism to map data to latent variables.
• Latent interpolations in ALI show smooth transitions between data points, indicating coherent latent structure.
• ALI-inferred features yield competitive SVHN and CIFAR-10 semi-supervised results, often surpassing prior GAN-based approaches without feature matching.
• Conditional ALI demonstrates ability to control generated outputs via observed attributes (e.g., CelebA attributes).
• An experiment comparing ALI to GAN-based inference shows joint learning of inference and generation improves mode coverage relative to post-hoc or inverted mappings.