[Paper Review] Variational inference for Monte Carlo objectives

Andriy Mnih, Danilo Jimenez Rezende|arXiv (Cornell University)|Feb 22, 2016

Generative Adversarial Networks and Image SynthesisComputer Science20 references81 citations

TL;DR

This paper introduces VIMCO, the first unbiased gradient estimator for multi-sample variational inference objectives, which reduces variance by leveraging per-sample learning signals without additional parameters. It enables effective training of models with discrete latent variables using tighter Monte Carlo objectives, outperforming NVIL and matching biased estimators in practice while enabling end-to-end training of complex models.

ABSTRACT

Recent progress in deep latent variable models has largely been driven by the development of flexible and scalable variational inference methods. Variational training of this type involves maximizing a lower bound on the log-likelihood, using samples from the variational posterior to compute the required gradients. Recently, Burda et al. (2016) have derived a tighter lower bound using a multi-sample importance sampling estimate of the likelihood and showed that optimizing it yields models that use more of their capacity and achieve higher likelihoods. This development showed the importance of such multi-sample objectives and explained the success of several related approaches. We extend the multi-sample approach to discrete latent variables and analyze the difficulty encountered when estimating the gradients involved. We then develop the first unbiased gradient estimator designed for importance-sampled objectives and evaluate it at training generative and structured output prediction models. The resulting estimator, which is based on low-variance per-sample learning signals, is both simpler and more effective than the NVIL estimator proposed for the single-sample variational objective, and is competitive with the currently used biased estimators.

Motivation & Objective

• To address the high variance in gradient estimation when training models with discrete latent variables using multi-sample objectives.
• To develop an unbiased gradient estimator that avoids the need for learned baselines, unlike NVIL, for improved sample efficiency.
• To enable end-to-end training of generative and structured output prediction models using tighter, multi-sample variational bounds.
• To demonstrate that multi-sample objectives, when combined with effective gradient estimation, lead to better model capacity utilization and higher likelihoods.

Proposed method

• Proposes VIMCO, a novel unbiased gradient estimator for objectives based on averaging K independent samples from a proposal distribution.
• Uses per-sample learning signals to reduce gradient variance, eliminating the need for learned baselines as in NVIL.
• Applies the estimator to both generative models and structured output prediction tasks with discrete latent variables.
• Employs a proposal distribution that can be conditioned on input data, improving sample quality and reducing variance.
• Derives the gradient of the multi-sample lower bound using reparameterization and importance weighting, ensuring unbiased estimation.
• Evaluates the method on binarized MNIST for both generative modeling and conditional image completion tasks.

Experimental results

Research questions

• RQ1 Can an unbiased gradient estimator be designed for multi-sample variational objectives that avoids the high variance of naive estimators?
• RQ2 Does VIMCO outperform NVIL in training models with discrete latent variables under multi-sample objectives?
• RQ3 How does using a learned proposal distribution affect performance compared to using the prior in multi-sample objectives?
• RQ4 To what extent do multi-sample objectives improve model capacity utilization and likelihood in discrete latent variable models?

Key findings

• VIMCO consistently outperformed NVIL in training both generative and structured output prediction models, especially as the number of samples increased.
• With 50 samples, VIMCO achieved a negative log-likelihood of 46.1 nats on the structured output prediction task, surpassing the previous best result of 53.8 nats.
• Using a learned proposal distribution led to significantly better performance than using the prior, with the worst VIMCO result exceeding the best prior-based result.
• The performance gap between VIMCO and NVIL widened with more samples, indicating VIMCO's superior variance reduction.
• Models trained with VIMCO captured multimodal structures in conditional image completion tasks effectively, as shown in generated samples.
• VIMCO achieved competitive performance with biased estimators while being fully unbiased, enabling more reliable and stable training.