[Paper Review] Filtering Variational Objectives

Chris J. Maddison, Dieterich Lawson+6 more|arXiv (Cornell University)|May 25, 2017

Gaussian Processes and Bayesian Inference53 references3 citations

TL;DR

This paper introduces Filtering Variational Objectives (FIVOs), a family of tighter lower bounds for maximum likelihood estimation in sequential latent variable models by leveraging particle filter-based estimators of marginal likelihood. FIVO outperforms ELBO and IWAE in training deep generative models on sequential data, achieving significantly higher log-likelihoods with improved variational posterior quality and reduced posterior collapse.

ABSTRACT

When used as a surrogate objective for maximum likelihood estimation in latent variable models, the evidence lower bound (ELBO) produces state-of-the-art results. Inspired by this, we consider the extension of the ELBO to a family of lower bounds defined by a particle filter's estimator of the marginal likelihood, the filtering variational objectives (FIVOs). FIVOs take the same arguments as the ELBO, but can exploit a model's sequential structure to form tighter bounds. We present results that relate the tightness of FIVO's bound to the variance of the particle filter's estimator by considering the generic case of bounds defined as log-transformed likelihood estimators. Experimentally, we show that training with FIVO results in substantial improvements over training the same model architecture with the ELBO on sequential data.

Motivation & Objective

• Address the limitation of the Evidence Lower Bound (ELBO) in capturing complex posterior dependencies in sequential models.
• Improve the tightness of variational bounds for maximum likelihood estimation in models with sequential structure.
• Develop a new class of objectives—Filtering Variational Objectives (FIVOs)—that exploit sequential modeling structure to yield tighter bounds than ELBO and IWAE.
• Investigate the relationship between the variance of particle filter estimators and the tightness of the resulting variational bounds.
• Demonstrate empirically that FIVO leads to better model performance on sequential data, including reduced posterior collapse and higher log-likelihoods.

Proposed method

• Define FIVOs as the log of a particle filter’s marginal likelihood estimator, forming a lower bound on the true log-likelihood.
• Use sequential importance sampling with resampling to construct a particle filter that estimates the marginal likelihood efficiently.
• Formulate FIVO as a Monte Carlo objective (MCO), where the bound is derived from the log of an unbiased estimator of the marginal likelihood.
• Optimize FIVO via stochastic gradient ascent using unbiased gradient estimators, including a novel gradient estimator for the resampling step.
• Introduce a smoothing variant of the VRNN that allows the variational posterior to condition on future observations, enabling comparison with sharp objectives like ELBO and IWAE.
• Apply control variates and variance reduction techniques to stabilize training, particularly for the resampling gradient component.

Experimental results

Research questions

• RQ1 Can particle filter-based estimators of marginal likelihood yield tighter variational bounds than ELBO and IWAE in sequential models?
• RQ2 How does the variance of the particle filter’s likelihood estimator affect the tightness of the resulting FIVO bound?
• RQ3 Does FIVO training lead to better generalization and higher log-likelihoods compared to ELBO and IWAE on sequential data?
• RQ4 Why do models trained with FIVO avoid posterior collapse more effectively than those trained with ELBO?
• RQ5 Can FIVO be optimized stably despite the use of biased gradients in the resampling step, and what are the empirical consequences?

Key findings

• Models trained with FIVO achieved significantly higher marginal log-likelihoods than those trained with ELBO or IWAE across four music modeling datasets and the TIMIT speech dataset.
• On the JSB Chorales dataset, FIVO achieved a train set log-likelihood of -4.08 nats/timestep, outperforming ELBO (-5.48) and IWAE (-5.77).
• On TIMIT, FIVO achieved a relative log-likelihood improvement of 6,991 nats per sequence over non-smoothed ELBO, with the best performance among all methods.
• Models trained with FIVO maintained high KL divergence from the prior, indicating reduced posterior collapse, while ELBO-trained models showed strong posterior collapse.
• Training without the resampling gradient term led to faster convergence and better performance than with it, suggesting that the resampling gradient may be harmful or highly variable.
• Smoothing the variational posterior to include future information did not improve FIVO performance, unlike for IWAE, indicating FIVO’s bound is already tight and robust.