[Paper Review] Variational Rejection Sampling

Aditya Grover, Ramki Gummadi+3 more|arXiv (Cornell University)|Apr 5, 2018

Gaussian Processes and Bayesian Inference30 references3 citations

TL;DR

This paper introduces Variational Rejection Sampling (VRS), a method that improves variational inference in latent variable models by applying a differentiable accept-reject test to filter low-likelihood samples from the variational posterior. By trading computation for statistical accuracy, VRS achieves state-of-the-art performance, reducing test negative log-likelihood by 3.71 nats over single-sample baselines and 0.21 nats over multi-sample methods on binarized MNIST using sigmoid belief networks.

ABSTRACT

Learning latent variable models with stochastic variational inference is challenging when the approximate posterior is far from the true posterior, due to high variance in the gradient estimates. We propose a novel rejection sampling step that discards samples from the variational posterior which are assigned low likelihoods by the model. Our approach provides an arbitrarily accurate approximation of the true posterior at the expense of extra computation. Using a new gradient estimator for the resulting unnormalized proposal distribution, we achieve average improvements of 3.71 nats and 0.21 nats over state-of-the-art single-sample and multi-sample alternatives respectively for estimating marginal log-likelihoods using sigmoid belief networks on the MNIST dataset.

Motivation & Objective

• To address high-variance gradient estimates in stochastic variational inference when the variational posterior poorly approximates the true posterior.
• To develop a method that improves posterior approximation without requiring explicit reparameterization or complex model changes.
• To enable a flexible trade-off between computational cost and statistical accuracy in posterior approximation.
• To derive unbiased gradient estimators for model parameters despite the unnormalized nature of the resampled proposal distribution.
• To empirically demonstrate superior performance over existing single-sample and multi-sample variational inference methods on benchmark generative modeling tasks.

Proposed method

• Introduces a two-step sampling process: first sample from the variational posterior, then apply a differentiable accept-reject test based on the model's likelihood.
• Uses a threshold parameter γ to control rejection rate, allowing interpolation from standard variational inference (γ → 1) to exact posterior sampling (γ → 0) in the limit.
• Derives an unbiased gradient estimator for model parameters using only unnormalized density estimates of the resampled proposal, avoiding explicit normalization.
• Employs a resampled proposal distribution Rθ,φ that implicitly modifies the original variational posterior to better approximate the true posterior.
• Applies importance sampling and resampling-based estimators (IS and RS) to evaluate the lower bound, with RS showing improved performance.
• Supports both importance sampling and resampling-based evaluation of the evidence lower bound (ELBO), with the latter using Monte Carlo estimates of the partition function.

Experimental results

Research questions

• RQ1 Can a differentiable rejection mechanism improve the quality of variational posterior approximation in latent variable models?
• RQ2 Does the proposed method achieve better marginal log-likelihood estimation than existing single-sample and multi-sample variational inference approaches?
• RQ3 Can the method provide a smooth trade-off between computational cost and statistical accuracy in posterior approximation?
• RQ4 Is it possible to derive unbiased gradient estimators for model parameters when the proposal distribution is unnormalized due to rejection sampling?
• RQ5 How does the performance of VRS compare to state-of-the-art methods like NVIL, REBAR, and VIMCO on standard benchmarks such as binarized MNIST?

Key findings

• VRS reduces test negative log-likelihood by 3.71 nats on average compared to state-of-the-art single-sample methods on binarized MNIST.
• VRS improves test log-likelihood by 0.21 nats on average over multi-sample baselines, even with fewer samples than the k=50 baseline.
• The resampled (RS) estimator consistently outperforms the importance-sampled (IS) estimator, indicating that the resampling step improves posterior approximation.
• Lowering the threshold parameter γ (e.g., from 0.95 to 0.9) improves performance at the cost of increased computation, confirming the controllable trade-off between accuracy and cost.
• VRS with γ=0.95 and γ=0.9 achieves better performance than multi-sample methods with k=50, demonstrating that fewer but higher-quality samples can outperform many low-quality ones.
• The method achieves state-of-the-art results on sigmoid belief networks for binarized MNIST, outperforming published baselines including REBAR, Concrete, and VIMCO.