[Paper Review] Measuring the non-asymptotic convergence of sequential Monte Carlo samplers using probabilistic programming

Marco Cusumano-Towner, Vikash K. Mansinghka|arXiv (Cornell University)|Dec 7, 2016

Bayesian Methods and Mixture Models8 references3 citations

TL;DR

This paper proposes a probabilistic programming-based meta-inference method to estimate upper bounds on the symmetric KL divergence between the output of sequential Monte Carlo (SMC) samplers and their target posterior distributions. By constructing a meta-sampler that generates execution histories of the SMC algorithm, the method enables non-asymptotic convergence analysis using unbiased Monte Carlo estimates, with tightness governed by the accuracy of the meta-inference distribution.

ABSTRACT

A key limitation of sampling algorithms for approximate inference is that it is difficult to quantify their approximation error. Widely used sampling schemes, such as sequential importance sampling with resampling and Metropolis-Hastings, produce output samples drawn from a distribution that may be far from the target posterior distribution. This paper shows how to upper-bound the symmetric KL divergence between the output distribution of a broad class of sequential Monte Carlo (SMC) samplers and their target posterior distributions, subject to assumptions about the accuracy of a separate gold-standard sampler. The proposed method applies to samplers that combine multiple particles, multinomial resampling, and rejuvenation kernels. The experiments show the technique being used to estimate bounds on the divergence of SMC samplers for posterior inference in a Bayesian linear regression model and a Dirichlet process mixture model.

Motivation & Objective

• To address the challenge of quantifying approximation error in sequential Monte Carlo (SMC) samplers, which often produce outputs far from the true posterior.
• To extend existing methods for bounding symmetric KL divergence to a broad class of SMC samplers, including those using multiple particles and MCMC rejuvenation kernels.
• To enable practical, non-asymptotic convergence assessment of SMC samplers using probabilistic programming frameworks.
• To improve the tightness of divergence bounds by leveraging accurate meta-inference samplers that approximate the conditional distribution of auxiliary variables given outputs.

Proposed method

• Construct a meta-inference sampler $ q(u;z) $ that generates execution histories $ u $ (auxiliary variables) of an SMC sampler given its output $ z $, enabling joint sampling from $ p(u,z) $ and $ q(u;z) $.
• Use unbiased Monte Carlo estimates of the symmetric KL divergence on the extended space via the estimator in Equation (2), which compares log-odds ratios of unnormalized posteriors and joint sampler densities.
• Generalize existing meta-inference constructions for AIS and SIR to handle generic SMC samplers with multinomial resampling, multiple particles, and MCMC rejuvenation kernels.
• Enable the use of cycles of detailed balance kernels in the SMC algorithm without altering the weight computation, by collapsing them into single steps and preserving the log-weight expression.
• Leverage probabilistic programming to automatically derive the joint probability $ p(u,z) $ and the proposal $ q(u;z) $, allowing end-to-end estimation of divergence bounds.
• Apply the method to real-world models, including Bayesian linear regression and Dirichlet process mixture models, to empirically validate the bounds.

Experimental results

Research questions

• RQ1 Can we estimate an upper bound on the symmetric KL divergence between the output of a generic SMC sampler and its target posterior using only access to the sampler and a gold-standard posterior sampler?
• RQ2 How does the accuracy of the meta-inference distribution $ q(u;z) $ affect the tightness of the divergence bound?
• RQ3 Can the proposed method be applied to SMC samplers that use MCMC rejuvenation kernels and multiple particles, beyond single-particle or annealing-based schemes?
• RQ4 Does the method remain valid when the SMC algorithm includes cycles of detailed balance kernels that do not individually satisfy detailed balance?
• RQ5 How can the method be integrated into probabilistic programming platforms to enable automated convergence diagnostics?

Key findings

• The method provides a non-asymptotic upper bound on the symmetric KL divergence between the SMC sampler output and the true posterior, using only the sampler and a gold-standard posterior sampler.
• The bound is estimated via an unbiased Monte Carlo estimator that requires sampling from the posterior $ \pi(z) $ and the SMC sampler $ p(u,z) $, along with the meta-inference distribution $ q(u;z) $.
• The tightness of the bound is directly governed by how well $ q(u;z) $ approximates the true conditional $ p(u|z) $, with better approximations yielding tighter bounds.
• The method generalizes prior work on AIS and SIR to full SMC samplers with multiple particles and MCMC rejuvenation, increasing its applicability to real-world models.
• The approach remains valid when SMC kernels include cycles of detailed balance, as long as the reverse cycle is used in the meta-inference, preserving the weight expression.
• Empirical evaluation on Bayesian linear regression and Dirichlet process mixture models confirms the method's feasibility and utility in practical inference settings.