[Paper Review] Sequential Markov Chain Monte Carlo

Yun Yang, David B. Dunson|arXiv (Cornell University)|Aug 18, 2013

Bayesian Methods and Mixture Models6 references18 citations

TL;DR

This paper proposes Sequential Markov Chain Monte Carlo (SMCMC), a population-based MCMC method that updates posterior distributions sequentially as new data arrive, avoiding particle degeneracy common in sequential Monte Carlo (SMC). It ensures marginal convergence with theoretical guarantees and enables real-time inference with stable mixing, even in high-dimensional or nonparametric models.

ABSTRACT

We propose a sequential Markov chain Monte Carlo (SMCMC) algorithm to sample from a sequence of probability distributions, corresponding to posterior distributions at different times in on-line applications. SMCMC proceeds as in usual MCMC but with the stationary distribution updated appropriately each time new data arrive. SMCMC has advantages over sequential Monte Carlo (SMC) in avoiding particle degeneracy issues. We provide theoretical guarantees for the marginal convergence of SMCMC under various settings, including parametric and nonparametric models. The proposed approach is compared to competitors in a simulation study. We also consider an application to on-line nonparametric regression.

Motivation & Objective

• Address the challenge of efficient Bayesian inference in on-line, streaming data settings where full data MCMC is computationally prohibitive.
• Overcome particle degeneracy in sequential Monte Carlo (SMC) by using a population-based MCMC approach with adaptive resampling and annealing.
• Provide theoretical guarantees for marginal convergence of SMCMC under both parametric and nonparametric models.
• Enable real-time monitoring and inference by distributing computational burden over time, unlike batch MCMC.
• Develop a scalable framework for nonparametric regression and dynamic models with time-varying dimensionality.

Proposed method

• Propose a sequential MCMC algorithm that maintains a population of samples (ensemble) and updates them at each time step using a transition kernel $T_t$ and a jumping kernel $J_t$.
• Use a hybrid transition kernel combining Gibbs sampling and Metropolis-Hastings steps to improve mixing, especially in high-dimensional or complex posterior spaces.
• Introduce a sequential updating scheme where each new data point triggers a resampling and rejuvenation step to maintain sample diversity.
• Apply annealing via a sequence of tempered targets to enhance exploration in multimodal or high-energy regions.
• Leverage parallel computation to maintain computational efficiency comparable to standard MCMC, despite sequential data arrival.
• Use a forgetting mechanism or sliding window strategy to reduce long-term memory burden while preserving convergence to the true posterior.

Experimental results

Research questions

• RQ1 Can a sequential MCMC framework be designed to avoid particle degeneracy while maintaining theoretical convergence in on-line Bayesian inference?
• RQ2 How does the mixing behavior of the transition kernel $T_t$ affect the computational burden and convergence rate of SMCMC as data accumulate?
• RQ3 To what extent can SMCMC achieve comparable accuracy to batch MCMC with significantly reduced per-iteration cost and real-time adaptability?
• RQ4 Can SMCMC be effectively applied to nonparametric models with increasing dimensionality, such as nonparametric probit regression?
• RQ5 How do numerical errors from early stages propagate, and can they be controlled through sequential updating and annealing?

Key findings

• SMCMC achieves marginal convergence to the true posterior distribution under mild regularity conditions, with explicit error bounds depending on mixing and tempering parameters.
• The method avoids particle degeneracy by maintaining a diverse ensemble of samples through resampling and transition kernels, unlike SMC which suffers from weight degeneracy.
• In the nonparametric probit regression example, the number of iterations $m_t$ stabilizes around 150–200 for $t > 100$, indicating robust mixing despite increasing sample size.
• The total computational cost is $O(n^3)$, matching batch MCMC, but the work is distributed over time, enabling real-time inference and monitoring.
• Fitted probability contours for hypertension risk stabilize by $t=350$, showing minimal change by $t=462$, indicating convergence of posterior estimates.
• Results from SMCMC are indistinguishable from those of a full batch MCMC run, validating its accuracy and reliability in practice.