[Paper Review] Quasi Markov Chain Monte Carlo Methods

Tobias Schwedes, Ben Calderhead|arXiv (Cornell University)|Jun 29, 2018

Mathematical Approximation and Integration59 references3 citations

TL;DR

This paper introduces a novel parallel quasi-Markov chain Monte Carlo (QMCMC) framework that leverages completely uniformly distributed (CUD) numbers to achieve faster convergence rates in Bayesian inference. By combining multiple proposals, non-reversible transitions, adaptive kernels, and importance sampling, the method achieves mean squared error convergence close to $n^{-2}$, significantly outperforming standard MCMC's $n^{-1}$ rate in practical models.

ABSTRACT

Quasi-Monte Carlo (QMC) methods for estimating integrals are attractive since the resulting estimators typically converge at a faster rate than pseudo-random Monte Carlo. However, they can be difficult to set up on arbitrary posterior densities within the Bayesian framework, in particular for inverse problems. We introduce a general parallel Markov chain Monte Carlo (MCMC) framework, for which we prove a law of large numbers and a central limit theorem. In that context, non-reversible transitions are investigated. We then extend this approach to the use of adaptive kernels and state conditions, under which ergodicity holds. As a further extension, an importance sampling estimator is derived, for which asymptotic unbiasedness is proven. We consider the use of completely uniformly distributed (CUD) numbers within the above mentioned algorithms, which leads to a general parallel quasi-MCMC (QMCMC) methodology. We prove consistency of the resulting estimators and demonstrate numerically that this approach scales close to $n^{-2}$ as we increase parallelisation, instead of the usual $n^{-1}$ that is typical of standard MCMC algorithms. In practical statistical models we observe multiple orders of magnitude improvement compared with pseudo-random methods.

Motivation & Objective

• To develop a scalable, parallel MCMC framework that overcomes the slow convergence of standard pseudo-random MCMC methods.
• To extend quasi-Monte Carlo (QMC) methods to arbitrary posterior distributions beyond hierarchical or low-dimensional models.
• To establish theoretical guarantees—such as ergodicity and asymptotic unbiasedness—for adaptive and non-reversible MCMC using CUD sequences.
• To demonstrate numerically that the proposed method achieves convergence rates close to $n^{-2}$, matching traditional QMC, in non-trivial statistical models.

Proposed method

• Proposes a multiple proposal MCMC (MP-MCMC) framework that generates and evaluates multiple candidate states in each iteration, enabling parallelization.
• Introduces non-reversible transition kernels to improve mixing and reduce random walk behavior in the sampling process.
• Develops an adaptive kernel mechanism that updates proposal distributions based on past samples, ensuring ergodicity under regularity conditions.
• Derives an importance sampling estimator that assigns weights to all proposed points, removing the discontinuity from acceptance thresholds.
• Generalizes the framework to use completely uniformly distributed (CUD) sequences as driving random numbers, replacing pseudo-random numbers.
• Proves consistency and asymptotic unbiasedness of the resulting estimators under suitable regularity conditions, including the existence of a coupling region.

Experimental results

Research questions

• RQ1 Can quasi-Monte Carlo (QMC) methods be effectively applied to general, non-hierarchical posterior distributions in Bayesian inference?
• RQ2 Does using CUD sequences in a parallel multiple-proposal MCMC framework lead to faster convergence than standard MCMC?
• RQ3 Can importance sampling within MP-MCMC eliminate the discontinuity from acceptance thresholds and improve QMC performance?
• RQ4 Under what conditions does the use of adaptive kernels in CUD-driven MP-MCMC ensure ergodicity and consistency?
• RQ5 Is it possible to achieve convergence rates closer to $n^{-2}$ in MCMC using QMC techniques, even in high-dimensional, non-exchangeable models?

Key findings

• The proposed quasi-MCMC method achieves a mean squared error convergence rate close to $n^{-2}$, significantly faster than the standard $n^{-1}$ rate of pseudo-random MCMC.
• Numerical experiments show multiple orders of magnitude improvement in convergence speed compared to standard MCMC, even in non-hierarchical models with no explicit conditional distributions.
• The importance sampling variant of MP-MCMC enables consistent estimation using all proposed points and removes the discontinuity introduced by acceptance-rejection steps.
• Theoretical results establish ergodicity of the adaptive MP-MCMC algorithm and asymptotic unbiasedness of the importance sampling estimator under regularity conditions.
• The method is consistent when driven by CUD sequences, and the use of multiple proposals enhances the effectiveness of low-discrepancy sequences in covering the state space.
• Empirical results suggest that the coupling region condition, while sufficient for proof, may not be necessary, opening avenues for future consistency proofs based on contraction conditions.