[Paper Review] Metropolis-Hastings with Averaged Acceptance Ratios

Christophe Andrieu, Sinan Yıldırım+2 more|arXiv (Cornell University)|Dec 29, 2020

Markov Chains and Monte Carlo Methods29 references4 citations

TL;DR

This paper introduces Metropolis-Hastings with Averaged Acceptance Ratios (MHAAR), a novel MCMC framework that improves convergence to equilibrium and reduces variance in ergodic averages by averaging acceptance ratios across multiple independent proposals. The method preserves the target distribution as invariant, enables efficient parallelization, and demonstrates significant performance gains—especially in reducing burn-in—across diverse models including state-space models, transdimensional samplers, and doubly intractable targets.

ABSTRACT

Markov chain Monte Carlo (MCMC) methods to sample from a probability distribution $π$ defined on a space $(Θ,\mathcal{T})$ consist of the simulation of realisations of Markov chains $\{θ_{n},n\geq1\}$ of invariant distribution $π$ and such that the distribution of $θ_{i}$ converges to $π$ as $i ightarrow\infty$. In practice one is typically interested in the computation of expectations of functions, say $f$, with respect to $π$ and it is also required that averages $M^{-1}\sum_{n=1}^{M}f(θ_{n})$ converge to the expectation of interest. The iterative nature of MCMC makes it difficult to develop generic methods to take advantage of parallel computing environments when interested in reducing time to convergence. While numerous approaches have been proposed to reduce the variance of ergodic averages, including averaging over independent realisations of $\{θ_{n},n\geq1\}$ simulated on several computers, techniques to reduce the "burn-in" of MCMC are scarce. In this paper we explore a simple and generic approach to improve convergence to equilibrium of existing algorithms which rely on the Metropolis-Hastings (MH) update, the main building block of MCMC. The main idea is to use averages of the acceptance ratio w.r.t. multiple realisations of random variables involved, while preserving $π$ as invariant distribution. The methodology requires limited change to existing code, is naturally suited to parallel computing and is shown on our examples to provide substantial performance improvements both in terms of convergence to equilibrium and variance of ergodic averages. In some scenarios gains are observed even on a serial machine.

Motivation & Objective

• Address the lack of generic, effective methods to reduce MCMC burn-in periods, especially in complex models with intractable likelihoods.
• Improve convergence to equilibrium and reduce variance in ergodic averages for existing MH-based algorithms without altering the target invariant distribution.
• Enable efficient parallelization of MCMC by decoupling the computation of multiple acceptance ratio estimates across independent proposals.
• Provide a general framework applicable to a wide range of models, including state-space models, transdimensional models, and doubly intractable distributions.
• Develop a computationally efficient alternative to pseudo-marginal MCMC that avoids the high cost of exact likelihood estimation.

Proposed method

• Propose a generalized MH algorithm (MHAAR) that uses the average of multiple independent acceptance ratio estimators instead of a single one.
• Use multiple i.i.d. proposals from the same proposal kernel to compute multiple acceptance ratios, then average them to form a single acceptance probability.
• Preserve the target distribution π as the invariant distribution by ensuring detailed balance through proper construction of the acceptance ratio averaging mechanism.
• Introduce MHAAR-RB (rejection-based) and MHAAR-S (subsampled) variants for state-space models using conditional SMC (cSMC) to estimate the acceptance ratio unbiasedly.
• Leverage backward sampling in cSMC to generate multiple latent path estimates, which are used to compute unbiased estimators of the acceptance ratio.
• Apply subsampling to reduce computational cost from O(M²T) to O(NMT) per iteration by using only N out of M×T possible paths in the acceptance ratio estimator.

Experimental results

Research questions

• RQ1 Can averaging multiple independent acceptance ratio estimates improve the convergence to equilibrium in MH algorithms without altering the target distribution?
• RQ2 How can the Metropolis-Hastings algorithm be modified to allow for parallel computation of acceptance ratios while maintaining detailed balance?
• RQ3 To what extent can MHAAR reduce the burn-in period and variance of ergodic averages in complex models such as state-space models and transdimensional samplers?
• RQ4 Can MHAAR serve as an efficient alternative to pseudo-marginal MCMC with reduced computational cost and improved mixing properties?
• RQ5 What is the impact of subsampling latent path estimates on the performance and computational cost of MHAAR in large-scale state-space models?

Key findings

• MHAAR reduces the integrated autocorrelation time (IAC) by up to 70% compared to standard MwPG in a hidden Markov model, with IAC dropping from 3.5337×10³ to 1.4047×10³ when N=60.
• The MHAAR-RB-R variant achieves an IAC of 0.4332×10³, significantly lower than MwPG’s 3.5337×10³, indicating faster convergence and reduced variance.
• Even on a serial machine, MHAAR shows performance gains over standard MCMC, demonstrating its effectiveness beyond parallel architectures.
• The MHAAR-S variant reduces computational cost from O(M²T) to O(NMT), enabling scalable application to large-scale state-space models with massive latent spaces.
• Ensemble averages from MHAAR-S converge faster to the posterior mean than MwPG and MHAAR-RB, as shown in Figure 7, confirming improved mixing and reduced burn-in.
• The method maintains exact invariance to the target distribution π, ensuring validity despite the use of averaged and unbiased acceptance ratio estimators.