[Paper Review] Fast mixing of Metropolized Hamiltonian Monte Carlo: Benefits of multi-step gradients

Yuansi Chen, Raaz Dwivedi+2 more|arXiv (Cornell University)|May 29, 2019

Markov Chains and Monte Carlo MethodsMathematics58 references19 citations

TL;DR

This paper provides non-asymptotic mixing time bounds for Metropolized Hamiltonian Monte Carlo (HMC) with multi-step leapfrog integrators, demonstrating faster convergence than Metropolized random walk or Langevin algorithms. It establishes that Metropolized HMC achieves logarithmic dependence on error tolerance, making it superior for estimating high-order moments in high-dimensional, strongly log-concave distributions.

ABSTRACT

Hamiltonian Monte Carlo (HMC) is a state-of-the-art Markov chain Monte Carlo sampling algorithm for drawing samples from smooth probability densities over continuous spaces. We study the variant most widely used in practice, Metropolized HMC with the Störmer-Verlet or leapfrog integrator, and make two primary contributions. First, we provide a non-asymptotic upper bound on the mixing time of the Metropolized HMC with explicit choices of step-size and number of leapfrog steps. This bound gives a precise quantification of the faster convergence of Metropolized HMC relative to simpler MCMC algorithms such as the Metropolized random walk, or Metropolized Langevin algorithm. Second, we provide a general framework for sharpening mixing time bounds of Markov chains initialized at a substantial distance from the target distribution over continuous spaces. We apply this sharpening device to the Metropolized random walk and Langevin algorithms, thereby obtaining improved mixing time bounds from a non-warm initial distribution.

Motivation & Objective

• To provide a non-asymptotic upper bound on the mixing time of Metropolized HMC with explicit step-size and leapfrog step choices.
• To develop a general framework for sharpening mixing time bounds from non-warm initial distributions in continuous-state Markov chains.
• To compare the gradient complexity of Metropolized HMC with unadjusted HMC and ODE-based HMC variants in terms of error tolerance and dimensionality.
• To quantify the advantage of Metropolized HMC for estimating high-order moments, where its logarithmic error dependence outperforms polynomial dependencies in unadjusted methods.

Proposed method

• Derives a non-asymptotic mixing time bound for Metropolized HMC using a leapfrog integrator with Metropolis-Hastings correction.
• Introduces a general sharpening technique for mixing time bounds when the initial distribution is far from the target, applicable to various MCMC algorithms.
• Applies the sharpening framework to Metropolized random walk and Langevin algorithms to improve their mixing time bounds.
• Analyzes the number of gradient evaluations required to achieve a given estimation error for different MCMC samplers, using Wasserstein and total variation distance metrics.
• Compares gradient complexity across unadjusted HMC, ODE-based HMC, and Metropolized HMC, focusing on error tolerance and dimension dependence.
• Uses Lipschitz and bounded function assumptions to derive error bounds in terms of Wasserstein and total variation distances, linking them to estimation error in Monte Carlo sampling.

Experimental results

Research questions

• RQ1 How does the mixing time of Metropolized HMC scale with dimension and error tolerance compared to simpler MCMC methods?
• RQ2 What is the impact of multi-step leapfrog integrators on the convergence rate of Metropolized HMC?
• RQ3 Can a general framework be developed to improve mixing time bounds when the initial distribution is far from the target?
• RQ4 How does the gradient complexity of Metropolized HMC compare to unadjusted HMC and ODE-based HMC for estimating high-order moments?
• RQ5 What is the dependence of the required number of gradient evaluations on the error tolerance and dimension for different MCMC samplers?

Key findings

• Metropolized HMC achieves a mixing time bound with logarithmic dependence on the error tolerance $\epsilon$, specifically $O(d \log(\omega \sqrt{d}/\delta))$ gradient evaluations for $\delta$-accurate estimation.
• For $\omega$-Lipschitz functions, the Wasserstein error bound is $J_{\text{Wass}} = \omega \epsilon$, and for bounded functions, the TV error is $J_{\text{TV}} = B\epsilon$, with $B = \omega R$.
• When estimating high-order moments $g(x) = \|x\|^{1+\nu}$, Metropolized HMC requires $O(d(1+\nu)\log(d/\delta))$ gradient evaluations, outperforming unadjusted HMC ($O(d^{(1+\nu)/2}/\sqrt{\delta})$) for $\nu > 1$.
• ODE-based HMC requires $O(\omega \sqrt{d}/\delta)$ evaluations, and Metropolized HMC is superior when $\nu > 1/2$ due to its logarithmic error dependence.
• The sharpening framework improves mixing time bounds for Metropolized random walk and Langevin algorithms when initialized far from the target distribution.
• The results show that Metropolized HMC's logarithmic error scaling provides a significant advantage over polynomial scaling in unadjusted HMC variants, especially for high-dimensional or high-moment estimation tasks.