[Paper Review] Continuously tempered Hamiltonian Monte Carlo

Matthew M. Graham, Amos Storkey|arXiv (Cornell University)|Apr 11, 2017

Markov Chains and Monte Carlo Methods34 references3 citations

TL;DR

This paper proposes Continuously Tempered Hamiltonian Monte Carlo (CT-HMC), a method that augments the Hamiltonian Monte Carlo state space with a continuous temperature control variable to improve mixing in multimodal distributions and estimate normalizing constants. By enabling gradient-based updates of temperature alongside state variables, CT-HMC achieves efficient exploration and accurate normalizing constant estimation with minimal code changes to existing HMC implementations.

ABSTRACT

Hamiltonian Monte Carlo (HMC) is a powerful Markov chain Monte Carlo (MCMC) method for performing approximate inference in complex probabilistic models of continuous variables. In common with many MCMC methods, however, the standard HMC approach performs poorly in distributions with multiple isolated modes. We present a method for augmenting the Hamiltonian system with an extra continuous temperature control variable which allows the dynamic to bridge between sampling a complex target distribution and a simpler unimodal base distribution. This augmentation both helps improve mixing in multimodal targets and allows the normalisation constant of the target distribution to be estimated. The method is simple to implement within existing HMC code, requiring only a standard leapfrog integrator. We demonstrate experimentally that the method is competitive with annealed importance sampling and simulating tempering methods at sampling from challenging multimodal distributions and estimating their normalising constants.

Motivation & Objective

• To address the poor mixing of standard HMC in multimodal target distributions with isolated modes.
• To enable accurate estimation of the normalizing constant of complex target distributions, a key quantity for model comparison.
• To develop a simple, plug-and-play extension to existing HMC code that avoids discrete temperature schedules.
• To improve sampling efficiency and normalizing constant estimation compared to annealed importance sampling and simulated tempering.
• To leverage continuous temperature control for better integration with adaptive HMC methods like NUTS.

Proposed method

• Augments the standard HMC state space with a continuous inverse temperature variable β, allowing joint dynamics over state x and β.
• Defines a joint potential energy function that smoothly interpolates between a base distribution and the target distribution via β.
• Uses a standard leapfrog integrator to simulate Hamiltonian dynamics in the extended space, enabling gradient-based updates.
• Employs a Metropolis-Hastings accept-reject step on the joint space to ensure detailed balance and correct stationary distribution.
• Applies the method in two variants: joint CT-HMC (simultaneous updates of x and β) and Gibbs CT-HMC (separate β updates via truncated exponential sampling).
• Utilizes variational approximations as a proposal for β to improve mixing, while maintaining asymptotic exactness in the limit.

Experimental results

Research questions

• RQ1 Can a continuous temperature variable improve mixing in multimodal distributions compared to standard HMC?
• RQ2 Can joint continuous tempering enable accurate estimation of the normalizing constant of a target distribution?
• RQ3 How does CT-HMC compare in sampling efficiency and normalizing constant estimation to annealed importance sampling and simulated tempering?
• RQ4 Can CT-HMC be seamlessly integrated into existing HMC-based probabilistic programming frameworks?
• RQ5 Does the continuous temperature control outperform discrete tempering schemes in terms of computational efficiency and sampling quality?

Key findings

• CT-HMC achieves competitive performance with annealed importance sampling and simulated tempering in sampling from challenging multimodal distributions.
• The method enables accurate estimation of the normalizing constant of the target distribution, which is often intractable in standard HMC.
• Joint CT-HMC outperforms simulated tempering in GPU implementations due to avoiding costly discrete categorical sampling for β updates.
• Gibbs CT-HMC eliminates the need to tune the temperature control mass parameter and achieves similar or better sampling efficiency than joint CT-HMC in experiments.
• The approach is simple to implement within existing HMC codebases, requiring only a standard leapfrog integrator and no changes to the core HMC logic.
• Adaptive HMC variants like NUTS can be directly applied to the extended space, enabling black-box tempered inference without manual tuning.