[Paper Review] Hybrid Monte Carlo methods for sampling probability measures on submanifolds

Tony Lelièvre, Mathias Rousset+1 more|arXiv (Cornell University)|Jul 6, 2018

Markov Chains and Monte Carlo Methods37 references41 citations

TL;DR

This paper proposes a generalized Hybrid Monte Carlo (GHMC) method for sampling probability measures on submanifolds by integrating a reverse projection check into the Metropolis-Hastings correction step. The method ensures detailed balance and eliminates bias in the invariant measure even for large timesteps, with numerical experiments demonstrating that the reverse check is essential for accuracy and efficiency, particularly when timesteps are large enough to break reversibility of the underlying integrator.

ABSTRACT

Probability measures supported on submanifolds can be sampled by adding an extra momentum variable to the state of the system, and discretizing the associated Hamiltonian dynamics with some stochastic perturbation in the extra variable. In order to avoid biases in the invariant probability measures sampled by discretizations of these stochastically perturbed Hamiltonian dynamics, a Metropolis rejection procedure can be considered. The so-obtained scheme belongs to the class of generalized Hybrid Monte Carlo (GHMC) algorithms. We show here how to generalize to GHMC a procedure suggested by Goodman, Holmes-Cerfon and Zappa for Metropolis random walks on submanifolds, where a reverse projection check is performed to enforce the reversibility of the algorithm for large timesteps and hence avoid biases in the invariant measure. We also provide a full mathematical analysis of such procedures, as well as numerical experiments demonstrating the importance of the reverse projection check on simple toy examples.

Motivation & Objective

• To develop a generalized Hybrid Monte Carlo (GHMC) algorithm that samples probability measures on submanifolds without bias from discretization errors.
• To extend the reverse projection check technique from Goodman et al. (2018) to GHMC schemes with non-zero gradient forces.
• To mathematically formalize the reversibility properties of GHMC discretizations on submanifolds for arbitrary timesteps.
• To demonstrate numerically that the reverse projection check is essential for unbiased sampling, especially when timesteps are large.
• To provide a complete analysis of the impact of the reversibility check on the invariant measure and sampling efficiency.

Proposed method

• Introduces a GHMC scheme where the proposal step uses a constrained Hamiltonian dynamics integrator (e.g., RATTLE) with momentum resampling.
• Applies a Metropolis-Hastings correction using the ratio of target probability densities at the initial and proposed states.
• Incorporates a reverse projection check: verifies that the reverse move from the proposed state returns to the original state via the same projection procedure.
• Uses Lagrange multiplier functions to define the projection of unconstrained dynamics steps back onto the submanifold.
• Employs Newton's method to solve the nonlinear equations arising from the projection and reverse projection steps.
• Implements a full rejection mechanism that includes: (1) failure of Newton convergence in forward or reverse steps, (2) non-reversibility (i.e., (q,p) ≠ ΨΔt ◦ ΨΔt(q,p)), and (3) Metropolis acceptance/rejection based on density ratio.

Experimental results

Research questions

• RQ1 Can the reverse projection check be generalized from Metropolis random walks to GHMC schemes with non-zero gradient forces?
• RQ2 Does the inclusion of the reverse projection check eliminate bias in the invariant measure for large timesteps in GHMC on submanifolds?
• RQ3 How does the rejection rate due to non-reversibility scale with timestep size in GHMC on constrained manifolds?
• RQ4 What is the impact of the reverse projection check on sampling efficiency, particularly in metastable systems?
• RQ5 Can the method achieve unbiased sampling without relying on small timesteps, as required in standard HMC?

Key findings

• For large timesteps (Δt = 1), the non-reversibility rejection rate reached 15–20% in GHMC with forces, indicating that the reverse check is critical for bias removal.
• When forces were set to zero (reducing to a Metropolis random walk), the optimal timestep was around 1, with non-reversibility rejections at ~5%, showing the check remains essential.
• The mean residence duration in metastable states scaled as 1/Δt for GHMC and 1/Δt² for MALA, confirming the correct scaling of the underlying dynamics.
• At Δt = 0.1, the non-reversibility rejection rate dropped to near zero (10⁻⁹), but the total rejection rate remained high (~6.7×10⁻⁴), indicating that Metropolis rejection dominates at small timesteps.
• The optimal sampling efficiency occurred at timesteps where the reverse projection check was most frequently triggered, proving its necessity for unbiased sampling.
• Numerical experiments confirmed that omitting the reverse check leads to significant bias, especially at large timesteps, even when Metropolis correction is applied.