[Paper Review] The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo

Matthew D. Homan, Andrew Gelman|arXiv (Cornell University)|Jan 1, 2014

Markov Chains and Monte Carlo Methods27 references3,275 citations

TL;DR

The No-U-Turn Sampler (NUTS) automates Hamiltonian Monte Carlo (HMC) by eliminating the need to manually set the number of steps L and adapting the step size e on the fly. It uses a recursive tree-growth algorithm to automatically terminate sampling when the trajectory begins to retrace its path, achieving efficient, turnkey inference without tuning.

ABSTRACT

Hamiltonian Monte Carlo (HMC) is a Markov chain Monte Carlo (MCMC) algorithm that avoids the random walk behavior and sensitivity to correlated parameters that plague many MCMC methods by taking a series of steps informed by first-order gradient information. These features allow it to converge to high-dimensional target distributions much more quickly than simpler methods such as random walk Metropolis or Gibbs sampling. However, HMC's performance is highly sensitive to two user-specified parameters: a step size e and a desired number of steps L. In particular, if L is too small then the algorithm exhibits undesirable random walk behavior, while if L is too large the algorithm wastes computation. We introduce the No-U-Turn Sampler (NUTS), an extension to HMC that eliminates the need to set a number of steps L. NUTS uses a recursive algorithm to build a set of likely candidate points that spans a wide swath of the target distribution, stopping automatically when it starts to double back and retrace its steps. Empirically, NUTS performs at least as efficiently as (and sometimes more effciently than) a well tuned standard HMC method, without requiring user intervention or costly tuning runs. We also derive a method for adapting the step size parameter e on the fly based on primal-dual averaging. NUTS can thus be used with no hand-tuning at all, making it suitable for applications such as BUGS-style automatic inference engines that require efficient turnkey samplers.

Motivation & Objective

• To address the sensitivity of HMC to user-specified parameters L (number of steps) and ε (step size), which critically affect sampling efficiency.
• To eliminate the need for manual tuning of the number of steps L in HMC, which otherwise leads to random walk behavior or wasted computation.
• To develop an automatic mechanism for adapting the step size ε during sampling using primal-dual averaging.
• To create a turnkey MCMC sampler suitable for automated inference engines like BUGS, requiring no user intervention.

Proposed method

• NUTS employs a recursive tree-building algorithm that extends the HMC trajectory in both forward and backward directions, doubling the path length at each step.
• The algorithm stops when the trajectory starts to double back on itself, detected by checking whether the angular momentum of the trajectory reverses direction.
• It uses a criterion based on the difference in position and momentum between the endpoints of the trajectory to detect when the path has begun to retrace its steps.
• The step size ε is adapted on the fly using primal-dual averaging, which maintains a stable, convergent adaptation schedule over time.
• The sampler integrates the leapfrog integrator to simulate Hamiltonian dynamics, with gradient information used to guide the proposal distribution.
• The method ensures detailed balance and correct stationary distribution by preserving the invariance of the target distribution under the dynamics.

Experimental results

Research questions

• RQ1 Can a fully automatic HMC sampler be designed that eliminates the need for manual tuning of the number of steps L?
• RQ2 How can the step size ε be adaptively adjusted during sampling to maintain efficiency and convergence?
• RQ3 Can a stopping criterion be developed that automatically detects when a trajectory has reached a point of diminishing returns in exploration?
• RQ4 Does the resulting NUTS sampler match or exceed the efficiency of well-tuned standard HMC in high-dimensional target distributions?
• RQ5 Can NUTS be used as a plug-in replacement in automated inference systems like BUGS without requiring user configuration?

Key findings

• NUTS performs at least as efficiently as a well-tuned standard HMC, and often more efficiently, due to its adaptive path length selection.
• The algorithm automatically terminates when the trajectory begins to retrace its path, avoiding the need for manual L specification.
• The primal-dual averaging method for step size adaptation ensures stable and convergent tuning without requiring costly calibration runs.
• Empirical results show that NUTS achieves efficient exploration of high-dimensional target distributions without user intervention.
• The sampler maintains detailed balance and correct stationary distribution, ensuring valid Bayesian inference.
• NUTS is suitable for integration into automated inference engines, enabling efficient turnkey MCMC sampling.