[Paper Review] Irreversible Monte Carlo Algorithms for Efficient Sampling

Konstantin Turitsyn, Michael Chertkov+1 more|OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information)|Sep 4, 2008

Theoretical and Computational Physics20 references3 citations

TL;DR

This paper proposes irreversible Monte Carlo algorithms that dramatically accelerate sampling by breaking detailed balance while preserving the target equilibrium distribution. By introducing controlled probability fluxes through cycle decompositions in the transition graph, the method reduces mixing time—demonstrated to eliminate critical slowdown in a mean-field spin cluster model—offering a general framework to enhance convergence in systems with high degeneracy or slow mixing.

ABSTRACT

Equilibrium systems evolve according to Detailed Balance (DB). This principe guided development of the Monte-Carlo sampling techniques, of which Metropolis-Hastings (MH) algorithm is the famous representative. It is also known that DB is sufficient but not necessary. We construct irreversible deformation of a given reversible algorithm capable of dramatic improvement of sampling from known distribution. Our transformation modifies transition rates keeping the structure of transitions intact. To illustrate the general scheme we design an Irreversible version of Metropolis-Hastings (IMH) and test it on example of a spin cluster. Standard MH for the model suffers from the critical slowdown, while IMH is free from critical slowdown.

Motivation & Objective

• To address the critical slowdown in Markov Chain Monte Carlo (MCMC) sampling for systems with high degeneracy or soft modes.
• To develop a general method for constructing irreversible MCMC algorithms that maintain the correct stationary distribution while accelerating convergence.
• To demonstrate that breaking detailed balance via non-reversible fluxes can outperform standard reversible MCMC, especially in systems near phase transitions.
• To provide a practical, general-purpose irreversible variant of the Metropolis-Hastings algorithm (IMH) applicable to complex systems.

Proposed method

• Construct an irreversible deformation of a reversible Markov chain by adding non-zero probability fluxes along cycles in the state graph, preserving the stationary distribution.
• Use cycle decomposition to represent the antisymmetric part of the transition matrix, introducing rotational components to the probability flow.
• Define the irreversible Metropolis-Hastings (IMH) algorithm by modifying transition rates while maintaining detailed balance in the stationary distribution.
• Ensure positivity of transition probabilities by adjusting proposal and acceptance rates to maintain ergodicity and convergence to the target distribution.
• Apply the method to a mean-field spin cluster model to test performance against standard Metropolis-Hastings.
• Use the cycle flux representation to quantify the improvement in mixing time, showing reduction from diffusive to sub-diffusive scaling.

Experimental results

Research questions

• RQ1 Can irreversible Markov chains be constructed to sample from a given equilibrium distribution faster than reversible chains?
• RQ2 What is the impact of breaking detailed balance on mixing time in systems with high entropic degeneracy or critical slowing down?
• RQ3 How can irreversible dynamics be systematically embedded into existing reversible MCMC algorithms like Metropolis-Hastings without altering the target distribution?
• RQ4 To what extent can cycle-based probability fluxes reduce the effective mixing time in models with soft modes or long correlation lengths?
• RQ5 Can irreversible algorithms outperform even advanced reversible methods like cluster algorithms or worm algorithms in specific models?

Key findings

• The irreversible Metropolis-Hastings (IMH) algorithm successfully eliminates critical slowdown in the mean-field spin cluster model, where standard Metropolis-Hastings suffers from slow mixing.
• The mixing time of the IMH algorithm scales as $ T \sim L^{4/3} $, compared to $ T \sim L^2 $ for reversible MCMC, indicating a significant acceleration.
• The improvement arises from introducing non-zero rotational components in the probability flow, enabling faster exploration of the state space via directed cycles.
• The method achieves faster convergence without requiring model-specific transformations or non-local moves, unlike cluster or worm algorithms.
• The framework is general and can be applied to any reversible MCMC algorithm by adding controlled fluxes along cycles in the state graph.
• The approach demonstrates that breaking detailed balance can lead to substantial performance gains in systems with high degeneracy or critical behavior.