[Paper Review] Local-Global MCMC kernels: the best of both worlds

Sergey Samsonov, Evgeny Lagutin+4 more|arXiv (Cornell University)|Nov 4, 2021

Markov Chains and Monte Carlo Methods4 citations

TL;DR

This paper proposes Ex²MCMC, a hybrid Markov chain Monte Carlo method that combines local (e.g., MALA, HMC) and global (i-SIR) samplers to achieve faster mixing and better exploration in high-dimensional and heavy-tailed distributions. It proves V-uniform geometric ergodicity without requiring uniform adaptation of the global proposal, and introduces FlEx²MCMC, an adaptive version using normalizing flows trained during sampling, showing superior performance on GAN-based energy models.

ABSTRACT

Recent works leveraging learning to enhance sampling have shown promising results, in particular by designing effective non-local moves and global proposals. However, learning accuracy is inevitably limited in regions where little data is available such as in the tails of distributions as well as in high-dimensional problems. In the present paper we study an Explore-Exploit Markov chain Monte Carlo strategy ($Ex^2MCMC$) that combines local and global samplers showing that it enjoys the advantages of both approaches. We prove $V$-uniform geometric ergodicity of $Ex^2MCMC$ without requiring a uniform adaptation of the global sampler to the target distribution. We also compute explicit bounds on the mixing rate of the Explore-Exploit strategy under realistic conditions. Moreover, we also analyze an adaptive version of the strategy ($FlEx^2MCMC$) where a normalizing flow is trained while sampling to serve as a proposal for global moves. We illustrate the efficiency of $Ex^2MCMC$ and its adaptive version on classical sampling benchmarks as well as in sampling high-dimensional distributions defined by Generative Adversarial Networks seen as Energy Based Models. We provide the code to reproduce the experiments at the link: https://github.com/svsamsonov/ex2mcmc_new.

Motivation & Objective

• To address slow mixing and poor exploration in high-dimensional and heavy-tailed distributions using MCMC.
• To combine local and global MCMC kernels to leverage the strengths of both: local for accurate sampling in tails and global for rapid exploration.
• To provide theoretical guarantees for convergence and mixing speed of the hybrid strategy without requiring uniform adaptation of the global proposal.
• To develop and evaluate an adaptive version, FlEx²MCMC, where a normalizing flow is trained in real-time to improve global proposals.
• To demonstrate the method’s effectiveness on challenging benchmarks, including GANs as energy-based models.

Proposed method

• Introduces Ex²MCMC, a Markov chain that alternates between local MCMC kernels (e.g., MALA, HMC) and a global i-SIR kernel for independent, parallel proposals.
• Uses the i-SIR kernel to generate multiple global proposals per iteration, improving parallelization and exploration efficiency.
• Employs a rejuvenation kernel (e.g., 3 MALA steps) to refine samples and reduce rejection rates in high-dimensional or heavy-tailed regions.
• Proposes FlEx²MCMC, an adaptive variant where a normalizing flow is trained online to refine the global proposal distribution during sampling.
• Establishes theoretical convergence via V-uniform geometric ergodicity under mild conditions, even when the global proposal is not uniformly well-adapted.
• Uses iterative importance resampling to maintain proposal quality and ensure robustness in high-dimensional spaces.

Experimental results

Research questions

• RQ1 Can combining local and global MCMC kernels significantly improve mixing time and exploration in high-dimensional and heavy-tailed distributions?
• RQ2 Under what conditions can a hybrid Ex²MCMC strategy achieve V-uniform geometric ergodicity without requiring uniform adaptation of the global proposal?
• RQ3 How does the performance of Ex²MCMC compare to purely local or purely global MCMC methods on complex energy-based models such as GANs?
• RQ4 Can an adaptive strategy like FlEx²MCMC, where a normalizing flow is trained during sampling, improve proposal quality and sampling efficiency?
• RQ5 What is the empirical impact of the hybrid strategy on metrics like FID and Inception Score when sampling from trained GANs?

Key findings

• Ex²MCMC achieves V-uniform geometric ergodicity under significantly milder conditions than required for the global i-SIR kernel alone, enabling robust convergence without uniform proposal adaptation.
• The method demonstrates faster mixing and better exploration than MALA or i-SIR alone, with trajectories in SN-GAN and DC-GAN experiments showing both mode-hopping and accurate sampling in low-density regions.
• FlEx²MCMC achieves lower energy values and improved FID and Inception Score metrics on DC-GAN, indicating higher-quality sample generation over 10,000 trajectories.
• In both DC-GAN and SN-GAN benchmarks, Ex²MCMC and FlEx²MCMC produce lower average energy values than i-SIR or MALA, indicating better sampling efficiency and coverage of the target distribution.
• The adaptive training of normalizing flows in FlEx²MCMC leads to progressively better global proposals, reducing rejection rates and improving convergence speed.
• Empirical results show that local rejuvenation steps significantly reduce the risk of getting trapped in local modes, even when global proposals are imperfect.