[Paper Review] Selection of a MCMC simulation strategy via an entropy convergence criterion

Didier Chauveau, Pierre Vandekerkhove|ArXiv.org|May 10, 2006

Markov Chains and Monte Carlo Methods30 references3 citations

TL;DR

This paper proposes a simulation-based entropy convergence criterion to select the optimal MCMC strategy—such as Metropolis-Hastings or Gibbs samplers—by comparing parallel independent Markov chains. The method uses kernel density estimation and Monte Carlo integration to consistently estimate an entropy-based convergence measure, enabling early selection of the fastest-converging algorithm without prior knowledge of mixing rates.

ABSTRACT

In MCMC methods, such as the Metropolis-Hastings (MH) algorithm, the Gibbs sampler, or recent adaptive methods, many different strategies can be proposed, often associated in practice to unknown rates of convergence. In this paper we propose a simulation-based methodology to compare these rates of convergence, grounded on an entropy criterion computed from parallel (i.i.d.) simulated Markov chains coming from each candidate strategy. Our criterion determines on the very first iterations the best strategy among the candidates. Theoretically, we give for the MH algorithm general conditions under which its successive densities satisfy adequate smoothness and tail properties, so that this entropy criterion can be estimated consistently using kernel density estimate and Monte Carlo integration. Simulated examples are provided to illustrate this convergence criterion.

Motivation & Objective

• To address the challenge of selecting the best MCMC strategy when convergence rates are unknown and difficult to compute analytically.
• To develop a practical, simulation-based criterion that enables early comparison of MCMC algorithms—such as Metropolis-Hastings, Gibbs samplers, or adaptive methods—based on convergence speed.
• To establish theoretical conditions under which the entropy convergence criterion can be consistently estimated for the Metropolis-Hastings algorithm using nonparametric density estimation.
• To provide a methodology applicable to both standard and adaptive MCMC methods, including comparison across different proposal distributions or Gibbs sampling decompositions.
• To demonstrate the feasibility and consistency of the entropy criterion in finite samples through simulated examples, even when theoretical convergence bounds are unavailable.

Proposed method

• Generate multiple independent, parallel Markov chains from each candidate MCMC strategy (e.g., different proposal densities in Metropolis-Hastings).
• Compute the empirical entropy of the chain states at each iteration using kernel density estimation to approximate the density of the current distribution.
• Use Monte Carlo integration to estimate the entropy criterion over time, tracking how quickly it stabilizes toward the target entropy.
• Define the entropy convergence criterion as the rate at which the empirical entropy approaches the limiting entropy of the target distribution.
• Apply the criterion to compare strategies on the very first iterations, selecting the one with the fastest entropy convergence as the optimal choice.
• Establish theoretical consistency of the entropy estimator under smoothness and tail conditions on the target and proposal densities, particularly for symmetric random walk and independence samplers.

Experimental results

Research questions

• RQ1 Can a simulation-based entropy criterion consistently identify the fastest-converging MCMC strategy without relying on theoretical convergence bounds?
• RQ2 What regularity conditions on the target and proposal densities ensure the consistency of the entropy estimator via kernel density and Monte Carlo integration?
• RQ3 How early in the simulation can the entropy convergence criterion reliably distinguish between competing MCMC strategies?
• RQ4 Can the entropy criterion be applied to compare different Gibbs samplers or to contrast Gibbs samplers with Metropolis-Hastings algorithms?
• RQ5 What are the theoretical properties of the entropy convergence criterion under general Metropolis-Hastings kernels, particularly regarding Lipschitz continuity and integrability of transition densities?

Key findings

• The entropy convergence criterion consistently estimates the true entropy of the target distribution under mild regularity conditions, including smoothness and tail behavior of the target and proposal densities.
• For the Metropolis-Hastings algorithm, the criterion is consistent when the proposal density is symmetric and the target density satisfies appropriate tail decay and smoothness conditions.
• The method enables early selection of the optimal MCMC strategy based on entropy convergence, even after only a few hundred iterations, without requiring knowledge of the true convergence rate.
• Theoretical analysis confirms that the entropy criterion is well-behaved under the assumption that the transition kernel and acceptance probability are Lipschitz continuous, ensuring stability in estimation.
• Simulations demonstrate that the entropy criterion reliably ranks MCMC strategies by convergence speed, with faster-converging chains showing earlier entropy stabilization.
• The approach is robust to model complexity and applicable to both standard and adaptive MCMC methods, including comparisons between independence and random walk Metropolis-Hastings algorithms.