[Paper Review] Quasi-Reliable Estimates of Effective Sample Size

Youhan Fang, Yudong Cao+1 more|arXiv (Cornell University)|May 10, 2017

Markov Chains and Monte Carlo Methods4 citations

TL;DR

This paper proposes a quasi-reliable method for estimating effective sample size (ESS) in Markov chain Monte Carlo (MCMC) by identifying the longest autocorrelation time, τ_max, across all possible observables. By estimating τ_max via a generalized eigenvalue problem on basis functions, the method provides a more robust indicator of sampling adequacy than standard ESS estimates, especially in multimodal distributions where individual observables may give deceptively low τ values.

ABSTRACT

The efficiency of a Markov chain Monte Carlo algorithm might be measured by the cost of generating one independent sample, or equivalently, the total cost divided by the effective sample size, defined in terms of the integrated autocorrelation time. To ensure the reliability of such an estimate, it is suggested that there be an adequate sampling of state space--- to the extent that this can be determined from the available samples. A possible method for doing this is derived and evaluated.

Motivation & Objective

• To improve the reliability of effective sample size (ESS) estimation in MCMC, particularly for multimodal distributions where standard ESS estimates may be misleading.
• To address the risk of incomplete state space sampling by introducing a measure of 'thoroughness' in sampling coverage.
• To develop a computationally feasible method for estimating the maximum possible integrated autocorrelation time τ_max across all observables.
• To provide a more conservative and reliable convergence diagnostic than individual observable-based τ estimates.
• To demonstrate that τ_max serves as a better indicator of sampling adequacy than ESS or τ for specific observables.

Proposed method

• Derives τ_max as the longest autocorrelation time over all functions on the state space, defined as the supremum of integrated autocorrelation times.
• Uses a finite set of basis functions to approximate the maximizing observable, reducing the infinite-dimensional problem to a generalized eigenvalue problem.
• Solves the generalized eigenvalue problem involving the forward transfer operator and lag-windowed autocovariance matrices to estimate τ_max.
• Applies the method to multiple test cases, including Gaussian mixtures, neural networks, and logistic regression, comparing τ_max with individual observable τ and ESS.
• Employs a lag window method for estimating τ, with an alternative derivation in Appendix A to improve reliability over prior tools like acor.
• Validates the method by showing that τ_max convergence coincides with convergence of key metrics like training error and MSE.

(a) 5000 samples

Experimental results

Research questions

• RQ1 Can the maximum possible autocorrelation time τ_max serve as a more reliable indicator of MCMC sampling adequacy than individual observable-based τ estimates?
• RQ2 How can one estimate τ_max efficiently from finite MCMC samples without exhaustive function enumeration?
• RQ3 To what extent does τ_max detect incomplete sampling in multimodal distributions where individual observables appear well-mixed?
• RQ4 Does τ_max convergence align with convergence of practical metrics like prediction error or training loss in machine learning models?
• RQ5 Can the generalized eigenvalue approach on basis functions reliably approximate τ_max with moderate computational cost?

Key findings

• In the one-node neural network example, τ_max converged only after N > 2×10⁴ samples, while individual observables like q₁ and prediction had τ ≈ 1, indicating false convergence.
• For logistic regression, both training and testing errors converged only after N > 1000, but individual observables showed τ ≈ 1, falsely suggesting independence.
• The ESS for the prediction in the neural network model was misleadingly high at N ≈ 100, while τ_max required much larger N to stabilize.
• The generalized eigenvalue method successfully identified τ_max as significantly larger than individual τ values, confirming its role in detecting slow mixing.
• The method’s estimates of τ_max were more reliable than those from the acor program, especially under high τ conditions.
• Convergence of τ_max coincided with convergence of mean squared error and prediction accuracy, validating its use as a convergence diagnostic.

(b) Trajectories of $q_{1}$ and $q_{2}$