[Paper Review] Slice Sampling

Radford M. Neal|arXiv (Cornell University)|Sep 7, 2000

Bayesian Methods and Mixture ModelsComputer Science1,178 citations

TL;DR

This paper introduces slice sampling, a Markov chain Monte Carlo method that adaptively samples from univariate and multivariate distributions by uniformly sampling from the region under the density curve. By alternately sampling vertically and horizontally within slices, it efficiently explores the target distribution without manual tuning, outperforming Gibbs and Metropolis methods in many cases due to adaptive step size selection and reduced random walk behavior.

ABSTRACT

Markov chain sampling methods that automatically adapt to characteristics of the distribution being sampled can be constructed by exploiting the principle that one can sample from a distribution by sampling uniformly from the region under the plot of its density function. A Markov chain that converges to this uniform distribution can be constructed by alternating uniform sampling in the vertical direction with uniform sampling from the horizontal `slice' defined by the current vertical position, or more generally, with some update that leaves the uniform distribution over this slice invariant. Variations on such `slice sampling' methods are easily implemented for univariate distributions, and can be used to sample from a multivariate distribution by updating each variable in turn. This approach is often easier to implement than Gibbs sampling, and more efficient than simple Metropolis updates, due to the ability of slice sampling to adaptively choose the magnitude of changes made. It is therefore attractive for routine and automated use. Slice sampling methods that update all variables simultaneously are also possible. These methods can adaptively choose the magnitudes of changes made to each variable, based on the local properties of the density function. More ambitiously, such methods could potentially allow the sampling to adapt to dependencies between variables by constructing local quadratic approximations. Another approach is to improve sampling efficiency by suppressing random walks. This can be done using `overrelaxed' versions of univariate slice sampling procedures, or by using `reflective' multivariate slice sampling methods, which bounce off the edges of the slice.

Motivation & Objective

• Develop a self-adapting MCMC sampling method that requires no manual tuning of proposal distributions.
• Address the inefficiency of random walk behavior in standard MCMC methods like Metropolis and Gibbs sampling.
• Enable automatic adaptation of step sizes based on local density properties for improved convergence and mixing.
• Extend the method to multivariate settings by updating variables sequentially or jointly using local approximations.
• Improve sampling efficiency through overrelaxed and reflective variants that reduce random walk behavior.

Proposed method

• Sample uniformly from the region under the density function by alternating between vertical and horizontal uniform sampling within a slice defined by the current density value.
• Construct a Markov chain that converges to the uniform distribution over the slice, ensuring detailed balance and invariance.
• Use univariate slice sampling as a building block, updating one variable at a time while conditioning on others.
• Enable multivariate slice sampling by updating all variables simultaneously using local quadratic approximations to capture dependencies.
• Apply overrelaxed transformations to reduce random walk behavior in univariate updates by introducing antithetic moves.
• Implement reflective slice sampling in multivariate settings by bouncing the sampler off the boundaries of the slice to improve exploration.

Experimental results

Research questions

• RQ1 Can a sampling method be designed that automatically adapts step sizes to the local geometry of the target density without manual tuning?
• RQ2 How can slice sampling be extended to multivariate distributions while preserving efficiency and convergence?
• RQ3 To what extent can overrelaxed and reflective variants reduce random walk behavior in slice sampling?
• RQ4 Can local quadratic approximations in multivariate slice sampling effectively model variable dependencies and improve mixing?
• RQ5 How does slice sampling compare in efficiency and robustness to Gibbs and Metropolis-Hastings sampling in practice?

Key findings

• Slice sampling achieves automatic adaptation of step sizes by exploiting the geometry of the target density, eliminating the need for manual tuning.
• The method outperforms standard Metropolis updates and is often more efficient than Gibbs sampling due to adaptive step size selection.
• Overrelaxed and reflective variants significantly reduce random walk behavior, improving mixing and convergence speed.
• Multivariate slice sampling with simultaneous updates can adapt to local dependencies through local quadratic approximations.
• Sequential and joint update strategies provide flexible and efficient sampling for both univariate and multivariate targets.
• The approach is well-suited for routine and automated Bayesian inference due to its robustness and ease of implementation.