[Paper Review] The MCMC split sampler: A block Gibbs sampling scheme for latent Gaussian models

Óli Páll Geirsson, Birgir Hrafnkelsson+2 more|arXiv (Cornell University)|Jun 20, 2015

Statistical Methods and Bayesian Inference7 references4 citations

TL;DR

The paper proposes the MCMC split sampler, a two-block Gibbs sampling scheme for latent Gaussian models (LGMs) that improves computational efficiency by splitting latent parameters into data-rich and data-poor blocks. By exploiting the Gaussian structure of the data-poor block, the method enables efficient sampling independent of the likelihood function, scaling well with high-dimensional latent fields and supporting any parametric data density.

ABSTRACT

A novel computationally efficient Markov chain Monte Carlo (MCMC) scheme for latent Gaussian models (LGMs) is proposed in this paper. The sampling scheme is a two block Gibbs sampling scheme designed to exploit the model structure of LGMs. We refer to the proposed sampling scheme as the MCMC split sampler. The principle idea behind the MCMC split sampler is to split the latent Gaussian parameters into two vectors. The former vector consists of latent parameters which appear in the data density function, while the latter vector consists of latent parameters which do not appear in it. The former vector is placed in the first block of the proposed sampling scheme and the latter vector is placed in the second block along with any potential hyperparameters. The resulting conditional posterior density functions within the blocks allow the MCMC split sampler to handle, by design, LGMs with latent models imposed on more than just the mean structure of the data density function. The MCMC split sampler is also designed to be applicable for any choice of a parametric data density function. Moreover, it scales well in terms of computational efficiency when the dimension of the latent model increase.

Motivation & Objective

• To address computational challenges in posterior inference for latent Gaussian models (LGMs) with complex likelihood structures.
• To improve mixing and convergence in MCMC sampling when latent fields influence more than just the mean of the data density.
• To develop a scalable, efficient MCMC scheme applicable to LGMs with non-Gaussian likelihoods and high-dimensional latent structures.
• To enable modular design of sampling schemes by isolating the Gaussian conditional posterior in one block.

Proposed method

• The method splits latent parameters into two blocks: one containing parameters that appear in the data density (data-rich), and another containing those that do not (data-poor).
• The data-poor block's conditional posterior is a known Gaussian distribution, invariant to the choice of data density function.
• The data-poor block is sampled using efficient algorithms such as the one-block sampler of Knorr-Held and Rue (2002), leveraging sparse GMRF precision structures for scalability.
• The data-rich block is sampled using flexible schemes like MALA or HMC, tailored to the structure of the likelihood.
• The overall scheme uses Gibbs sampling by alternating between the two blocks, with the data-poor block's posterior being analytically tractable and computationally efficient.
• The method is modular: sampling strategies for each block can be developed and improved independently.

Experimental results

Research questions

• RQ1 How can MCMC sampling in latent Gaussian models be made more efficient when latent effects influence multiple parameters in the likelihood beyond just the mean?
• RQ2 Can a block Gibbs sampler be designed such that one block’s posterior is invariant to the data density function, enabling faster and more stable sampling?
• RQ3 To what extent does the proposed split sampler scale with increasing dimensionality of the latent field?
• RQ4 How can the computational efficiency of MCMC for LGMs be improved without restricting the likelihood to exponential family forms?

Key findings

• The conditional posterior for the data-poor block is a known Gaussian distribution, independent of the data density function, enabling highly efficient sampling.
• The method scales well with increasing dimension of the latent field due to the preservation of sparse GMRF precision structures in the data-poor block.
• The computational efficiency of the data-poor block depends only on the hyperparameter sampling scheme, making the overall method modular and extensible.
• The data-poor block’s Gaussian structure allows the use of the efficient one-block sampler of Knorr-Held and Rue (2002), regardless of the likelihood type.
• The method supports any parametric data density function, including non-Gaussian ones, broadening applicability beyond standard models.
• The approach enables independent optimization of sampling schemes for each block, enhancing flexibility and performance.