[Paper Review] Coupled Markov chain Monte Carlo for high-dimensional regression with Half-t priors

Niloy Biswas, Anirban Bhattacharya+2 more|arXiv (Cornell University)|Dec 9, 2020

Bayesian Methods and Mixture Models1 references4 citations

TL;DR

This paper introduces coupled Markov chain Monte Carlo (MCMC) algorithms for high-dimensional Bayesian regression using Half-t priors, enhancing computational scalability and convergence diagnostics. The method demonstrates efficient performance on a genome-wide association study with 100,000 covariates, showing that Half-t priors with degrees of freedom >1 improve both statistical robustness and computational efficiency.

ABSTRACT

Continuous shrinkage priors are commonly used in Bayesian analysis of high-dimensional data, due to both their computational advantages and favorable statistical properties. We develop coupled Markov chain Monte Carlo (MCMC) algorithms for Bayesian shrinkage regression in high dimensions. Following Glynn & Rhee (2014), these couplings can be used in parallel computation strategies and practical diagnostics of convergence. Focusing on a class of shrinkage priors which includes the Horseshoe, we demonstrate the scalability of the proposed couplings with data from a genome-wide association study with 2000 rows and 100,000 covariates. The results highlight the impact of the shrinkage prior on the computational efficiency of the coupling procedure, and motivate priors where the local precisions are Half-t distributed with degree of freedom larger than one, which are statistically justifiable in terms of posterior concentration, and lead to practical computations.

Motivation & Objective

• To develop scalable MCMC algorithms for high-dimensional regression with continuous shrinkage priors.
• To improve convergence diagnostics and parallel computation efficiency using couplings inspired by Glynn & Rhee (2014).
• To evaluate the impact of prior choice—specifically Half-t priors—on computational performance in high-dimensional settings.
• To identify priors that balance statistical optimality with computational tractability in large-scale regression.

Proposed method

• Adapt coupled MCMC techniques to Bayesian shrinkage regression, enabling parallel execution and convergence monitoring.
• Use a class of shrinkage priors that includes the Horseshoe, with local precision parameters drawn from a Half-t distribution.
• Employ Half-t priors with degrees of freedom greater than one to ensure favorable posterior concentration and computational stability.
• Design the coupling mechanism to preserve detailed balance and enable effective convergence diagnostics.
• Implement the algorithm on a high-dimensional genome-wide association study (GWAS) with 2000 samples and 100,000 covariates.
• Leverage the coupling structure to estimate Monte Carlo error and assess mixing efficiency in high-dimensional parameter spaces.

Experimental results

Research questions

• RQ1 How can coupled MCMC be effectively applied to high-dimensional regression with continuous shrinkage priors?
• RQ2 What is the impact of the Half-t prior’s degrees of freedom on computational efficiency and convergence?
• RQ3 Can coupled MCMC enable reliable convergence diagnostics in high-dimensional Bayesian models?
• RQ4 How do Half-t priors with degrees of freedom >1 compare to other shrinkage priors in terms of posterior concentration and computation?
• RQ5 What is the scalability of the proposed method on real-world high-dimensional data, such as GWAS datasets?

Key findings

• The proposed coupled MCMC algorithm achieves scalable performance on a high-dimensional GWAS dataset with 100,000 covariates and 2,000 samples.
• Half-t priors with degrees of freedom greater than one yield better computational efficiency compared to other shrinkage priors.
• The coupling mechanism enables effective convergence diagnostics and parallel computation in high-dimensional settings.
• The method demonstrates practical convergence monitoring through coupled chains, reducing reliance on heuristics.
• Posterior concentration properties of the Half-t prior are statistically justified, supporting its use in high-dimensional regression.
• The results confirm that prior choice significantly influences both computational scalability and statistical reliability in high-dimensional Bayesian inference.