[Paper Review] Bayesian Variable Selection for Multi-Outcome Models Through Shared Shrinkage

Debamita Kundu, Riten Mitra+1 more|arXiv (Cornell University)|Jan 1, 2019

Statistical Methods and Bayesian Inference26 references34 citations

TL;DR

Extends global-local shrinkage to multivariate regression to identify covariates affecting multiple outcomes, with three multi-outcome shrinkage models and posterior-consistency results.

ABSTRACT

Variable selection over a potentially large set of covariates in a linear model is quite popular. In the Bayesian context, common prior choices can lead to a posterior expectation of the regression coefficients that is a sparse (or nearly sparse) vector with a few non-zero components, those covariates that are most important. This article extends the global-local shrinkage idea to a scenario where one wishes to model multiple response variables simultaneously. Here, we have developed a variable selection method for a K-outcome model (multivariate regression) that identifies the most important covariates across all outcomes. The prior for all regression coefficients is a mean zero normal with coefficient-specific variance term that consists of a predictor-specific factor (shared local shrinkage parameter) and a model-specific factor (global shrinkage term) that differs in each model. The performance of our modeling approach is evaluated through simulation studies and a data example.

Motivation & Objective

• Motivate variable selection in high-dimensional multivariate regression.
• Develop a shared local shrinkage framework across K outcomes to borrow strength across models.
• Compare three multi-outcome shrinkage priors (Normal-gamma, horseshoe, Dirichlet-Laplace) and assess performance via simulations and a yeast dataset.

Proposed method

• Propose a GL (global-local) shrinkage prior extended to B for a K-outcome model: beta_jk ~ N(0, lambda_j^2 tau_k^2); lambda_j ~ f(.); tau_k ~ g(.)
• Instantiate three versions: Multi-outcome Normal-gamma (MONG), Multi-outcome Horseshoe (MOHS), Multi-outcome Dirichlet-Laplace (MODL).
• Provide sampling schemes: conjugate updates where possible; gamma/IG priors for hyperparameters; adaptive Metropolis-Hastings steps for non-conjugate blocks.
• Establish posterior consistency under fixed Psi and standard regularity assumptions (A1–A3).
• Discuss practical implementation details and comparisons against naive models and MBSP.

Experimental results

Research questions

• RQ1 Can sharing the local shrinkage parameter across multiple outcomes improve variable selection and prediction in multivariate regression?
• RQ2 How do MONG, MOHS, and MODL perform relative to naive independent priors and competing methods in terms of MSPE and estimation accuracy?
• RQ3 Does the proposed framework maintain favorable properties when the true sparsity structure varies across outcomes?
• RQ4 What are the computational considerations and convergence properties of the MCMC samplers for these models?

Key findings

• Shared shrinkage models reduce MSPE and SSE relative to naive models across simulated scenarios.
• In the first simulation, MONG, MODL (a=0.5), MOHS, and MBSP show competitive MSPE around 1.0 and significantly lower SSE for non-zero coefficients compared to naive approaches.
• MODL with a=0.5 and a=1/p balance shrinkage and signal preservation, outperforming naive Dirichlet-Laplace in several settings.
• Across settings with both identical and varying predictor relevance across outcomes, sharing shrinkage information yields lower SSE for zero coefficients, indicating better variable selection.
• Selection-prior and MBSP provide competitive predictive performance, but MBSP exhibits relatively higher SSE for zero signals due to fixed tau, whereas the proposed models estimate global scales from data.