[论文解读] Bayes Shrinkage at GWAS scale: Convergence and Approximation Theory of a Scalable MCMC Algorithm for the Horseshoe Prior

James E. Johndrow, Paulo Orenstein+1 more|arXiv (Cornell University)|May 2, 2017

Statistical Methods and Inference参考文献 40被引用 23

一句话总结

本文提出了一种可扩展的MCMC算法，用于高维贝叶斯回归中的horseshoe先验，通过块更新和矩阵近似实现几何遍历性，并获得数量级的加速。该方法在GWAS规模问题（N=2,267，p=98,385）中实现了精确的后验推断，相比先前方法具有更优的收敛性、更低的均方误差以及更优的可信区间覆盖性能。

ABSTRACT

The horseshoe prior is frequently employed in Bayesian analysis of high-dimensional models, and has been shown to achieve minimax optimal risk properties when the truth is sparse. While optimization-based algorithms for the extremely popular Lasso and elastic net procedures can scale to dimension in the hundreds of thousands, algorithms for the horseshoe that use Markov chain Monte Carlo (MCMC) for computation are limited to problems an order of magnitude smaller. This is due to high computational cost per step and growth of the variance of time-averaging estimators as a function of dimension. We propose two new MCMC algorithms for computation in these models that have improved performance compared to existing alternatives. One of the algorithms also approximates an expensive matrix product to give orders of magnitude speedup in high-dimensional applications. We prove that the exact algorithm is geometrically ergodic, and give guarantees for the accuracy of the approximate algorithm using perturbation theory. Versions of the approximation algorithm that gradually decrease the approximation error as the chain extends are shown to be exact. The scalability of the algorithm is illustrated in simulations with problem size as large as $N=5,000$ observations and $p=50,000$ predictors, and an application to a genome-wide association study with $N=2,267$ and $p=98,385$. The empirical results also show that the new algorithm yields estimates with lower mean squared error, intervals with better coverage, and elucidates features of the posterior that were often missed by previous algorithms in high dimensions, including bimodality of posterior marginals indicating uncertainty about which covariates belong in the model.

研究动机与目标

• 为解决在高维设置下（特别是p ≫ N的GWAS）horseshoe先验缺乏可扩展MCMC算法的问题。
• 克服现有horseshoe MCMC采样器中昂贵的矩阵运算和缓慢混合的计算瓶颈。
• 建立所提精确算法的几何遍历性，确保快速收敛和有效的渐近推断。
• 开发一种通过矩阵乘积近似降低计算成本的近似算法，同时保持理论上的准确性保证。
• 在高维模拟和真实GWAS数据中展示该方法的实证优越性，包括检测后验双峰性及改善区间覆盖性能。

提出的方法

• 该算法采用块Gibbs采样，联合更新β、σ²、ξ和η，以改善混合性并实现几何遍历性。
• 构造一个李雅普诺夫函数以证明几何遍历性，其各项可控制βj²σ⁻²远离0和∞，即使在高维设置下亦成立。
• 通过近似将昂贵的WDW′矩阵乘积替换为快速稀疏近似，当η⁻¹近似稀疏时可显著降低计算成本。
• 应用摄动理论以界定近似算法与精确算法的不变测度之间的误差，确保准确性。
• 提出一种逐渐减小近似误差的方案，证明其在极限下为精确，从而保持渐近有效性。
• 该方法在合成高维数据（N=5,000，p=50,000）和真实GWAS数据集（N=2,267，p=98,385）上实现并进行了测试。

实验结果

研究问题

• RQ1 能否设计一种可扩展的MCMC算法用于horseshoe先验，使其在高维设置下保持几何遍历性？
• RQ2 如何在不牺牲准确性的前提下降低horseshoe后验更新中矩阵运算的计算成本？
• RQ3 在horseshoe先验背景下，对近似MCMC算法的准确性可提供哪些理论保证？
• RQ4 与现有方法相比，所提算法是否在高维回归中提升了后验估计准确性和区间覆盖性能？
• RQ5 该算法能否检测到后验边际分布中的复杂特征（如双峰性），从而揭示变量选择中的不确定性？

主要发现

• 精确MCMC算法被证明具有几何遍历性，确保快速收敛，并为时间平均估计量提供有效的中心极限定理。
• 近似算法通过近似WDW′矩阵乘积实现了数量级的加速，尤其在η⁻¹为稀疏时效果显著。
• 利用摄动理论证明了近似算法的不变测度收敛于精确算法，且提供了误差界。
• 实证结果表明，与现有方法相比，该方法在高维设置下具有更低的均方误差和更优的可信区间覆盖性能。
• 该算法成功揭示了后验边际的双峰性，表明变量选择中存在不确定性，而此前的算法往往未能捕捉到这一现象。
• 该方法可扩展至GWAS规模问题，成功分析了包含p=98,385个预测变量和N=2,267个观测值的数据，其在准确性和计算效率方面均优于先前方法。