[论文解读] Scalable Bayesian shrinkage and uncertainty quantification in high-dimensional regression

Bala Rajaratnam, Doug Sparks+2 more|arXiv (Cornell University)|Mar 27, 2017

Statistical Methods and Inference参考文献 39被引用 4

一句话总结

本文提出了一种用于高维回归中贝叶斯收缩的新型两步分块吉布斯采样器，相较于标准的三步吉布斯采样器，显著提升了收敛速度。该方法确保了几何遍历性与迹类马尔可夫算子，从而实现更快的不确定性量化，并为与夹心算法的严格理论比较提供了基础。

ABSTRACT

Bayesian shrinkage methods have generated a lot of recent interest as tools for high-dimensional regression and model selection. These methods naturally facilitate tractable uncertainty quantification and incorporation of prior information. A common feature of these models, including the Bayesian lasso, global-local shrinkage priors, and spike-and-slab priors is that the corresponding priors on the regression coefficients can be expressed as scale mixture of normals. While the three-step Gibbs sampler used to sample from the often intractable associated posterior density has been shown to be geometrically ergodic for several of these models (Khare and Hobert, 2013; Pal and Khare, 2014), it has been demonstrated recently that convergence of this sampler can still be quite slow in modern high-dimensional settings despite this apparent theoretical safeguard. We propose a new method to draw from the same posterior via a tractable two-step blocked Gibbs sampler. We demonstrate that our proposed two-step blocked sampler exhibits vastly superior convergence behavior compared to the original three- step sampler in high-dimensional regimes on both real and simulated data. We also provide a detailed theoretical underpinning to the new method in the context of the Bayesian lasso. First, we derive explicit upper bounds for the (geometric) rate of convergence. Furthermore, we demonstrate theoretically that while the original Bayesian lasso chain is not Hilbert-Schmidt, the proposed chain is trace class (and hence Hilbert-Schmidt). The trace class property has useful theoretical and practical implications. It implies that the corresponding Markov operator is compact, and its eigenvalues are summable. It also facilitates a rigorous comparison of the two-step blocked chain with "sandwich" algorithms which aim to improve performance of the two-step chain by inserting an inexpensive extra step.

研究动机与目标

• 为解决尽管具有几何遍历性，标准三步吉布斯采样器在高维贝叶斯收缩模型中收敛缓慢的问题。
• 开发一种更高效的采样算法，在保持相同后验分布的同时，加速高维设置下的混合速度。
• 为新采样器提供理论基础，包括收敛速率界限与算子类性质。
• 通过将新链定义为迹类，实现与“夹心”算法的严格比较。
• 在模拟与真实高维数据上展示新采样器的优越经验性能。

提出的方法

• 提出一种两步分块吉布斯采样器，通过更有效地分组变量来重新组织条件更新，从而改善混合效果。
• 为使用新采样器的贝叶斯lasso推导出几何收敛速率的显式上界。
• 证明新马尔可夫链为迹类（因此为希尔伯特-施密特类），意味着紧致性与可求和的特征值。
• 利用迹类性质，实现与插入辅助步骤以提升性能的夹心算法之间的理论比较。
• 通过利用其正态尺度混合表示，将该方法应用于全局-局部先验与点阵-板先验。
• 运用马尔可夫链理论中的理论工具，包括马尔可夫算子的谱性质，分析收敛行为。

实验结果

研究问题

• RQ1 在高维贝叶斯收缩模型中，两步分块吉布斯采样器是否能实现比标准三步采样器更快的收敛速度？
• RQ2 所提出的两步采样器的理论收敛速率为何？与原始链相比如何？
• RQ3 所提出的马尔可夫链是否为迹类？其谱性质与理论分析有何影响？
• RQ4 新链的迹类性质如何实现与夹心算法的严格理论比较？
• RQ5 在真实与模拟的高维数据上，新采样器在混合速度与不确定性量化方面是否优于原始采样器？

主要发现

• 所提出的两步分块吉布斯采样器在高维设置下，与原始三步采样器相比，表现出截然不同的优越收敛行为。
• 推导出几何收敛速率的显式上界，证明在新采样器下混合速度更快。
• 证明新马尔可夫链为迹类，意味着紧致性与可求和的特征值，而原始链不具备此性质。
• 迹类性质使得与夹心算法的严格理论比较成为可能，揭示了更优的收敛结构。
• 真实与模拟数据的实证结果证实了新采样器更快的混合速度与改进的不确定性量化。
• 该方法在保持与原始采样器相同后验分布的同时，显著提升了采样效率。