[论文解读] Robust MCMC Sampling with Non-Gaussian and Hierarchical Priors in High Dimensions

Victor Chen, Matthew M. Dunlop+2 more|arXiv (Cornell University)|Mar 9, 2018

Machine Learning and Algorithms被引用 12

一句话总结

本文针对具有非高斯和层次化先验的高维贝叶斯反问题，开发了鲁棒的马尔可夫链蒙特卡洛（MCMC）采样方法，利用标准白噪声表示法，确保收敛速率与离散化无关。该方法将鲁棒性从高斯先验扩展至贝索夫、深度高斯和分段常数等复杂先验，实现了无限维设定下的可扩展推断。

ABSTRACT

A key problem in inference for high dimensional unknowns is the design of sampling algorithms whose performance scales favourably with the dimension of the unknown. A typical setting in which these problems arise is the area of Bayesian inverse problems. In such problems, which include graph-based learning, nonparametric regression and PDE-based inversion, the unknown can be viewed as an infinite-dimensional parameter (such as a function) that has been discretised. This results in a high-dimensional space for inference. Here we study robustness of an MCMC algorithm for posterior inference; this refers to MCMC convergence rates that do not deteriorate as the discretisation becomes finer. When a Gaussian prior is employed there is a known methodology for the design of robust MCMC samplers. However, one often requires more flexibility than a Gaussian prior can provide: hierarchical models are used to enable inference of parameters underlying a Gaussian prior; or non-Gaussian priors, such as Besov, are employed to induce sparse MAP estimators; or deep Gaussian priors are used to represent other non-Gaussian phenomena; and piecewise constant functions, which are necessarily non-Gaussian, are required for classification problems. The purpose of this article is to show that the simulation technology available for Gaussian priors can be exported to such non-Gaussian priors. The underlying methodology is based on a white noise representation of the unknown. This is exploited both for robust posterior sampling and for joint inference of the function and parameters involved in the specification of its prior, in which case our framework borrows strength from the well-developed non-centred methodology for Bayesian hierarchical models. The desired robustness of the proposed sampling algorithms is supported by some theory and by extensive numerical evidence from several challenging problems.

研究动机与目标

• 为解决在贝叶斯反问题中，随着未知量维度增加，MCMC采样器收敛速率下降的挑战。
• 将先前在高斯先验下建立的MCMC采样鲁棒性，扩展至非高斯和层次化先验，如贝索夫、深度高斯和分段常数先验。
• 通过利用非中心化参数化技术，实现层次化模型中函数与超参数的联合推断。
• 在基于PDE的反演、非参数回归和图基学习中出现的高维、无限维推断问题中，确保可扩展性和鲁棒性。
• 为所提出的采样框架在多样且具有挑战性的反问题中的鲁棒性，提供理论和数值支持。

提出的方法

• 利用未知量的白噪声表示对先验和后验进行重参数化，实现维度无关的采样。
• 采用非中心化参数化策略，稳定后验几何结构，提升层次化模型中MCMC的混合效率。
• 通过利用白噪声基，将高斯先验的处理方法拓展至非高斯先验，实现一致的先验设定。
• 设计MCMC算法（如预处理Crank-Nicolson或随机游走Metropolis-Hastings），在离散化程度增加时仍保持鲁棒收敛。
• 通过将先验超参数视为层次贝叶斯模型中的未知量，实现函数与超参数的联合推断。
• 利用白噪声框架将先验结构与采样算法解耦，确保在不同先验类型下均保持稳定性。

实验结果

研究问题

• RQ1 能否将专为高斯先验设计的MCMC采样器，适配至非高斯先验，以在高维推断中保持鲁棒性？
• RQ2 如何在不降低收敛速率的前提下，将层次化先验整合进鲁棒的MCMC采样中？
• RQ3 白噪声表示在多大程度上能够实现对贝索夫或分段常数等多样化非高斯先验结构的鲁棒性？
• RQ4 非中心化参数化策略能否推广至非高斯先验，以改善MCMC的混合与收敛性能？
• RQ5 有哪些理论和数值证据支持所提出采样框架的维度无关收敛性？

主要发现

• 所提出的MCMC采样器即使在使用贝索夫或分段常数函数等非高斯先验时，也能保持与离散化水平无关的鲁棒收敛速率。
• 白噪声表示法在包括深度高斯和层次化先验在内的多种先验类型中，实现了稳定且一致的采样。
• 通过非中心化参数化策略，实现了函数与超参数的稳定混合与收敛的联合推断。
• 理论分析支持采样算法的鲁棒性，收敛速率在维度上保持一致有界。
• 在基于PDE的反演、非参数回归和图基学习中的大量数值实验表明，随着离散化程度增加，性能依然稳定。
• 该框架成功将MCMC采样的鲁棒性从高斯先延展至非高斯先验，实现了无限维贝叶斯问题中的可扩展推断。