[论文解读] A Bayesian View of the Poisson-Dirichlet Process

Wray Buntine, Marcus Hütter|arXiv (Cornell University)|Jul 2, 2010

Bayesian Methods and Mixture Models参考文献 27被引用 48

一句话总结

本文通过推导样本大小为N时不同物种数量（M）分布的递归表征，为Poisson-Dirichlet过程提供了贝叶斯解释。研究证明广义Stirling数S(N,M; -1,-a,0)精确匹配归一化的概率质量函数p(M|N)，并通过递归关系与边界条件，为该过程建立了组合与分析基础。

ABSTRACT

The two parameter Poisson-Dirichlet Process (PDP), a generalisation of the Dirichlet Process, is increasingly being used for probabilistic modelling in discrete areas such as language technology, bioinformatics, and image analysis. There is a rich literature about the PDP and its derivative distributions such as the Chinese Restaurant Process (CRP). This article reviews some of the basic theory and then the major results needed for Bayesian modelling of discrete problems including details of priors, posteriors and computation. The PDP allows one to build distributions over countable partitions. The PDP has two other remarkable properties: first it is partially conjugate to itself, which allows one to build hierarchies of PDPs, and second using a marginalised relative the CRP, one gets fragmentation and clustering properties that lets one layer partitions to build trees. This article presents the basic theory for understanding the notion of partitions and distributions over them, the PDP and the CRP, and the important properties of conjugacy, fragmentation and clustering, as well as some key related properties such as consistency and convergence. This article also presents a Bayesian interpretation of the Poisson-Dirichlet process based on an improper and infinite dimensional Dirichlet distribution. This means we can understand the process as just another Dirichlet and thus all its sampling properties emerge naturally. The theory of PDPs is usually presented for continuous distributions (more generally referred to as non-atomic distributions), however, when applied to discrete distributions its remarkable conjugacy property emerges. This context and basic results are also presented, as well as techniques for computing the second order Stirling numbers that occur in the posteriors for discrete distributions.

研究动机与目标

• 通过样本中不同物种数量的分布，为Poisson-Dirichlet过程提供贝叶斯解释。
• 基于预测抽样动态，推导p(M|N)的递归公式。
• 建立物种数量分布与广义Stirling数S(N,M; -1,-a,0)之间的等价性。
• 通过显式表达式验证分布的边界条件与渐近行为。

提出的方法

• 利用Dirichlet过程的预测分布，推导p(M_{N+1} = m | M_N)的递归关系。
• 使用引理LABEL:lem-exp中的显式形式p(M_N = m) = S_{m,a}^N (b|a)^m / (b)_N。
• 应用该递归关系，推导出递推关系：S_{m,a}^{N+1} = S_{m-1,a}^N + (N - m a) S_{m,a}^N。
• 将广义Stirling数S(n,k; @, β,r)与参数(-1,-a,0)识别为与物种数量分布相匹配。
• 通过定义与组合解释，验证边界条件S_{m,a}^N = 0（当m > N时）与S_{0,a}^N = δ_{N,0}。
• 通过与偏导数及插值的关系，展示在a → 0极限下的连续性。

实验结果

研究问题

• RQ1 如何利用贝叶斯非参数方法表征样本中不同物种数量的分布？
• RQ2 随着样本量增加，物种数量转移概率的递归结构是什么？
• RQ3 广义Stirling数S(N,M; -1,-a,0)如何与物种数量的归一化概率质量函数相关联？
• RQ4 参数a与b在塑造物种分布及其递归关系中起什么作用？
• RQ5 当a → 0时，如何恢复物种数量分布的偏导数形式？

主要发现

• 从预测抽样分布推导出p(M_{N+1} = m)的递归关系，其与递推关系S_{m,a}^{N+1} = S_{m-1,a}^N + (N - m a) S_{m,a}^N完全一致。
• 证明广义Stirling数S(N,M; -1,-a,0)在Poisson-Dirichlet过程中等于归一化概率p(M_N = m)。
• 通过显式公式与过程解释，确认边界条件S_{m,a}^N = 0（当m > N时）与S_{0,a}^N = δ_{N,0}。
• 表明a = 0的情况对应于通过插值得到的M阶偏导数，从而连接离散与连续形式。
• 通过参数代入与递归匹配，严格建立了物种数量分布与广义Stirling数表达式的等价性。