[论文解读] Efficient Inference and Learning in a Large Knowledge Base: Reasoning with Extracted Information using a Locally Groundable First-Order Probabilistic Logic

William Yang Wang, Kathryn Mazaitis+3 more|arXiv (Cornell University)|Apr 12, 2014

Bayesian Modeling and Causal Inference参考文献 1被引用 8

一句话总结

该论文提出了ProPPR，一种一阶概率逻辑，通过使用可局部生成的、基于个性化PageRank的证明过程，实现了在大型知识库中的高效推理与学习。通过偏向于短推导路径并利用近似PageRank计算，ProPPR实现了与数据库规模无关的推理与学习时间——相较于马尔可夫逻辑网络展现出数量级的加速，并可扩展至包含百万数量级实体及数百个相互依赖谓词的知识库。

ABSTRACT

One important challenge for probabilistic logics is reasoning with very large knowledge bases (KBs) of imperfect information, such as those produced by modern web-scale information extraction systems. One scalability problem shared by many probabilistic logics is that answering queries involves "grounding" the query---i.e., mapping it to a propositional representation---and the size of a "grounding" grows with database size. To address this bottleneck, we present a first-order probabilistic language called ProPPR in which that approximate "local groundings" can be constructed in time independent of database size. Technically, ProPPR is an extension to stochastic logic programs (SLPs) that is biased towards short derivations; it is also closely related to an earlier relational learning algorithm called the path ranking algorithm (PRA). We show that the problem of constructing proofs for this logic is related to computation of personalized PageRank (PPR) on a linearized version of the proof space, and using on this connection, we develop a proveably-correct approximate grounding scheme, based on the PageRank-Nibble algorithm. Building on this, we develop a fast and easily-parallelized weight-learning algorithm for ProPPR. In experiments, we show that learning for ProPPR is orders magnitude faster than learning for Markov logic networks; that allowing mutual recursion (joint learning) in KB inference leads to improvements in performance; and that ProPPR can learn weights for a mutually recursive program with hundreds of clauses, which define scores of interrelated predicates, over a KB containing one million entities.

研究动机与目标

• 为解决概率一阶逻辑中的可扩展性瓶颈问题，即完全生成过程随数据库规模增长而受限，从而影响大规模知识库上的推理效率。
• 开发一种一阶概率逻辑，支持在包含噪声抽取事实的网络规模知识库中实现高效、近似的推理与学习。
• 通过将学习过程分解为每个训练样本的局部生成子任务，实现快速、可并行化的权重学习。
• 建立基于个性化PageRank与局部划分的近似生成的理论基础，确保正确性与有界误差。

提出的方法

• 在随机逻辑程序（SLPs）基础上扩展重启机制，偏向于短推导路径，从而构建一种可局部生成的逻辑。
• 将证明搜索建模为线性化证明空间上的个性化PageRank（PPR）计算，实现高效的近似推理。
• 使用PageRank-Nibble算法在O(1/(αε))时间内计算局部生成，与数据库规模无关，其中α为重启概率，ε为误差容限。
• 设计基于梯度的权重学习算法，按查询划分学习过程，实现易于并行化与高效优化。
• 采用局部划分技术（Andersen et al., 2006, 2008），为每个查询隔离相关子图，降低计算成本。
• 将生成过程与参数学习集成，支持在保持可扩展性的前提下迭代优化模型权重。

实验结果

研究问题

• RQ1 能否设计一种一阶概率逻辑，实现在无需完全生成的前提下，在大规模知识库中高效推理与学习？
• RQ2 是否可能通过基于局部化PageRank的证明过程，实现可证明正确的近似推理？
• RQ3 能否通过局部生成将权重学习分解为独立、可并行化的子任务，从而实现可扩展的优化？
• RQ4 在大规模知识库上，ProPPR相较于马尔可夫逻辑网络等现有方法在速度与准确性方面表现如何？
• RQ5 联合学习相互递归的谓词是否相比单独学习每个谓词能提升性能？

主要发现

• ProPPR实现推理与学习的时间复杂度为O(1/(αε))，与数据库规模无关，支持在大规模知识库上实现可扩展推理。
• ProPPR的学习速度相较马尔可夫逻辑网络快出数量级，实验中观察到10倍加速。
• ProPPR在仅数秒内即可完成对包含百万数量级实体的知识库中数百个子句的相互递归程序的权重学习，且运行于桌面计算机上。
• 对相互关联谓词进行联合推理与学习，相比单独学习每个谓词，能显著提升性能。
• 局部生成的推理机制支持有效并行化，通过多线程处理实现学习速度的额外一个数量级加速。
• 该方法在实体消解与分类任务中表现出色，所有中间证明阶段的近似误差均被控制在ε以内。