[论文解读] Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning

Wen Zhang, Bibek Paudel+6 more|Zurich Open Repository and Archive (University of Zurich)|Mar 21, 2019

Advanced Graph Neural Networks参考文献 44被引用 20

一句话总结

本文提出 IterE，一种通过相互增强方式迭代学习知识图嵌入与逻辑规则的框架。通过利用嵌入指导规则发现，并利用规则改进稀疏实体表示，IterE 在链接预测性能方面优于当前最优方法（如 AMIE+），同时实现了更高效、更高质量的规则生成。

ABSTRACT

Reasoning is essential for the development of large knowledge graphs, especially for completion, which aims to infer new triples based on existing ones. Both rules and embeddings can be used for knowledge graph reasoning and they have their own advantages and difficulties. Rule-based reasoning is accurate and explainable but rule learning with searching over the graph always suffers from efficiency due to huge search space. Embedding-based reasoning is more scalable and efficient as the reasoning is conducted via computation between embeddings, but it has difficulty learning good representations for sparse entities because a good embedding relies heavily on data richness. Based on this observation, in this paper we explore how embedding and rule learning can be combined together and complement each other's difficulties with their advantages. We propose a novel framework IterE iteratively learning embeddings and rules, in which rules are learned from embeddings with proper pruning strategy and embeddings are learned from existing triples and new triples inferred by rules. Evaluations on embedding qualities of IterE show that rules help improve the quality of sparse entity embeddings and their link prediction results. We also evaluate the efficiency of rule learning and quality of rules from IterE compared with AMIE+, showing that IterE is capable of generating high quality rules more efficiently. Experiments show that iteratively learning embeddings and rules benefit each other during learning and prediction.

研究动机与目标

• 为解决知识图嵌入学习中的稀疏性问题，即由于训练三元组数量有限，导致罕见实体表示质量差。
• 克服规则学习的计算低效问题，该问题在探索潜在规则结构时面临巨大的搜索空间。
• 开发一种联合学习框架，使嵌入与规则相互改进，结合嵌入的可扩展性与规则的可解释性。
• 设计一种剪枝策略，利用嵌入语义减少规则搜索空间，同时保持高质量的规则归纳。

提出的方法

• IterE 在两个阶段间交替进行：(1) 使用选定的嵌入模型（如 TransE、ComplEx）从现有三元组和规则推断的三元组中学习实体与关系嵌入；(2) 基于 OWL2 对象属性公理的语义规则格式，从当前嵌入中归纳新规则。
• 在规则归纳过程中应用一种新颖的剪枝策略，提前过滤掉低置信度或冗余的规则，显著减少搜索空间，同时不损失规则质量。
• 根据规则的置信度与语义合理性对规则进行评分，其中置信度基于知识图中支持该规则的三元组数量计算。
• 将新发现的规则重新注入知识图，丰富后续嵌入学习迭代的训练数据。
• 该方法兼容多种嵌入模型，并支持不同类型逻辑规则，尤其基于线性映射假设的规则，以增强与规则学习的兼容性。

实验结果

研究问题

• RQ1 嵌入学习与规则学习之间的迭代优化能否提升知识图中稀疏实体的表示质量？
• RQ2 基于嵌入的引导能否在保持或提升规则质量的同时降低规则学习的计算成本？
• RQ3 嵌入与规则之间的相互反馈机制是否能带来优于单独学习任一组件的整体推理性能？
• RQ4 基于 OWL2 公理的语义规则集成如何影响推断知识的质量与可解释性？

主要发现

• 通过引入提供额外训练信号的规则，IterE 显著提升了稀疏实体的链接预测性能，缩小了高频实体与稀有实体之间的性能差距。
• 与 AMIE+ 相比，该框架生成的规则质量更高，置信度与覆盖度更优，且由于基于嵌入的有效剪枝，计算时间更短。
• 迭代优化在多轮迭代中持续提升了嵌入质量与规则质量，表明两个组件之间存在正向反馈循环。
• 基于 OWL2 的对象属性公理使框架能够发现语义上合理且可解释的规则，符合标准知识表示原则。
• 在 FB15k-237 及其他基准数据集上的实验表明，IterE 在链接预测与规则质量指标上均优于独立的嵌入与规则学习基线方法。