[论文解读] TorusE: Knowledge Graph Embedding on a Lie Group

Takuma Ebisu, Ryutaro Ichise|arXiv (Cornell University)|Nov 15, 2017

Advanced Graph Neural Networks参考文献 21被引用 11

一句话总结

TorusE 提出了一种新颖的知识图谱嵌入模型，用环面——一种紧致李群——替代 TransE 的向量空间，从而在不依赖正则化的情况下保留了平移原则。通过将实体和关系嵌入环面，TorusE 在链接预测任务上实现了最先进性能（例如，在 WN18 上 HITS@1 达 94.3%，在 FB15K 上达 67.4%），在准确率和速度上均优于 TransE、DistMult 和 ComplEx。

ABSTRACT

Knowledge graphs are useful for many artificial intelligence (AI) tasks. However, knowledge graphs often have missing facts. To populate the graphs, knowledge graph embedding models have been developed. Knowledge graph embedding models map entities and relations in a knowledge graph to a vector space and predict unknown triples by scoring candidate triples. TransE is the first translation-based method and it is well known because of its simplicity and efficiency for knowledge graph completion. It employs the principle that the differences between entity embeddings represent their relations. The principle seems very simple, but it can effectively capture the rules of a knowledge graph. However, TransE has a problem with its regularization. TransE forces entity embeddings to be on a sphere in the embedding vector space. This regularization warps the embeddings and makes it difficult for them to fulfill the abovementioned principle. The regularization also affects adversely the accuracies of the link predictions. On the other hand, regularization is important because entity embeddings diverge by negative sampling without it. This paper proposes a novel embedding model, TorusE, to solve the regularization problem. The principle of TransE can be defined on any Lie group. A torus, which is one of the compact Lie groups, can be chosen for the embedding space to avoid regularization. To the best of our knowledge, TorusE is the first model that embeds objects on other than a real or complex vector space, and this paper is the first to formally discuss the problem of regularization of TransE. Our approach outperforms other state-of-the-art approaches such as TransE, DistMult and ComplEx on a standard link prediction task. We show that TorusE is scalable to large-size knowledge graphs and is faster than the original TransE.

研究动机与目标

• 为解决 TransE 中的正则化问题，该问题会扭曲实体嵌入并降低链接预测准确率。
• 探索能够保留 TransE 平移原则但无需正则化的替代嵌入空间。
• 证明紧致李群（尤其是环面）适用于知识图谱嵌入。
• 开发一种可扩展的高性能模型，在标准基准上超越现有最先进方法。

提出的方法

• 用环面（一种紧致李群）替代 TransE 中的实向量空间，以自然方式约束嵌入而无需正则化。
• 将 TransE 原则 $\mathbf{h} + \mathbf{r} = \mathbf{t}$ 适配到环面上的群运算，嵌入空间中使用模运算。
• 采用基于参数化旋转的变换，将关系建模为环面群的元素。
• 使用基于边距的排序损失函数进行模型优化，与 TransE 类似，但无需球面正则化。
• 利用环面的紧致性，防止训练过程中嵌入发散。
• 通过高效的矩阵运算和模运算确保计算可扩展性。

实验结果

研究问题

• RQ1 能否用环面等紧致李群替代 TransE 中的向量空间，从而消除对正则化的依赖？
• RQ2 在环面上进行嵌入是否能在保留 TransE 平移原则的同时提升链接预测准确率？
• RQ3 在性能和速度方面，TorusE 与 TransE、DistMult 和 ComplEx 等最先进模型相比如何？
• RQ4 基于李群的嵌入空间能否减少模型冗余并提升泛化能力？

主要发现

• 在 WN18 数据集上，TorusE 达到 94.3% HITS@1，优于 ComplEx（93.6%）和 TransE（68.1%）。
• 在 FB15K 上，TorusE 达到 67.4% HITS@1，显著优于 TransE（44.3%）和 ComplEx（65.4%）。
• 由于消除了正则化计算并采用高效的模运算，TorusE 比 TransE 更快。
• 该模型在多样化关系上泛化良好，多数 WN18 关系的 MRR 得分均等于或优于 ComplEx。
• 高维环面嵌入空间减少了过拟合，并支持更丰富的表征学习。
• TorusE 表明 TransE 原理本身并非固有缺陷；问题根源在于正则化，而非该原则本身。