[论文解读] Toward Understanding The Effect of Loss Function on The Performance of Knowledge Graph Embedding

Mojtaba Nayyeri, Chengjin Xu+3 more|arXiv (Cornell University)|Sep 25, 2019

Advanced Graph Neural Networks参考文献 28被引用 8

一句话总结

本文揭示了损失函数的选择对知识图嵌入中TransE性能具有决定性影响，挑战了以往认为其局限性仅源于打分函数的假设。通过理论分析与实验验证损失函数，作者表明恰当的损失选择可有效缓解TransE在捕捉对称性和自反性等关系模式方面的不足。

ABSTRACT

Knowledge graphs (KGs) represent world's facts in structured forms. KG completion exploits the existing facts in a KG to discover new ones. Translation-based embedding model (TransE) is a prominent formulation to do KG completion. Despite the efficiency of TransE in memory and time, it suffers from several limitations in encoding relation patterns such as symmetric, reflexive etc. To resolve this problem, most of the attempts have circled around the revision of the score function of TransE i.e., proposing a more complicated score function such as Trans(A, D, G, H, R, etc) to mitigate the limitations. In this paper, we tackle this problem from a different perspective. We show that existing theories corresponding to the limitations of TransE are inaccurate because they ignore the effect of loss function. Accordingly, we pose theoretical investigations of the main limitations of TransE in the light of loss function. To the best of our knowledge, this has not been investigated so far comprehensively. We show that by a proper selection of the loss function for training the TransE model, the main limitations of the model are mitigated. This is explained by setting upper-bound for the scores of positive samples, showing the region of truth (i.e., the region that a triple is considered positive by the model). Our theoretical proofs with experimental results fill the gap between the capability of translation-based class of embedding models and the loss function. The theories emphasise the importance of the selection of the loss functions for training the models. Our experimental evaluations on different loss functions used for training the models justify our theoretical proofs and confirm the importance of the loss functions on the performance.

研究动机与目标

• 探究TransE为何无法捕捉诸如对称关系和自反关系等复杂关系模式。
• 挑战当前普遍认为TransE的局限性仅源于其打分函数而非损失函数的观点。
• 从理论上分析不同损失函数如何影响模型学习正确三元组打分的能力。
• 证明仅通过损失函数的选择即可缓解TransE的核心局限，而无需修改打分函数。
• 弥合基于翻译的模型理论能力与损失函数在训练中作用之间的差距。

提出的方法

• 提出一个理论框架，分析损失函数对TransE性能的影响，重点关注正样本三元组打分的上界。
• 将“真实区域”定义为模型将三元组分类为正样本的嵌入集合。
• 通过理论证明表明，特定损失函数可约束打分分布，使正样本被限制在理想区域内。
• 引入形式化分析，建立损失函数选择与模型在关系模式上泛化能力之间的关联。
• 在多个标准知识图基准上开展受控实验，评估不同损失函数的性能。
• 通过测量均倒数排名（MRR）和Hits@10等性能指标，验证理论假设。

实验结果

研究问题

• RQ1 损失函数的选择在多大程度上影响TransE对对称关系和自反关系的建模能力？
• RQ2 为何现有关于TransE局限性的理论未能考虑损失函数的作用？
• RQ3 是否可通过合适的损失函数弥补TransE打分函数的固有局限？
• RQ4 损失函数与知识图嵌入中“真实区域”之间的理论关系是什么？
• RQ5 在不修改打分函数的前提下，损失函数的选择能在多大程度上提升性能？

主要发现

• 理论分析表明，损失函数直接控制正样本三元组打分的上界，从而塑造真实区域。
• 基于边界（margin-based）排序损失和基于Sigmoid的损失显著提升了模型在对称和自反关系上的性能。
• 实验结果证实，即使使用相同的打分函数，不同损失函数下的模型性能存在显著差异。
• 损失函数的选择可缓解TransE在捕捉复杂关系模式方面的不足，与以往假设相反。
• 理论洞见解释了为何某些损失函数能带来更好的泛化性能，并提升MRR和Hits@10得分。
• 本研究弥合了理解损失函数在基于翻译的知识图嵌入模型中作用的关键空白。