[论文解读] How Does Knowledge Graph Embedding Extrapolate to Unseen Data: A Semantic Evidence View

Ren Li, Yanan Cao+5 more|arXiv (Cornell University)|Sep 24, 2021

Advanced Graph Neural Networks被引用 7

一句话总结

本文提出语义证据（Semantic Evidence, SE）——一种新颖的框架，通过识别三个层次的语义支持来解释知识图嵌入（KGE）外推：关系级共现、实体级路径连接以及三元组级相似性。该框架提出SE-GNN，一种基于图神经网络（GNN）的KGE模型，通过结构化的邻居模式和多层聚合显式建模这些证据，在FB15k-237和WN18RR上实现了最先进性能，并展现出更强的外推能力。

ABSTRACT

Knowledge Graph Embedding (KGE) aims to learn representations for entities and relations. Most KGE models have gained great success, especially on extrapolation scenarios. Specifically, given an unseen triple (h, r, t), a trained model can still correctly predict t from (h, r, ?), or h from (?, r, t), such extrapolation ability is impressive. However, most existing KGE works focus on the design of delicate triple modeling function, which mainly tells us how to measure the plausibility of observed triples, but offers limited explanation of why the methods can extrapolate to unseen data, and what are the important factors to help KGE extrapolate. Therefore in this work, we attempt to study the KGE extrapolation of two problems: 1. How does KGE extrapolate to unseen data? 2. How to design the KGE model with better extrapolation ability? For the problem 1, we first discuss the impact factors for extrapolation and from relation, entity and triple level respectively, propose three Semantic Evidences (SEs), which can be observed from train set and provide important semantic information for extrapolation. Then we verify the effectiveness of SEs through extensive experiments on several typical KGE methods. For the problem 2, to make better use of the three levels of SE, we propose a novel GNN-based KGE model, called Semantic Evidence aware Graph Neural Network (SE-GNN). In SE-GNN, each level of SE is modeled explicitly by the corresponding neighbor pattern, and merged sufficiently by the multi-layer aggregation, which contributes to obtaining more extrapolative knowledge representation. Finally, through extensive experiments on FB15k-237 and WN18RR datasets, we show that SE-GNN achieves state-of-the-art performance on Knowledge Graph Completion task and performs a better extrapolation ability. Our code is available at https://github.com/renli1024/SE-GNN.

研究动机与目标

• 理解KGE模型能够泛化到未见三元组的潜在机制。
• 识别训练数据中支持外推到未见三元组的关键语义因素。
• 设计一种显式利用这些因素以提升外推性能的KGE模型。
• 通过在标准基准上的大量实验验证语义证据的有效性。

提出的方法

• 提出三种语义证据（SE）类型：关系级共现、实体级路径连接，以及候选实体与真实实体之间的三元组级相似性。
• 使用不同的度量指标量化每类SE：共现频率、最短路径数量以及嵌入相似度。
• 设计SE-GNN，一种基于GNN的KGE模型，通过独立的邻居模式编码每一层SE。
• 采用多层图聚合机制，融合并优化各层的SE表征。
• 在训练过程中应用边掩码，以防止信息泄露并模拟真实的外推场景。
• 通过引入逆关系和构建无向图，统一双向预测，以改善消息传递。

实验结果

研究问题

• RQ1 训练数据中的哪些语义因素使KGE模型能够外推到未见三元组？
• RQ2 关系级、实体级和三元组级语义证据如何分别贡献于外推性能？
• RQ3 显式建模语义证据是否能超越现有方法，提升KGE模型的泛化能力？
• RQ4 语义证据的强度与模型在未见数据上的性能之间是否存在相关性？

主要发现

• 语义证据（SE）度量——共现、路径连接和相似度——在多种KGE模型中均与外推性能强相关。
• 当对应SE值较高时，模型在未见三元组上的性能显著提升，尤其在实体和三元组层级表现更明显。
• SE-GNN在FB15k-237和WN18RR上均优于最先进KGE模型，展现出更优的外推能力。
• 消融实验表明，显式建模全部三类SE均有助于性能提升，其中在WN18RR中实体级和三元组级SE的影响更为显著，原因在于其关系集合较小。
• 在两个数据集上，不同KGE模型的SE强度与模型性能之间均保持一致的相关性，验证了SE框架的普适性。