[论文解读] Semantic Triple Encoder for Fast Open-Set Link Prediction.

Bo Wang, Tao Shen+3 more|arXiv (Cornell University)|Apr 30, 2020

Advanced Graph Neural Networks参考文献 55被引用 5

一句话总结

本文提出了一种语义三元组编码器，通过类似孪生网络的Transformer架构，对实体和关系的上下文语义进行建模，从而提升开放集链接预测性能。通过将三元组拆分为头实体+关系和尾实体上下文两部分，该方法在推理成本较以往基于上下文的方法降低一到两个数量级的同时，实现了最先进性能。

ABSTRACT

We improve both the open-set generalization and efficiency of link prediction on knowledge graphs by leveraging the contexts of entities and relations in a novel semantic triple encoder. Most previous methods, e.g., translation-based and GCN-based embedding approaches, were built upon graph embedding models. They simply treat the entities/relations as a closed set of graph nodes regardless of their context semantics, which however cannot provide critical information for the generalization to unseen entities/relations. In this paper, we partition each graph triple and develop a novel context-based encoder that separately maps each part and its context into a latent semantic space. We train this semantic triple encoder by optimizing two objectives specifically designed for link prediction. In particular, (1) We split each triple into two parts, i.e., i) head entity plus relation and ii) tail entity, process both contexts separately by a Transformer encoder, and combine the encoding outputs to derive the prediction. This Siamese-like architecture avoids the combinatorial explosion of candidate triples and significantly improves the efficiency, especially during inference; (2) We cover the contextualized semantics of the triples in the encoder so it can handle unseen entities during inference, which promisingly improves the generalization ability; (3) We train the model by optimizing two complementary objectives defined on the triple, i.e., classification and contrastive losses, for natural and reliable ranking scores during inference. In experiments, we achieve the state-of-the-art or competitive performance on three popular link prediction benchmarks. In addition, we empirically reduce the inference costs by one or two orders of magnitude compared to a recent context-based encoding approach and meanwhile keep a superior quality of prediction.

研究动机与目标

• 解决现有图嵌入模型将实体和关系视为封闭集合的局限性，忽略对泛化至关重要的上下文语义。
• 通过编码三元组的上下文语义，提升在开放世界知识图谱中对未见实体和关系的泛化能力。
• 通过孪生编码器设计，避免候选三元组的组合爆炸，提升推理效率。
• 通过双目标优化——分类损失与对比损失——获得在链接预测中更鲁棒、更可靠的排序分数。

提出的方法

• 将每个三元组划分为两部分：(i) 头实体与关系，以及 (ii) 尾实体，并分别使用Transformer编码器对两部分进行编码。
• 采用类似孪生网络的架构，两部分独立编码，其表示拼接后用于最终预测。
• 使用两个互补的目标进行模型训练：分类损失用于预测正样本/负样本链接，对比损失用于提升语义分离效果。
• 利用实体和关系的上下文表示，使模型在推理阶段能有效泛化到未见实体。
• 为两部分使用共享的编码器头，以确保参数效率和一致的表示学习。
• 将两部分的编码表示合并为单一向量，用于链接预测的打分。

实验结果

研究问题

• RQ1 上下文感知的编码器架构能否提升在开放集链接预测中对未见实体和关系的泛化能力？
• RQ2 与传统方法相比，语义三元组编码器的孪生设计在多大程度上降低了推理成本？
• RQ3 双训练目标（分类损失与对比损失）在多大程度上提升了排序质量与鲁棒性？
• RQ4 该模型能否在显著降低推理时间的同时保持高预测准确率？

主要发现

• 所提模型在三个标准链接预测基准上实现了最先进或具有竞争力的性能。
• 与近期基于上下文的编码方法相比，推理成本降低了1至2个数量级。
• 由于其上下文表示机制，模型能有效泛化到未见的实体和关系。
• 分类损失与对比损失的结合，使推理过程中的排序分数更加可靠且自然。
• 孪生架构避免了候选三元组的组合爆炸，显著提升了推理效率。