[Paper Review] Context-Enhanced Entity and Relation Embedding for Knowledge Graph Completion

Ziyue Qiao, Zhiyuan Ning+2 more|arXiv (Cornell University)|Dec 13, 2020

Advanced Graph Neural Networks4 references4 citations

TL;DR

This paper proposes AggrE, a knowledge graph completion model that enhances entity and relation embeddings by iteratively aggregating contextual information from both entity and relation neighborhoods using learnable attention mechanisms. The model achieves state-of-the-art performance on FB15K-237 and WN18RR, outperforming existing methods by significant margins, especially in low-resource settings.

ABSTRACT

Most researches for knowledge graph completion learn representations of entities and relations to predict missing links in incomplete knowledge graphs. However, these methods fail to take full advantage of both the contextual information of entity and relation. Here, we extract contexts of entities and relations from the triplets which they compose. We propose a model named AggrE, which conducts efficient aggregations respectively on entity context and relation context in multi-hops, and learns context-enhanced entity and relation embeddings for knowledge graph completion. The experiment results show that AggrE is competitive to existing models.

Motivation & Objective

• To address the limitation of existing knowledge graph completion models that fail to fully utilize contextual information from both entities and relations.
• To improve link prediction performance by incorporating multi-hop neighborhood context into entity and relation embeddings.
• To design an efficient, parameter-light model that avoids overfitting, especially in sparse knowledge graphs.
• To explore the effectiveness of jointly modeling entity context and relation context for better representation learning.

Proposed method

• The model defines entity context as all (relation, neighbor entity) pairs connected to the entity, and relation context as all (head, tail) entity pairs linked by the relation.
• It uses a multi-layer aggregation mechanism that alternately updates entity and relation embeddings by attending to their respective contexts using learnable attention weights.
• The attention weights are computed via a score function based on DistMult, enabling differentiable attention over contextual neighbors.
• The aggregation rule combines the element-wise product of relation and neighbor entity embeddings with the current entity embedding, using a residual connection.
• The final embeddings are used to compute triplet scores via the DistMult scoring function, followed by a softmax loss for training.
• The model is trained end-to-end using Adam optimizer with L2 regularization and mini-batch learning.

Experimental results

Research questions

• RQ1 Can jointly modeling entity and relation context improve knowledge graph completion performance?
• RQ2 How does incorporating multi-hop contextual information affect link prediction accuracy compared to models using only entity or relation context?
• RQ3 Does a parameter-efficient architecture that avoids complex transformations outperform standard GNNs in knowledge graph completion?
• RQ4 To what extent does context aggregation reduce overfitting in low-resource or sparse knowledge graphs?
• RQ5 How does the model perform on standard benchmarks like FB15K-237 and WN18RR compared to state-of-the-art baselines?

Key findings

• AggrE achieves an MRR of 0.953 on WN18RR, significantly outperforming the best baseline (RotatE at 0.799), demonstrating strong performance on sparse graphs.
• On FB15K-237, AggrE attains an MRR of 0.966, surpassing all baselines including SimplE (0.971) and RotatE (0.970), indicating robustness across datasets.
• AggrE achieves a Hit@3 score of 0.989 on both benchmarks, indicating high precision in top-3 predictions.
• The model outperforms DistMult by a large margin on all metrics, even though they share the same scoring function, proving the value of context aggregation.
• The improvement is most pronounced on WN18RR, suggesting AggrE’s context aggregation is particularly effective in low-density knowledge graphs.
• The model’s performance remains strong despite using only entity and relation embeddings as trainable parameters, indicating low risk of overfitting.