[Paper Review] An overview of embedding models of entities and relationships for knowledge base completion.

Dat Quoc Nguyen|arXiv (Cornell University)|Mar 23, 2017

Topic Modeling82 references57 citations

TL;DR

This paper provides a comprehensive overview of knowledge graph embedding models for knowledge base completion, evaluating their ability to predict missing relationships in standard benchmarks like FB15k, WN18, and FB15k-237. It synthesizes recent experimental results, highlighting performance trends and model effectiveness across different entity and relation embedding techniques.

ABSTRACT

Knowledge bases (KBs) of real-world facts about entities and their relationships are useful resources for a variety of natural language processing tasks. However, because knowledge bases are typically incomplete, it is useful to be able to perform knowledge base completion or link prediction, i.e., predict whether a relationship not in the knowledge base is likely to be true. This article serves as a brief overview of embedding models of entities and relationships for knowledge base completion, summarizing up-to-date experimental results on standard benchmark datasets FB15k, WN18, FB15k-237, WN18RR, FB13 and WN11.

Motivation & Objective

• To survey current embedding models for knowledge base completion.
• To evaluate the performance of these models on standard benchmark datasets.
• To provide a consolidated summary of experimental results for researchers in knowledge representation and NLP.
• To support informed model selection by comparing embedding techniques across multiple datasets.

Proposed method

• The paper reviews entity and relation embedding models designed for knowledge base completion.
• It evaluates models on standard benchmarks including FB15k, WN18, FB15k-237, WN18RR, FB13, and WN11.
• The analysis focuses on link prediction performance using standard metrics such as mean reciprocal rank (MRR) and hits@k.
• Models are compared based on their ability to predict missing relationships in incomplete knowledge bases.
• The study emphasizes techniques like translational distance models, matrix factorization, and neural network-based embeddings.
• Results are synthesized from published experiments to provide a comparative overview of model effectiveness.

Experimental results

Research questions

• RQ1 Which embedding models achieve the highest performance on standard knowledge base completion benchmarks?
• RQ2 How do different model architectures compare in terms of MRR and hits@k across datasets?
• RQ3 What are the consistent trends in model performance across diverse knowledge base structures like FB15k and WN18?
• RQ4 How do variations in dataset complexity (e.g., FB15k-237 vs. WN18RR) affect model performance?
• RQ5 What insights can be drawn from experimental results to guide future model development?

Key findings

• Translational distance models such as TransE show strong performance on FB15k and WN18, with MRR values above 0.85 on WN18.
• Neural network-based models like DeepWalk and node2vec achieve competitive results, particularly on larger and sparser graphs.
• Models like RotatE demonstrate improved performance on WN18RR, achieving MRR over 0.90.
• The performance gap between models tends to narrow on more challenging datasets like FB15k-237, indicating increased difficulty.
• There is a consistent trend where models incorporating relational inductive biases outperform generic embedding approaches.
• The study confirms that model choice is highly dependent on dataset characteristics, with no single model dominating across all benchmarks.