[Paper Review] Knowledge Graph Embedding with Entity Neighbors and Deep Memory Network

Kai Wang, Yu Liu+2 more|ArXiv.org|Aug 11, 2018

Advanced Graph Neural NetworksComputer Science32 references17 citations

TL;DR

This paper proposes Neighborhood Knowledge Graph Embedding (NKGE), a novel KGE model that enhances entity representation by integrating structural and semantic information from entity neighbors—topological neighbors (direct KG connections) and semantic neighbors (mutually mentioned in descriptions). Using a deep memory network (DMN) to encode neighbor features and a gating mechanism to fuse them with structural embeddings, NKGE achieves state-of-the-art performance on link prediction across four benchmark datasets, outperforming methods using entity descriptions.

ABSTRACT

Knowledge Graph Embedding (KGE) aims to represent entities and relations of knowledge graph in a low-dimensional continuous vector space. Recent works focus on incorporating structural knowledge with additional information, such as entity descriptions, relation paths and so on. However, common used additional information usually contains plenty of noise, which makes it hard to learn valuable representation. In this paper, we propose a new kind of additional information, called entity neighbors, which contain both semantic and topological features about given entity. We then develop a deep memory network model to encode information from neighbors. Employing a gating mechanism, representations of structure and neighbors are integrated into a joint representation. The experimental results show that our model outperforms existing KGE methods utilizing entity descriptions and achieves state-of-the-art metrics on 4 datasets.

Motivation & Objective

• To address the limitations of noisy, unstructured additional information (e.g., entity descriptions, relation paths) in existing KGE methods.
• To propose a new form of additional information—entity neighbors—that combines semantic and topological features while reducing noise.
• To design a deep memory network (DMN) encoder to effectively extract latent representations from entity neighbors.
• To integrate neighbor representations with structural KGE models (TransE and ConvE) via a learnable gating mechanism for joint representation learning.
• To empirically validate the superiority of entity neighbors and the DMN encoder over traditional description-based methods.

Proposed method

• Entity neighbors are defined as a union of topological neighbors (entities directly connected via relations) and semantic neighbors (entities mentioned in each other’s descriptions).
• A deep memory network (DMN) is employed to encode neighbor information, learning contextual representations through multiple memory hops and attention mechanisms.
• A gating mechanism dynamically combines the structural representation (from TransE or ConvE) with the neighbor representation to produce a joint entity embedding.
• The model is trained end-to-end using negative sampling and stochastic gradient descent to optimize link prediction loss.
• Two variants are implemented: NKGE (TransE) and NKGE (ConvE), both incorporating neighbor encoding into their respective architectures.
• A controlled ablation study isolates the impact of neighbor representation and encoder design by comparing with CBOW and LSTM-based encoders on descriptions and neighbors.

Experimental results

Research questions

• RQ1 Can entity neighbors, combining semantic and topological features, serve as a more effective and less noisy form of additional information than traditional entity descriptions?
• RQ2 Does a deep memory network (DMN) outperform simpler encoders (e.g., CBOW, LSTM) in capturing complex patterns from neighbor entities?
• RQ3 Can the integration of neighbor representations via a learnable gating mechanism improve KGE performance on sparse or incomplete knowledge graphs?
• RQ4 How does the performance of NKGE compare to state-of-the-art KGE models that use entity descriptions?
• RQ5 To what extent do topological and semantic neighbors contribute independently, and is their combination more effective?

Key findings

• NKGE (TransE) achieves state-of-the-art Mean Rank (MR) on all four benchmark datasets, outperforming TransE-based baselines that use entity descriptions.
• NKGE (ConvE) achieves state-of-the-art Mean Reciprocal Rank (MRR) and Hits@N scores on most datasets, including surpassing the performance of ANALOGY on FB15k.
• On FB15k, NKGE (DMN + Nei) achieves MR = 50 and Hits@10 = 78.5, outperforming DKRL (MR = 91, Hits@10 = 67.4) and Jointly (MR = 73, Hits@10 = 75.5).
• The ablation study confirms that entity neighbors alone (NKGE (CBOW + Nei)) outperform description-based methods using the same encoder, demonstrating the value of neighbor representation.
• The DMN encoder significantly improves performance over CBOW and LSTM encoders, with NKGE (DMN + Des) achieving MR = 53 and Hits@10 = 76.0, outperforming Jointly (A_LSTM + Des).
• Combining both topological and semantic neighbors (NKGE (T&S)) yields the best performance (MR = 237, Hits@10 = 51.0), confirming the complementary nature of both neighbor types.