[Paper Review] RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space

Zhiqing Sun, Zhihong Deng+2 more|arXiv (Cornell University)|Feb 26, 2019

Advanced Graph Neural Networks771 citations

TL;DR

RotatE models relations as rotations in complex space, enabling inference of symmetry/antisymmetry, inversion, and composition patterns; it surpasses state-of-the-art on multiple KG benchmarks and introduces self-adversarial negative sampling.

ABSTRACT

We study the problem of learning representations of entities and relations in knowledge graphs for predicting missing links. The success of such a task heavily relies on the ability of modeling and inferring the patterns of (or between) the relations. In this paper, we present a new approach for knowledge graph embedding called RotatE, which is able to model and infer various relation patterns including: symmetry/antisymmetry, inversion, and composition. Specifically, the RotatE model defines each relation as a rotation from the source entity to the target entity in the complex vector space. In addition, we propose a novel self-adversarial negative sampling technique for efficiently and effectively training the RotatE model. Experimental results on multiple benchmark knowledge graphs show that the proposed RotatE model is not only scalable, but also able to infer and model various relation patterns and significantly outperform existing state-of-the-art models for link prediction.

Motivation & Objective

• Motivate and address the limitation of existing KG embedding models in capturing all key relation patterns (symmetry/antisymmetry, inversion, composition).
• Propose RotatE to model relations as rotations in complex vector space and demonstrate its capacity to infer multiple relation patterns.
• Show that RotatE is scalable to large KGs and outperforms state-of-the-art baselines on standard benchmarks.

Proposed method

• Embed entities and relations in complex space and define each relation as an element-wise rotation: t = h ∘ r with |r_i| = 1 for all i.
• Use a distance-based score d_r(h,t) = || h ∘ r − t || to evaluate triplets.
• Leverage self-adversarial negative sampling to efficiently train RotatE by weighting negative samples with a temperature-based distribution proportional to f_r(h′,t′).
• Provide theoretical lemmas showing RotatE can model symmetry/antisymmetry, inversion, and composition patterns.
• Train with Adam, grid-search hyperparameters, and evaluate under a filtered setting on multiple benchmarks.

Experimental results

Research questions

• RQ1 Can RotatE model and infer symmetry/antisymmetry, inversion, and composition patterns simultaneously?
• RQ2 Does rotating embeddings in complex space improve link prediction over existing models across standard KG benchmarks?
• RQ3 What is the impact of self-adversarial negative sampling on training efficiency and predictive performance?
• RQ4 How does RotatE perform on datasets designed to stress composition patterns (e.g., FB15k-237, Countries) compared to other models?

Key findings

• RotatE outperforms state-of-the-art models on FB15k, WN18, FB15k-237, and WN18RR for link prediction.
• RotatE significantly improves on composition-pattern heavy datasets (FB15k-237, WN18RR) compared to baselines, with modulus information being important for composition modeling.
• Self-adversarial negative sampling improves performance over uniform sampling and over KBGAN baselines.
• Empirical analysis shows RotatE can implicitly encode symmetry via r_i ∈ {±1}, and inverse relations via conjugate embeddings, supporting theoretical claims.
• On Countries dataset, RotatE achieves competitive or superior results, particularly on tasks requiring longer composition.