[Paper Review] Quaternion Knowledge Graph Embeddings

Shuai Zhang, Yi Tay+2 more|arXiv (Cornell University)|Apr 23, 2019

Advanced Graph Neural NetworksComputer Science33 references206 citations

TL;DR

QuatE introduces quaternion-valued embeddings for entities and relations in knowledge graphs, enabling relational rotations in hypercomplex space to capture symmetry, antisymmetry, and inversion, achieving state-of-the-art results on four benchmarks.

ABSTRACT

In this work, we move beyond the traditional complex-valued representations, introducing more expressive hypercomplex representations to model entities and relations for knowledge graph embeddings. More specifically, quaternion embeddings, hypercomplex-valued embeddings with three imaginary components, are utilized to represent entities. Relations are modelled as rotations in the quaternion space. The advantages of the proposed approach are: (1) Latent inter-dependencies (between all components) are aptly captured with Hamilton product, encouraging a more compact interaction between entities and relations; (2) Quaternions enable expressive rotation in four-dimensional space and have more degree of freedom than rotation in complex plane; (3) The proposed framework is a generalization of ComplEx on hypercomplex space while offering better geometrical interpretations, concurrently satisfying the key desiderata of relational representation learning (i.e., modeling symmetry, anti-symmetry and inversion). Experimental results demonstrate that our method achieves state-of-the-art performance on four well-established knowledge graph completion benchmarks.

Motivation & Objective

• Motivate modeling entities and relations in a richer hypercomplex space beyond complex-valued representations.
• Propose a quaternion-based scoring function that rotates head entities via relation quaternions and assesses tail compatibility.
• Demonstrate that quaternion rotations capture symmetry, antisymmetry, and inversion more effectively than prior models.
• Show that QuatE achieves state-of-the-art results on multiple KG completion benchmarks with competitive parameter efficiency.

Proposed method

• Represent entities with quaternion embeddings Q in H^{N x k}.
• Represent relations with unit quaternion embeddings W_r in H^{M x k}, normalized to remove scaling effects.
• Rotate the head entity by the unit relation quaternion via Hamilton product: Q_h' = Q_h ⊗ W_r^{triangleleft}.
• Score a triplet with quaternion inner product: φ(h,r,t) = Q_h' · Q_t.
• Train with regularized logistic loss over observed and negative triplets, using Adagrad for optimization.
• Discuss initialization tailored for quaternion networks and optional normalization/regularization variants.

Experimental results

Research questions

• RQ1 Can quaternion-valued embeddings with Hamilton product rotations outperform complex-valued KG embeddings on standard benchmarks?
• RQ2 Do unit-quaternion normalizations and rotational quaternion formulations improve modeling of symmetry, antisymmetry, and inversion in relations?
• RQ3 How does QuatE compare to state-of-the-art models in parameter efficiency and predictive accuracy across datasets?
• RQ4 Is the QuatE composition flexible enough to capture multiple relation patterns without fixed composition operators?

Key findings

• QuatE achieves state-of-the-art performance on four KG benchmarks (WN18, FB15K, WN18RR, FB15K-237).
• Quaternion rotations provide richer interactions than complex-valued rotations, improving modeling of symmetry, antisymmetry, and inversion.
• Normalization of relation quaternions is critical; removing normalization degrades performance.
• QuatE requires fewer parameters than several baselines while delivering strong results, and variants with N3 and reciprocal learning further boost performance on some datasets.
• The framework generalizes ComplEx and can degenerate to DistMult under certain simplifications, offering a geometric interpretation beyond Hermitian products.