[Paper Review] QuatRE: Relation-Aware Quaternions for Knowledge Graph Embeddings

Dai Quoc Nguyen, Thanh Vu+2 more|arXiv (Cornell University)|Sep 26, 2020

Advanced Graph Neural Networks4 citations

TL;DR

QuatRE proposes a novel knowledge graph embedding model that enhances entity-relation correlations by applying two relation-aware quaternion rotations to head and tail entities using the Hamilton product. It achieves state-of-the-art performance on standard benchmarks like WN18RR and FB15k-237, outperforming prior models including QuatE and RotatE.

ABSTRACT

We propose a simple yet effective embedding model to learn quaternion embeddings for entities and relations in knowledge graphs. Our model aims to enhance correlations between head and tail entities given a relation within the Quaternion space with Hamilton product. The model achieves this goal by further associating each relation with two relation-aware rotations, which are used to rotate quaternion embeddings of the head and tail entities, respectively. Experimental results show that our proposed model produces state-of-the-art performances on well-known benchmark datasets for knowledge graph completion. Our code is available at: \url{https://github.com/daiquocnguyen/QuatRE}.

Motivation & Objective

• To address the limitation of existing hyper-complex embedding models in capturing strong correlations between head and tail entities in knowledge graphs.
• To reduce the computational complexity of translation-based models that rely on large relation-specific matrices.
• To enhance the representational power of quaternion embeddings by introducing relation-aware rotations for both head and tail entities.
• To establish a new strong baseline for knowledge graph completion using quaternion space with improved generalization across diverse relation types.

Proposed method

• QuatRE models entities and relations as normalized quaternions in the Hamilton space, using four real-valued components per embedding.
• It applies two relation-aware rotation vectors, $\boldsymbol{\mathsf{v}}_{r,1}$ and $\boldsymbol{\mathsf{v}}_{r,2}$, to rotate the head and tail entity embeddings, respectively, via the Hamilton product.
• The triple score is computed as the inner product of the rotated head embedding and the rotated tail embedding, with the relation embedding acting as a scaling factor.
• The model uses normalized quaternion vectors ($\boldsymbol{v}^{\triangleleft}$) to ensure stable optimization and improved generalization.
• It leverages the non-commutative nature of quaternion multiplication to model asymmetric relations more effectively.
• The architecture avoids parameter-heavy translation matrices by replacing them with two compact rotation vectors per relation, reducing model complexity.

Experimental results

Research questions

• RQ1 Can relation-aware quaternion rotations improve the modeling of correlations between head and tail entities in knowledge graphs?
• RQ2 Does introducing two distinct rotation vectors for head and tail entities lead to better performance than symmetric or single-rotation approaches?
• RQ3 Can a simpler rotation-based mechanism outperform complex translation-based models like TransR while maintaining lower parameter count?
• RQ4 How does QuatRE perform across different relation types (1-1, 1-M, M-1, M-M) compared to existing models?
• RQ5 To what extent do the individual components of the rotation mechanism contribute to the overall performance?

Key findings

• QuatRE achieves state-of-the-art performance on the WN18RR benchmark, with an MRR of 0.493 and Hits@10 of 59.2, surpassing QuatE (MRR: 0.488) and other baselines.
• On FB15k-237, QuatRE achieves an MRR of 0.367 and Hits@10 of 56.3, outperforming QuatE (MRR: 0.348) and other recent models.
• For the relation 'member_of_domain_region', QuatRE improves MRR from 0.193 (QuatE) to 0.364, demonstrating strong gains on complex, many-to-many relations.
• The ablation study confirms that both rotation components ($\boldsymbol{\mathsf{v}}_{r,1}$ and $\boldsymbol{\mathsf{v}}_{r,2}$) are essential, as removing either leads to performance degradation.
• Visualization on WN18RR shows that QuatRE learns more coherent and correlated entity clusters, especially for relations like 'synset_domain_topic_of' and 'instance_hypernym'.
• QuatRE maintains strong performance across all relation categories (1-1, 1-M, M-1, M-M), indicating robust generalization to diverse relational patterns.