[Paper Review] Temporal Knowledge Graph Completion with Time-sensitive Relations in Hypercomplex Space

Li Cai, Xin Mao+5 more|arXiv (Cornell University)|Mar 2, 2024

Cognitive Computing and Networks4 citations

TL;DR

This paper proposes TQuatE, a novel temporal knowledge graph completion model that leverages quaternion embeddings in hypercomplex space to capture time-sensitive relations through time-aware rotation and periodic time translation. By modeling dynamic relation patterns with superior representational capacity, TQuatE achieves state-of-the-art performance on ICEWS14, ICEWS05-15, and GDELT datasets, outperforming existing methods in handling complex temporal variability.

ABSTRACT

Temporal knowledge graph completion (TKGC) aims to fill in missing facts within a given temporal knowledge graph at a specific time. Existing methods, operating in real or complex spaces, have demonstrated promising performance in this task. This paper advances beyond conventional approaches by introducing more expressive quaternion representations for TKGC within hypercomplex space. Unlike existing quaternion-based methods, our study focuses on capturing time-sensitive relations rather than time-aware entities. Specifically, we model time-sensitive relations through time-aware rotation and periodic time translation, effectively capturing complex temporal variability. Furthermore, we theoretically demonstrate our method's capability to model symmetric, asymmetric, inverse, compositional, and evolutionary relation patterns. Comprehensive experiments on public datasets validate that our proposed approach achieves state-of-the-art performance in the field of TKGC.

Motivation & Objective

• To address the limitation of existing TKGC methods in modeling dynamic, time-varying relations beyond static or time-aware entities.
• To enhance representational capacity for temporal relations by leveraging the richer structure of hypercomplex space, specifically quaternions.
• To model complex temporal patterns such as symmetry, asymmetry, inverse, composition, and evolution in relations through geometric operations in quaternion space.
• To eliminate reliance on hyperparameter-sensitive time windows by introducing periodic time translation for more robust temporal modeling.
• To achieve state-of-the-art performance on standard TKGC benchmarks while providing theoretical justification for modeling diverse relation patterns.

Proposed method

• The model uses quaternion embeddings for entities, relations, and timestamps, with each embedding comprising one real and three imaginary components.
• Time-aware rotation is implemented via the Hamilton product of quaternions to capture local temporal interactions between entities and relations.
• Periodic time translation is introduced to model recurring temporal patterns within a time window, replacing the need for fixed, hyperparameter-dependent time windows.
• The model integrates temporal dynamics into relation representations by applying rotations and translations in the hypercomplex space, enabling dynamic relation evolution.
• Theoretical analysis confirms that the model can represent symmetric, asymmetric, inverse, compositional, and evolutionary relation patterns.
• A joint loss function with embedding and time regularizers is used to optimize the model, with hyperparameters tuned via ablation studies.

Figure 1: Subgraph example from ICEWS14 dataset for TKGC. Nodes on the graph represent entities, including Obama, Japan, South Korea, North Korea, and edges signify relations (with timestamps in parentheses), encompassing Host a visit, Make a visit, Express intent to meet or negotiate, Engage in neg

Experimental results

Research questions

• RQ1 Can quaternion-based representations in hypercomplex space better model time-sensitive relations compared to real or complex space methods?
• RQ2 How effectively can time-aware rotation and periodic time translation capture local and recurring temporal patterns in dynamic knowledge graphs?
• RQ3 Does the proposed method generalize across diverse temporal datasets with varying degrees of complexity, such as ICEWS14, ICEWS05-15, and GDELT?
• RQ4 To what extent does eliminating the need for fixed time windows improve model robustness and performance?
• RQ5 What is the theoretical capability of the model in representing fundamental relation patterns like symmetry, inverse, and composition?

Key findings

• TQuatE achieves state-of-the-art performance on ICEWS14, with an MRR of 0.639 and Hits@10 of 0.787, outperforming previous SOTA models.
• The ablation study confirms that periodic time modeling significantly improves performance, with MRR dropping to 0.629 when excluded.
• Optimal performance is achieved at an embedding dimension of 2000, with diminishing returns beyond this point.
• The best regularization weights are found to be λₑ ≈ 0.005 and λₜ ≈ 0.1, indicating stronger impact of temporal regularization.
• Despite higher computational cost (2011s vs. 1487s for TeAST), TQuatE achieves a marginal MRR improvement (0.639 vs. 0.637), validating its effectiveness.
• TQuatE achieves comparable parameter count to TeAST but outperforms it due to the enhanced representational power of quaternions in modeling temporal dynamics.

Figure 2: TKGC performance for different embedding dimensions $d$ on ICEWS14.