[Paper Review] T-GAP: Learning to Walk across Time for Temporal Knowledge Graph Completion

Jaehun Jung, Jinhong Jung+1 more|arXiv (Cornell University)|Dec 19, 2020

Advanced Graph Neural Networks27 references10 citations

TL;DR

T-GAP proposes a novel temporal knowledge graph completion model that leverages query-specific temporal displacement and attention-based path propagation to improve reasoning over time-aware knowledge graphs. It achieves state-of-the-art performance and superior interpretability by dynamically focusing on relevant past and future events relative to the query timestamp.

ABSTRACT

Temporal knowledge graphs (TKGs) inherently reflect the transient nature of real-world knowledge, as opposed to static knowledge graphs. Naturally, automatic TKG completion has drawn much research interests for a more realistic modeling of relational reasoning. However, most of the existing mod-els for TKG completion extend static KG embeddings that donot fully exploit TKG structure, thus lacking in 1) account-ing for temporally relevant events already residing in the lo-cal neighborhood of a query, and 2) path-based inference that facilitates multi-hop reasoning and better interpretability. In this paper, we propose T-GAP, a novel model for TKG completion that maximally utilizes both temporal information and graph structure in its encoder and decoder. T-GAP encodes query-specific substructure of TKG by focusing on the temporal displacement between each event and the query times-tamp, and performs path-based inference by propagating attention through the graph. Our empirical experiments demonstrate that T-GAP not only achieves superior performance against state-of-the-art baselines, but also competently generalizes to queries with unseen timestamps. Through extensive qualitative analyses, we also show that T-GAP enjoys from transparent interpretability, and follows human intuition in its reasoning process.

Motivation & Objective

• Address the limitations of existing static knowledge graph embedding models in capturing temporal dynamics and structural context in temporal knowledge graphs (TKGs).
• Improve TKG completion by encoding query-relevant substructures that account for temporal displacement between events and the query timestamp.
• Enable path-based reasoning in TKGs through soft, attention-driven propagation over graph structures, enhancing interpretability and multi-hop inference.
• Ensure generalization to unseen query timestamps by modeling relative temporal positions rather than absolute timestamps.
• Provide transparent, human-intuitive reasoning by analyzing attention distributions over time-aware paths.

Proposed method

• Introduce a temporal graph neural network (GNN) encoder that separately encodes the sign (past/present/future) and magnitude of temporal displacement between each edge and the query timestamp.
• Use query-specific attention to aggregate neighborhood information, focusing on events temporally relevant to the query time.
• Implement a soft path-based inference mechanism via attention flow propagation, where attention is diffused from nodes to neighbors rather than selecting a single path.
• Train the model end-to-end using supervised learning, enabling joint optimization of encoding and decoding components.
• Model temporal reasoning by propagating attention through edges, allowing the model to correct suboptimal initial choices through multi-step reasoning.
• Encode relational semantics using temporal predicates (e.g., occurSince, occurUntil) to guide attention toward temporally coherent and contextually relevant facts.

Experimental results

Research questions

• RQ1 Can modeling temporal displacement (relative time between events and query) improve the performance of TKG completion models?
• RQ2 Does attention-based path propagation enable more effective and interpretable multi-hop reasoning in temporal knowledge graphs compared to static embedding models?
• RQ3 Can a TKG model generalize to queries with unseen timestamps by relying on relative temporal positioning?
• RQ4 To what extent does the attention distribution in T-GAP reflect human-like reasoning patterns in temporal relational inference?
• RQ5 How does T-GAP’s attention mechanism correct for initial misjudgments in path selection during reasoning?

Key findings

• T-GAP achieves state-of-the-art performance across all three benchmark datasets (ICEWS14, ICEWS15, YAGO15K) in temporal knowledge graph completion, outperforming existing SOTA models.
• The model generalizes effectively to queries with unseen timestamps, demonstrating robustness to distributional shifts in temporal queries.
• Qualitative analysis shows that T-GAP’s attention distribution is highly interpretable, with attention focusing on temporally relevant events such as past threats or future affiliations.
• In case studies, T-GAP corrects initial misjudgments—e.g., shifting attention from Japan to South_Korea in a geopolitical threat query—by leveraging multi-step attention propagation.
• For relations like member_of_sports_team-occurUntil, T-GAP prioritizes past events, aligning with human intuition to find the last known affiliation.
• For relations like educated_at-occurSince, T-GAP focuses on future events, correctly identifying the first institution after the query time, demonstrating temporal logic alignment.