[Paper Review] xERTE: Explainable Reasoning on Temporal Knowledge Graphs for Forecasting Future Links

Zhen Han, Peng Chen+2 more|arXiv (Cornell University)|Dec 31, 2020

Advanced Graph Neural NetworksComputer Science44 references18 citations

TL;DR

xERTE is a novel explainable reasoning framework for forecasting future links in temporal knowledge graphs by iteratively expanding query-relevant subgraphs using a temporal relational graph attention mechanism and a reverse representation update scheme. It achieves state-of-the-art performance with up to 20% relative improvement in Hits@1 while providing human-interpretable evidence through structured subgraph reasoning and time-aware attention.

ABSTRACT

Modeling time-evolving knowledge graphs (KGs) has recently gained increasing interest. Here, graph representation learning has become the dominant paradigm for link prediction on temporal KGs. However, the embedding-based approaches largely operate in a black-box fashion, lacking the ability to interpret their predictions. This paper provides a link forecasting framework that reasons over query-relevant subgraphs of temporal KGs and jointly models the structural dependencies and the temporal dynamics. Especially, we propose a temporal relational attention mechanism and a novel reverse representation update scheme to guide the extraction of an enclosing subgraph around the query. The subgraph is expanded by an iterative sampling of temporal neighbors and by attention propagation. Our approach provides human-understandable evidence explaining the forecast. We evaluate our model on four benchmark temporal knowledge graphs for the link forecasting task. While being more explainable, our model obtains a relative improvement of up to 20% on Hits@1 compared to the previous best KG forecasting method. We also conduct a survey with 53 respondents, and the results show that the evidence extracted by the model for link forecasting is aligned with human understanding.

Motivation & Objective

• Address the lack of interpretability in black-box embedding-based models for temporal knowledge graph link forecasting.
• Enable transparent, human-understandable reasoning by extracting compact, evidence-rich subgraphs around queries.
• Model both structural dependencies and temporal dynamics in knowledge graphs to improve forecasting accuracy.
• Mimic human reasoning by enabling entities in the subgraph to receive messages from newly added prior neighbors.
• Develop a framework that supports causal, time-constrained reasoning while maintaining high predictive performance.

Proposed method

• Propose a temporal relational graph attention (TRGA) mechanism that aggregates messages only from prior temporal neighbors (i.e., those linked before the query timestamp), preserving causality.
• Introduce a reverse representation update scheme that propagates messages from the farthest prior neighbors to all entities in the subgraph, enabling dynamic belief updating.
• Use a hybrid entity embedding module combining stationary entity embeddings and functional time encoding to model both structural and temporal dynamics.
• Iteratively expand the subgraph by sampling relevant edges and pruning low-attention paths, guided by attention scores from the TRGA mechanism.
• Construct a reasoning path by propagating attention through the subgraph to predict the missing object in a query (subject, predicate, ?, timestamp).
• Ensure temporal consistency by enforcing that messages are only passed from earlier to later timestamps, maintaining causal order in reasoning.

Experimental results

Research questions

• RQ1 Can a subgraph-based reasoning framework achieve high predictive performance while providing human-interpretable explanations for link forecasting in temporal knowledge graphs?
• RQ2 How does incorporating time-aware attention mechanisms improve the model's ability to capture temporal dynamics and causal dependencies?
• RQ3 To what extent does the reverse representation update mechanism enhance reasoning by enabling entities to incorporate new evidence from distant prior neighbors?
• RQ4 How well do the model-generated explanations align with human intuition and understanding of temporal reasoning?
• RQ5 What is the impact of time encoding on attention scores and the interpretability of the reasoning process?

Key findings

• xERTE achieves a relative improvement of up to 20% in Hits@1 compared to the previous state-of-the-art method for temporal knowledge graph link forecasting.
• The model's subgraph-based explanations are highly aligned with human understanding, as confirmed by a survey of 53 respondents who evaluated the relevance and interpretability of the evidence.
• Time-aware attention scores are significantly more interpretable and temporally coherent when time encoding is used, as recent events receive higher attention scores.
• Without time encoding, attention scores for the same triple at different timestamps become random and inconsistent, undermining interpretability.
• The reverse representation update mechanism enables more effective propagation of evidence from newly added prior neighbors, improving reasoning completeness.
• The ablation study confirms that time encoding is crucial for modeling temporal dynamics, as its removal leads to non-intuitive and unstable attention patterns.