[Paper Review] Logic and Commonsense-Guided Temporal Knowledge Graph Completion

Guanglin Niu, Bo Li|arXiv (Cornell University)|Nov 30, 2022

Topic Modeling4 citations

TL;DR

This paper proposes LCGE, a novel temporal knowledge graph completion model that jointly learns time-sensitive representations for timeliness and causality, along with time-independent commonsense knowledge. By integrating temporal rule-guided predicate regularization and commonsense-aware embeddings, LCGE achieves state-of-the-art performance on three benchmark datasets and provides explainable predictions through causal inference via symbolic rules.

ABSTRACT

A temporal knowledge graph (TKG) stores the events derived from the data involving time. Predicting events is extremely challenging due to the time-sensitive property of events. Besides, the previous TKG completion (TKGC) approaches cannot represent both the timeliness and the causality properties of events, simultaneously. To address these challenges, we propose a Logic and Commonsense-Guided Embedding model (LCGE) to jointly learn the time-sensitive representation involving timeliness and causality of events, together with the time-independent representation of events from the perspective of commonsense. Specifically, we design a temporal rule learning algorithm to construct a rule-guided predicate embedding regularization strategy for learning the causality among events. Furthermore, we could accurately evaluate the plausibility of events via auxiliary commonsense knowledge. The experimental results of TKGC task illustrate the significant performance improvements of our model compared with the existing approaches. More interestingly, our model is able to provide the explainability of the predicted results in the view of causal inference. The source code and datasets of this paper are available at https://github.com/ngl567/LCGE.

Motivation & Objective

• To address the limitation of existing TKGC models in jointly representing timeliness and causality in temporal events.
• To enhance prediction accuracy by incorporating both time-sensitive event dynamics and time-independent commonsense knowledge.
• To improve model explainability by enabling causal inference through mined temporal rules.
• To develop a unified framework that integrates logic-based reasoning with neural embedding learning for temporal knowledge graphs.

Proposed method

• Proposes a temporal rule-guided predicate embedding regularization strategy to inject causality into predicate representations using mined temporal rules.
• Designs a joint scoring mechanism that combines time-sensitive embeddings (for timeliness and causality) and time-independent commonsense embeddings.
• Introduces a temporal rule learning algorithm to automatically extract diverse temporal rule patterns from TKGs, beyond single-time constraints.
• Employs a symbolic reasoning module that matches query events to high-quality temporal rules for explainable inference.
• Uses a dual-branch architecture: one for time-aware event scoring and another for commonsense-based plausibility assessment.
• Applies a differentiable loss function that optimizes both TKGC performance and rule consistency during training.

Experimental results

Research questions

• RQ1 Can a unified model effectively represent both timeliness and causality in temporal knowledge graphs?
• RQ2 How can commonsense knowledge improve the plausibility scoring of missing temporal events?
• RQ3 Can temporal rules be automatically mined and used to enhance explainability in TKGC?
• RQ4 Does combining logic-guided reasoning with neural embeddings lead to significant performance gains over existing data-driven TKGE models?

Key findings

• LCGE achieves significant performance improvements over state-of-the-art baselines on three benchmark datasets: ICEWS14, ICEWS05-15, and Wikidata12k.
• On ICEWS14, LCGE improves MRR by 6.7% and Hits@1 by 7.9% relative to the best baseline TeLM.
• On ICEWS05-15, LCGE achieves 7.7% relative gain in MRR and 9.3% in Hits@1 compared to TeLM.
• On Wikidata12k, LCGE records a 29.2% relative improvement in MRR and 31.6% in Hits@1, indicating strong generalization.
• The ablation study confirms that both the temporal rule-guided regularization and commonsense module are essential, with the latter contributing more to performance gain.
• A case study demonstrates that LCGE provides explainable predictions by matching queries to symbolic temporal rules, such as deducing 'South Korea' as the object of a visit event based on a rule and a grounded event.