[Paper Review] Joint Semantics and Data-Driven Path Representation for Knowledge Graph Inference

Guanglin Niu, Bo Li+5 more|arXiv (Cornell University)|Oct 6, 2020

Advanced Graph Neural Networks34 references4 citations

TL;DR

This paper proposes a joint semantics and data-driven path representation model for knowledge graph inference that combines explainable Horn rule-based path composition with an entity converter to reduce heterogeneity between entities and relations. By integrating semantic path condensation and attentive representation learning, the model achieves state-of-the-art performance on link prediction and path query answering, with Hits@1 of 0.987 on NELL-995.

ABSTRACT

Inference on a large-scale knowledge graph (KG) is of great importance for KG applications like question answering. The path-based reasoning models can leverage much information over paths other than pure triples in the KG, which face several challenges: all the existing path-based methods are data-driven, lacking explainability for path representation. Besides, some methods either consider only relational paths or ignore the heterogeneity between entities and relations both contained in paths, which cannot capture the rich semantics of paths well. To address the above challenges, in this work, we propose a novel joint semantics and data-driven path representation that balances explainability and generalization in the framework of KG embedding. More specifically, we inject horn rules to obtain the condensed paths by the transparent and explainable path composition procedure. The entity converter is designed to transform the entities along paths into the representations in the semantic level similar to relations for reducing the heterogeneity between entities and relations, in which the KGs both with and without type information are considered. Our proposed model is evaluated on two classes of tasks: link prediction and path query answering task. The experimental results show that it has a significant performance gain over several different state-of-the-art baselines.

Motivation & Objective

• Address the lack of explainability in data-driven path-based knowledge graph reasoning models.
• Overcome the limitation of existing methods that treat entities and relations asymmetrically in path representations.
• Improve path representation by jointly modeling semantics and data-driven patterns in knowledge graphs.
• Enable effective path query answering with human-interpretable reasoning via Horn rules.
• Support knowledge graphs both with and without entity type information through a unified entity conversion strategy.

Proposed method

• Use Horn rules to compose paths into condensed, explainable relations through transparent path composition.
• Design an entity converter that maps entities along paths into semantic representations similar to relations, reducing heterogeneity.
• Implement an attentive entity-to-type conversion module for KGs with type information and a projection module for those without.
• Employ a bidirectional path encoder to learn path representations from sequences of entities and relations.
• Fusion learning of multiple path representations between entity pairs to enhance embedding quality.
• Train the model end-to-end using a margin-based loss function that incorporates both translational distance and semantic similarity.

Experimental results

Research questions

• RQ1 Can a joint semantics and data-driven path representation improve explainability and performance in knowledge graph inference?
• RQ2 How does incorporating both entities and relations along paths affect path representation quality compared to relation-only models?
• RQ3 To what extent can Horn rules enable transparent, rule-based path composition for accurate inference?
• RQ4 How effective is the proposed entity converter in reducing heterogeneity between entities and relations in path representations?
• RQ5 Can the model generalize across KGs with and without type annotations for entity representation?

Key findings

• The model achieves a Hits@1 score of 0.987 on the NELL-995 dataset for path query answering, indicating 98.7% accuracy in predicting the correct relation.
• The model consistently outperforms SOTA baselines, including PTransE and TransE, on both link prediction and path query answering tasks.
• Horn rule-based path composition enables human-interpretable reasoning, with one example showing a correct relation inferred via a rule with confidence 1.
• The entity converter strategy effectively reduces heterogeneity between entities and relations, improving path representation learning.
• Fusion of multiple path representations between entity pairs leads to more robust and accurate embeddings.
• The model demonstrates strong generalization, performing well on KGs both with and without entity type information.