[Paper Review] Instance-based Learning for Knowledge Base Completion

Wanyun Cui, Xingran Chen|arXiv (Cornell University)|Nov 13, 2022

Semantic Web and Ontologies4 citations

TL;DR

This paper introduces instance-based learning (IBL) for knowledge base completion, where predictions are made by identifying prototype entities with shared relational properties—e.g., inferring Jill Biden's lived city via Joe Biden as a prototype. IBL achieves state-of-the-art performance and reveals that rule-based models rely critically on IBL rules (representing instance-based equivalence), challenging the long-held belief that rule-based reasoning depends on semantic compatibility.

ABSTRACT

In this paper, we propose a new method for knowledge base completion (KBC): instance-based learning (IBL). For example, to answer (Jill Biden, lived city,? ), instead of going directly to Washington D.C., our goal is to find Joe Biden, who has the same lived city as Jill Biden. Through prototype entities, IBL provides interpretability. We develop theories for modeling prototypes and combining IBL with translational models. Experiments on various tasks confirmed the IBL model's effectiveness and interpretability. In addition, IBL shed light on the mechanism of rule-based KBC models. Previous research has generally agreed that rule-based models provide rules with semantically compatible premises and hypotheses. We challenge this view. We begin by demonstrating that some logical rules represent {\it instance-based equivalence} (i.e. prototypes) rather than semantic compatibility. These are denoted as {\it IBL rules}. Surprisingly, despite occupying only a small portion of the rule space, IBL rules outperform non-IBL rules in all four benchmarks. We use a variety of experiments to demonstrate that rule-based models work because they have the ability to represent instance-based equivalence via IBL rules. The findings provide new insights of how rule-based models work and how to interpret their rules.

Motivation & Objective

• To develop a new, interpretable method for knowledge base completion (KBC) that avoids the limitations of traditional embedding and rule-based models.
• To formalize instance-based learning (IBL) in the context of KBC using prototypes derived from translational models.
• To investigate the role of IBL in rule-based KBC and challenge the assumption that rule-based models rely on semantic compatibility.
• To demonstrate that IBL rules—representing instance-based equivalence—outperform non-IBL rules despite occupying a small portion of the rule space.
• To establish a joint optimization framework between IBL and translational models for improved KBC performance.

Proposed method

• The IBL model identifies a prototype entity that shares the same relational value with the query head entity, e.g., Joe Biden as a prototype for Jill Biden in the 'lived city' relation.
• Prototypes are derived using closed-form expressions based on translational models, enabling efficient and interpretable inference.
• A joint optimization theory is developed to combine IBL with translational models, aligning their optimization objectives for improved performance.
• The method evaluates rule-based models by ablation, isolating IBL rules from non-IBL rules to assess their individual contributions.
• Rule collection is modified in RNNLogic to control for IBL vs. non-IBL rules, enabling direct comparison of rule quality and performance.
• Theoretical analysis shows that the low-rank assumption in knowledge graphs supports the existence of meaningful prototypes, validating the IBL approach.

Experimental results

Research questions

• RQ1 Can instance-based learning (IBL) provide an effective and interpretable alternative to traditional knowledge base completion methods?
• RQ2 How can prototypes in IBL be formally expressed using translational models, and what is their closed-form solution?
• RQ3 What is the role of IBL rules in rule-based KBC, and do they outperform rules based on semantic compatibility?
• RQ4 Why do IBL rules dominate performance in rule-based models despite their small representation in the rule space?
• RQ5 Can IBL be combined with translational models to improve KBC performance, and what is the theoretical basis for such combination?

Key findings

• IBL rules, which represent instance-based equivalence, outperform non-IBL rules in all four benchmarks, with hit@10 scores of 49.9 on FB15k-237 and 49.9 on WN18RR when used exclusively.
• In Kinship and UMLS, using only IBL rules results in no significant performance degradation compared to using all rules, achieving 96.6% H@10 on Kinship and 94.7% H@10 on UMLS.
• Despite occupying only a small fraction of the rule space, IBL rules have substantially higher average precision (3.7% vs. 1.7%) and support (708.3 vs. 281.4) than non-IBL rules.
• The combined IBL-translational model outperforms existing methods on two out of four benchmarks, demonstrating the effectiveness of joint optimization.
• CIBLE (the proposed IBL model) outperforms RNNLogic in three of four benchmarks, showing that IBL is a more effective alternative to rule-based models for modeling instance-based equivalence.
• The study challenges the prevailing view that rule-based models work via semantic compatibility, instead showing they function primarily through the representation of instance-based equivalence via IBL rules.