[Paper Review] Differentiable Learning of Logical Rules for Knowledge Base Reasoning

Fan Yang, Zhilin Yang+1 more|arXiv (Cornell University)|Feb 27, 2017

Bayesian Modeling and Causal InferenceComputer Science32 references173 citations

TL;DR

Neural LP learns both the structure and parameters of first-order logical rules in a differentiable framework, enabling end-to-end reasoning over knowledge bases and achieving strong results on multiple benchmarks.

ABSTRACT

We study the problem of learning probabilistic first-order logical rules for knowledge base reasoning. This learning problem is difficult because it requires learning the parameters in a continuous space as well as the structure in a discrete space. We propose a framework, Neural Logic Programming, that combines the parameter and structure learning of first-order logical rules in an end-to-end differentiable model. This approach is inspired by a recently-developed differentiable logic called TensorLog, where inference tasks can be compiled into sequences of differentiable operations. We design a neural controller system that learns to compose these operations. Empirically, our method outperforms prior work on multiple knowledge base benchmark datasets, including Freebase and WikiMovies.

Motivation & Objective

• Motivate learning probabilistic first-order logical rules for KB reasoning in a differentiable setting.
• Enable simultaneous learning of rule structure and parameters within a single end-to-end model.
• Leverage TensorLog-inspired differentiable operators within a neural controller to compose rules.
• Demonstrate empirical improvements on KB completion, path-finding, and QA tasks across diverse datasets.

Proposed method

• Represent KB reasoning as weighted, probabilistic first-order rules learned by a neural controller.
• Use TensorLog operators M_R for each relation to perform differentiable inferences via matrix operations.
• Reformulate rule learning to a recurrent, attention-based memory network that softly selects sequences of operators.
• Introduce memory vectors u_t and attentions a_t (operator) and b_t (memory) to handle variable rule lengths.
• Train end-to-end with gradient-based optimization; recover human-interpretable rules by tracing attention.

Experimental results

Research questions

• RQ1 Can a fully differentiable model learn both the structure and the parameters of logical rules for KB reasoning?
• RQ2 How well does Neural LP perform on standard KB completion benchmarks and on tasks requiring longer rule chains?
• RQ3 Is it possible to handle partially structured or natural-language queries within this differentiable rule-learning framework?
• RQ4 To what extent can the learned rules be recovered and interpreted from the model’s attentions?

Key findings

• Neural LP achieves strong performance on KB completion benchmarks (WordNet and Freebase variants) and the challenging Freebase15KSelected task.
• On a synthetic grid-path task, Neural LP learns longer rules (length 6–8) and shows robustness compared to prior methods.
• In knowledge base completion, Neural LP attains state-of-the-art results on WN18 and competitive results on FB15K and FB15KSelected, demonstrating effective rule learning.
• On WikiMovies QA, Neural LP attains competitive accuracy, illustrating capability to handle questions posed in natural language.
• The model allows recovering logical rules by executing the learned controller and inspecting attention distributions, yielding interpretable rule structures.