[논문 리뷰] Differentiable Learning of Logical Rules for Knowledge Base Reasoning

Fan Yang, Zhilin Yang외 1명|arXiv (Cornell University)|2017. 02. 27.

Bayesian Modeling and Causal Inference참고 문헌 32인용 수 173

한 줄 요약

Neural LP는 지식 베이스에 대한 엔드-투-엔드 추론을 가능하게 하는 differentiable 프레임워크에서 1차 논리 규칙의 구조와 매개변수를 모두 학습합니다.

ABSTRACT

We study the problem of learning probabilistic first-order logical rules for knowledge base reasoning. This learning problem is difficult because it requires learning the parameters in a continuous space as well as the structure in a discrete space. We propose a framework, Neural Logic Programming, that combines the parameter and structure learning of first-order logical rules in an end-to-end differentiable model. This approach is inspired by a recently-developed differentiable logic called TensorLog, where inference tasks can be compiled into sequences of differentiable operations. We design a neural controller system that learns to compose these operations. Empirically, our method outperforms prior work on multiple knowledge base benchmark datasets, including Freebase and WikiMovies.

연구 동기 및 목표

• Motivate learning probabilistic first-order logical rules for KB reasoning in a differentiable setting.
• Enable simultaneous learning of rule structure and parameters within a single end-to-end model.
• Leverage TensorLog-inspired differentiable operators within a neural controller to compose rules.
• Demonstrate empirical improvements on KB completion, path-finding, and QA tasks across diverse datasets.

제안 방법

• Represent KB reasoning as weighted, probabilistic first-order rules learned by a neural controller.
• Use TensorLog operators M_R for each relation to perform differentiable inferences via matrix operations.
• Reformulate rule learning to a recurrent, attention-based memory network that softly selects sequences of operators.
• Introduce memory vectors u_t and attentions a_t (operator) and b_t (memory) to handle variable rule lengths.
• Train end-to-end with gradient-based optimization; recover human-interpretable rules by tracing attention.

실험 결과

연구 질문

• RQ1 Can a fully differentiable model learn both the structure and the parameters of logical rules for KB reasoning?
• RQ2 How well does Neural LP perform on standard KB completion benchmarks and on tasks requiring longer rule chains?
• RQ3 Is it possible to handle partially structured or natural-language queries within this differentiable rule-learning framework?
• RQ4 To what extent can the learned rules be recovered and interpreted from the model’s attentions?

주요 결과

• Neural LP achieves strong performance on KB completion benchmarks (WordNet and Freebase variants) and the challenging Freebase15KSelected task.
• On a synthetic grid-path task, Neural LP learns longer rules (length 6–8) and shows robustness compared to prior methods.
• In knowledge base completion, Neural LP attains state-of-the-art results on WN18 and competitive results on FB15K and FB15KSelected, demonstrating effective rule learning.
• On WikiMovies QA, Neural LP attains competitive accuracy, illustrating capability to handle questions posed in natural language.
• The model allows recovering logical rules by executing the learned controller and inspecting attention distributions, yielding interpretable rule structures.