[Paper Review] Neural Logic Machines

Honghua Dong, Jiayuan Mao+4 more|arXiv (Cornell University)|Apr 26, 2019

Natural Language Processing Techniques46 citations

TL;DR

Neural Logic Machines combine neural networks with symbolic logic to learn lifted rules and generalize to larger instances, achieving strong generalization on relational reasoning, decision making, and planning tasks.

ABSTRACT

We propose the Neural Logic Machine (NLM), a neural-symbolic architecture for both inductive learning and logic reasoning. NLMs exploit the power of both neural networks---as function approximators, and logic programming---as a symbolic processor for objects with properties, relations, logic connectives, and quantifiers. After being trained on small-scale tasks (such as sorting short arrays), NLMs can recover lifted rules, and generalize to large-scale tasks (such as sorting longer arrays). In our experiments, NLMs achieve perfect generalization in a number of tasks, from relational reasoning tasks on the family tree and general graphs, to decision making tasks including sorting arrays, finding shortest paths, and playing the blocks world. Most of these tasks are hard to accomplish for neural networks or inductive logic programming alone.

Motivation & Objective

• Motivate a neural-symbolic system capable of learning lifted first-order logic rules.
• Address high-order relational data and quantifiers beyond standard graph networks.
• Achieve scalable rule induction without hand-crafted templates.
• Demonstrate generalization from small training instances to larger test instances.

Proposed method

• Represent logic predicates as probabilistic tensors grounded on a fixed object set.
• Implement logic rules as neural operators that perform lifted boolean logic and quantification.
• Use a multi-layer, multi-group architecture with inter- and intra-group computations to realize Horn-clause-like reasoning.
• Employ expansion and reduction meta-rules to implement universal and existential quantification.
• Control model capacity via depth D, breadth B, and a small number of output predicates per layer to manage complexity.
• Provide training as supervised learning or reinforcement learning depending on task.

Experimental results

Research questions

• RQ1 Can NLMs induce lifted rules that generalize to larger object sets than seen during training?
• RQ2 How can NLMs handle high-order relations and quantifiers within a neural framework?
• RQ3 What is the scalability of NLMs with respect to rule complexity and arity?
• RQ4 Are NLMs capable of solving relational reasoning, graph tasks, and planning problems more effectively than purely neural or purely symbolic baselines?

Key findings

• NLMs generalize from small-scale training to large-scale tasks in relational reasoning and decision making.
• NLMs achieve strong performance across blocks world, sorting, and shortest path tasks, often outperforming baselines.
• NLMs can recover lifted rules and perform forward chaining over a partial set of Horn clauses.
• The architecture supports multi-arity predicates and quantifiers, enabling reasoning over high-order relational data.
• Experiments show solid generalization across family trees, general graphs, and algorithmic tasks.