[Paper Review] Logical Neural Networks

Ryan Riegel, Alexander Gray+13 more|arXiv (Cornell University)|Jun 23, 2020

Bayesian Modeling and Causal InferenceComputer Science20 references81 citations

TL;DR

Logical Neural Networks (LNNs) integrate neural learning with symbolic logic by mapping neurons to elements of weighted real-valued logic, enabling end-to-end differentiable, bidirectional, and interpretable reasoning with open-world bounds.

ABSTRACT

We propose a novel framework seamlessly providing key properties of both neural nets (learning) and symbolic logic (knowledge and reasoning). Every neuron has a meaning as a component of a formula in a weighted real-valued logic, yielding a highly intepretable disentangled representation. Inference is omnidirectional rather than focused on predefined target variables, and corresponds to logical reasoning, including classical first-order logic theorem proving as a special case. The model is end-to-end differentiable, and learning minimizes a novel loss function capturing logical contradiction, yielding resilience to inconsistent knowledge. It also enables the open-world assumption by maintaining bounds on truth values which can have probabilistic semantics, yielding resilience to incomplete knowledge.

Motivation & Objective

• Introduce a neuro-symbolic framework that unifies neural learning with symbolic logic for interpretable knowledge representation.
• Build a 1-to-1 correspondence between neurons and logical formula elements to enable bidirectional, provable reasoning.
• Develop a differentiable learning objective that penalizes logical contradictions and supports open-world assumptions via truth-value bounds.
• Demonstrate inference efficiency with guaranteed convergence and applicability to first-order logic and theorem proving.

Proposed method

• Represent logical formulae as a graph where each neuron corresponds to a logical atom or connective with outputs in [0,1] as truth-value bounds.
• Use activation functions constrained to match logical connectives (AND, OR, NOT, IMPLIES) and add support for first-order quantifiers via min/max aggregations over grounding tuples.
• Introduce weighted nonlinear real-valued logics with parameters (weights, bias, function f) to realize conjunctions, disjunctions, and implications as differentiable operators.
• Employ an upward–downward inference algorithm that iteratively tightens bounds across subformulae until finite-step convergence.
• Allow bidirectional inference so that bounds on outputs constrain inputs, enabling rules like modus ponens and modus tollens within the LNN framework.
• Learn neural parameters (weights, bounds, and possibly atom confidences) via backpropagation with loss functions that penalize contradictions and encourage classical behavior on classical inputs.

Experimental results

Research questions

• RQ1 Can a neural network be structured so that each neuron corresponds to a component of a logical formula, enabling transparent truth-value bounds?
• RQ2 Does bidirectional inference and bound tightening yield reliable, convergent reasoning akin to theorem proving within a differentiable model?
• RQ3 How can open-world assumptions be represented in LNNs through truth-value bounds rather than binary true/false labels?
• RQ4 Do LNNs support general first-order logic and perform well on standard theorem-proving benchmarks compared to symbolic systems?
• RQ5 Can LNNs handle inconsistencies and noisy knowledge by adjusting bounds rather than discarding information?

Key findings

• LNNs provide a differentiable, end-to-end framework with a 1-to-1 neuron–formula correspondence enabling interpretable representations.
• Inference is bidirectional and converges in finite steps, enabling theorem-prover-like reasoning within a neural architecture.
• Truth values are represented as bounds, supporting open-world semantics and probabilistic interpretations while maintaining interpretability.
• Experiments show LNNs achieving complete recall on LUBM queries alongside exact symbolic systems in some settings, and they can locate and down-weight inconsistent axioms.
• On TPTP and related benchmarks, LNNs demonstrate competitive or superior performance to several symbolic/noisy-rule baselines and prove problems within seconds on a subset of ATP tasks.
• LNNs can explicitly penalize contradictions during learning, enabling learning that respects logical structure and improves explainability.