[Paper Review] Neural Logic Reasoning

Shaoyun Shi, Hanxiong Chen+4 more|arXiv (Cornell University)|Aug 20, 2020

Advanced Graph Neural Networks32 references4 citations

TL;DR

This paper proposes Logic-Integrated Neural Network (LINN), a dynamic neural architecture that learns logical operations (AND, OR, NOT) as differentiable modules to perform propositional logic reasoning. By constructing computation graphs based on input logical expressions and using vector representations for logic variables, LINN achieves state-of-the-art performance in both logical equation solving and Top-K recommendation tasks, demonstrating strong integration of neural generalization and symbolic reasoning.

ABSTRACT

Recent years have witnessed the success of deep neural networks in many research areas. The fundamental idea behind the design of most neural networks is to learn similarity patterns from data for prediction and inference, which lacks the ability of cognitive reasoning. However, the concrete ability of reasoning is critical to many theoretical and practical problems. On the other hand, traditional symbolic reasoning methods do well in making logical inference, but they are mostly hard rule-based reasoning, which limits their generalization ability to different tasks since difference tasks may require different rules. Both reasoning and generalization ability are important for prediction tasks such as recommender systems, where reasoning provides strong connection between user history and target items for accurate prediction, and generalization helps the model to draw a robust user portrait over noisy inputs. In this paper, we propose Logic-Integrated Neural Network (LINN) to integrate the power of deep learning and logic reasoning. LINN is a dynamic neural architecture that builds the computational graph according to input logical expressions. It learns basic logical operations such as AND, OR, NOT as neural modules, and conducts propositional logical reasoning through the network for inference. Experiments on theoretical task show that LINN achieves significant performance on solving logical equations and variables. Furthermore, we test our approach on the practical task of recommendation by formulating the task into a logical inference problem. Experiments show that LINN significantly outperforms state-of-the-art recommendation models in Top-K recommendation, which verifies the potential of LINN in practice.

Motivation & Objective

• To address the limitation of deep neural networks in performing logical reasoning despite strong generalization capabilities.
• To overcome the rigidity of traditional symbolic reasoning, which relies on hard-coded rules and lacks generalization across diverse tasks.
• To unify the representational power of deep learning with the logical inference capability of symbolic systems for improved reasoning in practical applications.
• To enable neural networks to reason over user-item interactions in recommender systems using logical constraints.
• To demonstrate that logic-integrated neural networks can achieve superior performance on both theoretical logic problems and real-world recommendation tasks.

Proposed method

• LINN represents logic variables as dense vectors and models basic logical operations (AND, OR, NOT) as learnable neural modules.
• The network dynamically constructs its computation graph based on the input logical expression, preserving structural information of the logic formula.
• Logic regularization is applied to ensure that learned neural modules respect the semantics of logical operations, such as associativity and idempotency.
• The model uses a differentiable inference mechanism to compute the truth value of a logical expression as a probability, enabling end-to-end training.
• For recommendation, LINN formulates user preferences as logical expressions (e.g., 'likes A but not B → likes C') and learns from interaction sequences using logic-aware loss functions.
• Negative interactions are modeled via logic expressions rather than simple embedding negation, avoiding strong assumptions about vector symmetry.

Experimental results

Research questions

• RQ1 Can a neural network architecture be designed to perform dynamic, structure-aware logical reasoning while maintaining generalization?
• RQ2 How well can a neural model learn to solve propositional logic equations compared to standard deep learning models?
• RQ3 Can logic-integrated reasoning improve performance in real-world recommendation tasks with sparse and noisy user interactions?
• RQ4 Does modeling negative interactions through logic expressions outperform traditional embedding-based negative representation methods?
• RQ5 Can LINN discover meaningful item co-occurrence patterns without using side information like categories or content?

Key findings

• LINN significantly outperforms standard neural networks on theoretical logic reasoning tasks, such as solving logical equations and variable inference, demonstrating its strong logical reasoning capability.
• On the ML-100k and Electronics recommendation datasets, LINN achieves higher Hit@1 and Top-K performance than state-of-the-art models like GRU4Rec and NARM.
• LINN achieves a Hit@1 score of 0.58 on ML-100k, outperforming GRU4Rec and NARM, even without explicit negative interaction modeling.
• The heatmap of item co-occurrence probabilities shows that LINN learns symmetric logical relationships (e.g., v_i ∧ v_j ≈ v_j ∧ v_i) without enforcing symmetry in the architecture.
• LINN successfully identifies non-obvious co-occurrence patterns, such as a popular musical (No.602) and an unpopular drama (No.1144), which are liked by the same users despite low popularity.
• Using logic expressions to model negative interactions yields better performance than using negative embeddings or unconstrained networks, as the latter weaken semantic relationships between positive and negative representations.