[Paper Review] What Can Neural Networks Reason About?

Keyulu Xu, Jingling Li+4 more|arXiv (Cornell University)|May 30, 2019

Multimodal Machine Learning ApplicationsComputer Science64 references58 citations

TL;DR

The paper introduces the algorithmic alignment framework to predict when neural network architectures generalize well on reasoning tasks, showing GNNs align with dynamic programming and NES can tackle NP-hard subset sum.

ABSTRACT

Neural networks have succeeded in many reasoning tasks. Empirically, these tasks require specialized network structures, e.g., Graph Neural Networks (GNNs) perform well on many such tasks, but less structured networks fail. Theoretically, there is limited understanding of why and when a network structure generalizes better than others, although they have equal expressive power. In this paper, we develop a framework to characterize which reasoning tasks a network can learn well, by studying how well its computation structure aligns with the algorithmic structure of the relevant reasoning process. We formally define this algorithmic alignment and derive a sample complexity bound that decreases with better alignment. This framework offers an explanation for the empirical success of popular reasoning models, and suggests their limitations. As an example, we unify seemingly different reasoning tasks, such as intuitive physics, visual question answering, and shortest paths, via the lens of a powerful algorithmic paradigm, dynamic programming (DP). We show that GNNs align with DP and thus are expected to solve these tasks. On several reasoning tasks, our theory is supported by empirical results.

Motivation & Objective

• Motivate and formalize the question of which reasoning tasks neural networks can learn efficiently.
• Introduce algorithmic alignment to connect network structure with the algorithmic structure of reasoning tasks.
• Provide theoretical and empirical evidence that alignment impacts sample complexity and generalization.
• Unify and analyze reasoning tasks (summary statistics, relational argmax, dynamic programming, NP-hard problems) under a common framework.

Proposed method

• Define a numeric measure of algorithmic alignment between network modules and reasoning algorithms.
• Develop a PAC-learning based analysis showing sample complexity decreases with better alignment (Theorem 3.6).
• Provide bounds on module-level sample complexity for overparameterized MLPs (Theorem 3.5).
• Show how DP naturally unifies tasks like intuitive physics, VQA, and shortest paths under GNNs.
• Design a neural network (NES) for subset sum based on exhaustive search to illustrate limitations of DP-aligned architectures.

Experimental results

Research questions

• RQ1 What tasks can a neural network learn to reason about efficiently, given its architecture?
• RQ2 How does the alignment between a network's computation structure and a reasoning algorithm affect sample complexity and generalization?
• RQ3 Can common architectures (MLP, Deep Sets, GNNs) learn categories of reasoning tasks, and where do they fail?
• RQ4 Can we design architectures aligned with non-DP reasoning (e.g., NP-hard problems) to achieve better generalization?

Key findings

• GNNs align with dynamic programming and generalize well to DP-like tasks such as shortest paths and DP-style visual reasoning.
• Deep Sets outperform MLP on permutation-invariant summary statistics tasks due to better alignment.
• One-iteration GNNs can handle relational argmax, while Deep Sets struggle due to needing loop-like computations.
• GNNs fail on NP-hard subset sum, but the proposed Neural Exhaustive Search (NES) architecture generalizes, illustrating alignment limits.
• Empirical results across tasks confirm that better algorithmic alignment correlates with improved sample efficiency and generalization.