[Paper Review] Neural Networks and the Chomsky Hierarchy

Grégoire Delétang, Anian Ruoss+9 more|arXiv (Cornell University)|Jul 5, 2022

Ferroelectric and Negative Capacitance DevicesEngineering45 citations

TL;DR

An extensive empirical study that places neural architectures (RNN, LSTM, Transformer, and memory-augmented variants) on the Chomsky hierarchy by testing their generalization on formal-language transduction tasks, showing memory-augmented models generalize higher, while plain Transformers/LSTMs have limitations.

ABSTRACT

Reliable generalization lies at the heart of safe ML and AI. However, understanding when and how neural networks generalize remains one of the most important unsolved problems in the field. In this work, we conduct an extensive empirical study (20'910 models, 15 tasks) to investigate whether insights from the theory of computation can predict the limits of neural network generalization in practice. We demonstrate that grouping tasks according to the Chomsky hierarchy allows us to forecast whether certain architectures will be able to generalize to out-of-distribution inputs. This includes negative results where even extensive amounts of data and training time never lead to any non-trivial generalization, despite models having sufficient capacity to fit the training data perfectly. Our results show that, for our subset of tasks, RNNs and Transformers fail to generalize on non-regular tasks, LSTMs can solve regular and counter-language tasks, and only networks augmented with structured memory (such as a stack or memory tape) can successfully generalize on context-free and context-sensitive tasks.

Motivation & Objective

• Assess how standard and memory-augmented neural networks generalize on sequence transduction tasks across the Chomsky hierarchy.
• Determine which architectures can learn algorithms required for regular, context-free, context-sensitive, and RE-style tasks under gradient-based training.
• Identify failure modes and propose memory augmentation as a path to higher-level generalization.

Proposed method

• Define input/output languages and construct a suite of sequence transduction tasks spanning the Chomsky hierarchy.
• Train a wide range of models (RNN, LSTM, Transformer, Stack-RNN, Tape-RNN) with gradient-based learning to map inputs to outputs.
• Use non-autoregressive setup with dummy tokens to signal input termination and compute cross-entropy loss across output sequences.
• Evaluate generalization by testing on out-of-distribution longer sequences (N+1 to M), reporting accuracy per task.
• Analyze internal representations and memory dynamics to interpret learned algorithms (e.g., stack/tape usage).
• Open-source the benchmark, models, and tasks for reproducibility.

Experimental results

Research questions

• RQ1 Can standard and memory-augmented neural networks generalize to longer sequences beyond training length for tasks at each level of the Chomsky hierarchy?
• RQ2 Which architectures (RNN, LSTM, Transformer, Stack-RNN, Tape-RNN) are capable of solving tasks corresponding to regular, context-free, context-sensitive, and recursively enumerable languages under gradient-based training?
• RQ3 Do memory augmentations (stack, tape) enable generalization beyond regular languages, and if so, to what extent?
• RQ4 How well do Transformers with various positional encodings cope with non-permutation-invariant tasks and extrapolate to longer sequences?

Key findings

• RNNs generalize up to regular languages; Stack-RNNs extend to context-free; Tape-RNNs reach context-sensitive levels; some tasks remain challenging even with memory augmentation.
• Transformers excel on permutation-invariant tasks (e.g., Bucket Sort) but fail on many non-permutation-invariant tasks across levels of the hierarchy, and can struggle with long-sequence extrapolation due to positional encodings.
• LSTMs outperform vanilla RNNs on counting-related tasks, enabling some higher-level generalization (e.g., Bucket Sort) that surpasses plain RNNs.
• Memory-augmented models reveal interpretable strategies: Stack-RNN learns stack-like solutions for DCF tasks; Tape-RNN shows tape-based manipulation for CS tasks.
• Even with capacity, increasing data alone does not guarantee generalization to higher-hierarchy tasks; structured memories appear necessary for higher-level generalization.
• The study provides a public benchmark to stress-test sequence generalization and highlights fundamental limits of gradient-based learning on algorithmic tasks.