[Paper Review] Limitations of Language Models in Arithmetic and Symbolic Induction

Qian Jing, Hong Wang+3 more|arXiv (Cornell University)|Aug 9, 2022

Topic Modeling4 citations

TL;DR

This paper investigates the limitations of large language models (LLMs) in symbolic manipulation tasks such as arithmetic addition, copying, and reversing sequences, particularly under out-of-distribution (OOD) settings and with high repetition. It proposes a novel method, LMs with tutor, which teaches models step-by-step using explicit reasoning demonstrations, achieving 100% accuracy on OOD and repetitive inputs where standard LLMs fail.

ABSTRACT

Recent work has shown that large pretrained Language Models (LMs) can not only perform remarkably well on a range of Natural Language Processing (NLP) tasks but also start improving on reasoning tasks such as arithmetic induction, symbolic manipulation, and commonsense reasoning with increasing size of models. However, it is still unclear what the underlying capabilities of these LMs are. Surprisingly, we find that these models have limitations on certain basic symbolic manipulation tasks such as copy, reverse, and addition. When the total number of symbols or repeating symbols increases, the model performance drops quickly. We investigate the potential causes behind this phenomenon and examine a set of possible methods, including explicit positional markers, fine-grained computation steps, and LMs with callable programs. Experimental results show that none of these techniques can solve the simplest addition induction problem completely. In the end, we introduce LMs with tutor, which demonstrates every single step of teaching. LMs with tutor is able to deliver 100% accuracy in situations of OOD and repeating symbols, shedding new insights on the boundary of large LMs in induction.

Motivation & Objective

• To investigate why large language models (LLMs) fail on basic symbolic manipulation tasks such as copying, reversing, and addition, especially under out-of-distribution (OOD) or high-repetition conditions.
• To identify the root causes of these failures, particularly the difficulty in handling repeating symbols and maintaining positional awareness during generation.
• To evaluate existing mitigation strategies—fine-grained computation steps, positional markers, and callable programs—for improving LLM generalization on symbolic tasks.
• To propose and validate a new method, LMs with tutor, that uses structured, step-by-step teaching to achieve robust, 100% accurate performance on OOD and repetitive symbolic induction tasks.

Proposed method

• Introduces 'LMs with tutor'—a training and inference paradigm where models are prompted with 15 exemplars of step-by-step reasoning, demonstrating each operation (e.g., move, copy, reverse) in a structured, atomic manner.
• Uses explicit, human-readable reasoning steps (e.g., 'move item 1 to position 3') to guide the model through symbolic operations, mimicking a tutor’s instructional role.
• Employs a prompt format that includes both input and intermediate steps, with operations like 'rmov' (reverse move) or 'lmov' (left move) to track item positions during transformation.
• Compares the tutor method against standard few-shot prompting, fine-grained steps, positional markers (implicit or explicit), and callable programs across multiple tasks: addition, copy, and reverse.
• Trains models on in-distribution data (up to 5 items or digits) and evaluates generalization on OOD data (up to 30 items or digits) with high repetition levels.
• Uses T5, GPT-3, and DeBERTa models with and without pretraining to isolate the impact of architectural components like relative positional embeddings.

Experimental results

Research questions

• RQ1 Why do large language models fail on simple symbolic manipulation tasks like copying and reversing sequences, especially when inputs contain repeated symbols or exceed training distribution?
• RQ2 To what extent can fine-grained computation steps or explicit positional markers improve LLM generalization on symbolic induction tasks?
• RQ3 Can LMs with callable programs or program-agnostic methods improve OOD generalization in arithmetic and symbolic reasoning?
• RQ4 Does a structured, step-by-step teaching method (LMs with tutor) enable LMs to achieve 100% accuracy on OOD and high-repetition symbolic tasks where standard prompting fails?

Key findings

• Standard LLMs like GPT-3 and T5 achieve high in-distribution accuracy on addition and copying but drop significantly on OOD inputs, especially with repeating digits or long sequences.
• Even with fine-grained steps or explicit positional markers, models like T5 and DeBERTa fail to generalize beyond 20–30 items in the reverse task, indicating limited positional awareness.
• DeBERTa with relative positional embeddings overfits quickly on symbolic tasks, achieving 100% in-distribution accuracy in under 2k steps but failing on OOD data, suggesting poor generalization.
• The 'LMs with tutor' method achieves 100% accuracy across all OOD and high-repetition settings in copy, reverse, and addition tasks, outperforming all baselines.
• Callable programs improve OOD performance in addition tasks, indicating that primitive symbolic operations must be explicitly modeled for robust reasoning.
• The study reveals that current LLMs lack reliable positional and symbolic induction capabilities, even when scaled, and that structured teaching is essential for overcoming these limitations.