[Paper Review] Solving Math Word Problems via Cooperative Reasoning induced Language Models

Xinyu Zhu, Junjie Wang+5 more|arXiv (Cornell University)|Oct 28, 2022

Topic Modeling4 citations

TL;DR

This paper proposes CoRe, a cooperative reasoning framework that mimics human dual-process cognition by combining a generator (System 1) for reasoning path generation and verifiers (System 2) for step-by-step feedback during inference. By integrating token-level and sentence-level verifiers within a Monte Carlo Tree Search (MCTS)-inspired loop, CoRe improves reasoning quality and achieves up to 9.6% relative improvement over state-of-the-art models on MultiArith, even with smaller models.

ABSTRACT

Large-scale pre-trained language models (PLMs) bring new opportunities to challenging problems, especially those that need high-level intelligence, such as the math word problem (MWPs). However, directly applying existing PLMs to MWPs can fail as the generation process lacks sufficient supervision and thus lacks fast adaptivity as humans. We notice that human reasoning has a dual reasoning framework that consists of an immediate reaction system (system 1) and a delicate reasoning system (system 2), where the entire reasoning is determined by their interaction. This inspires us to develop a cooperative reasoning-induced PLM for solving MWPs, called Cooperative Reasoning (CoRe), resulting in a human-like reasoning architecture with system 1 as the generator and system 2 as the verifier. In our approach, the generator is responsible for generating reasoning paths, and the verifiers are used to supervise the evaluation in order to obtain reliable feedback for the generator. We evaluate our CoRe framework on several mathematical reasoning datasets and achieve decent improvement over state-of-the-art methods, up to 9.6% increase over best baselines. Our codes are available at https://github.com/TianHongZXY/CoRe

Motivation & Objective

• To address the lack of generalization and adaptivity in existing large language models (LLMs) when solving math word problems (MWPs).
• To emulate human dual-process reasoning—fast intuition (System 1) and deliberate verification (System 2)—in neural models.
• To improve reasoning path quality by introducing feedback during generation, not just at the final answer stage.
• To enable zero-shot and fine-tuned reasoning with high-quality, self-supervised reasoning paths via a self-thinking strategy.

Proposed method

• The framework uses a generator (System 1) to produce reasoning paths token-by-token, modeled as a sequence of probabilistic decisions.
• Two types of verifiers—token-level and sentence-level—are introduced to score reasoning steps in real time during generation.
• The verifiers provide feedback via MCTS-style tree search, enabling dynamic evaluation and selection of high-quality reasoning paths.
• The training objective combines reasoning path perplexity and solution score to align generation with both logical coherence and correctness.
• A self-thinking strategy is applied using synthetic data from CoRe’s own generator and verifiers to boost reasoning capability without external supervision.
• The method operates in a closed-loop generation process, where feedback from verifiers guides the generator iteratively before finalizing the answer.

Experimental results

Research questions

• RQ1 Can a cooperative reasoning framework that mimics human dual-process cognition improve reasoning performance on math word problems?
• RQ2 How does real-time feedback during reasoning path generation affect model generalization and accuracy compared to end-of-sequence evaluation?
• RQ3 To what extent can a small language model outperform much larger models using this cooperative verification mechanism?
• RQ4 Can self-thinking based on model-generated data enhance reasoning ability without requiring large-scale annotated datasets?

Key findings

• CoRe achieves a 9.6% relative improvement over the best existing baselines on the MultiArith dataset, demonstrating strong generalization despite using smaller models.
• The framework significantly outperforms standard PLMs in both zero-shot and fine-tuning settings across multiple mathematical reasoning datasets.
• The integration of verifiers at the token and sentence level enables early detection of flawed reasoning paths, reducing reliance on final answer correctness alone.
• Self-thinking improves reasoning ability, allowing the model to correct errors in its own reasoning, as shown in qualitative examples.
• The method generalizes across datasets: zero-shot performance on unseen arithmetic datasets confirms transferability of reasoning skills.
• Even with only 7.5K training samples from GSM8K, CoRe demonstrates strong reasoning transfer, suggesting low data efficiency for reasoning task adaptation.