[Paper Review] A Weakly Supervised Model for Solving Math word Problems.

Oishik Chatterjee, Aashish Waikar+3 more|arXiv (Cornell University)|Apr 14, 2021

Natural Language Processing Techniques27 references4 citations

TL;DR

This paper proposes a weakly supervised model for solving math word problems using only the final answer as supervision, bypassing the need for costly equation annotations. By training a sequence-to-sequence model to generate equations from problem descriptions and answers, the approach achieves 56.0% accuracy on the Math23K dataset, demonstrating strong performance without full equation-level supervision.

ABSTRACT

Solving math word problems (MWPs) is an important and challenging problem in natural language processing. Existing approaches to solve MWPs require full supervision in the form of intermediate equations. However, labeling every math word problem with its corresponding equations is a time-consuming and expensive task. In order to address this challenge of equation annotation, we propose a weakly supervised model for solving math word problems by requiring only the final answer as supervision. We approach this problem by first learning to generate the equation using the problem description and the final answer, which we then use to train a supervised MWP solver. We propose and compare various weakly supervised techniques to learn to generate equations directly from the problem description and answer. Through extensive experiment, we demonstrate that even without using equations for supervision, our approach achieves an accuracy of 56.0 on the standard Math23K dataset. We also curate and release a new dataset for MWPs in English consisting of 10227 instances suitable for training weakly supervised models.

Motivation & Objective

• Address the high cost and effort of annotating intermediate equations for math word problems.
• Enable training of MWP solvers with only final answers as supervision, reducing annotation burden.
• Develop weakly supervised techniques to generate equations directly from problem text and answers.
• Curate and release a new English MWP dataset of 10,227 instances for training weakly supervised models.
• Demonstrate that high-quality MWP solving is achievable without full equation-level supervision.

Proposed method

• Train a sequence-to-sequence model to generate equations from problem descriptions and final answers, using only answer-level supervision.
• Apply various weakly supervised learning techniques to optimize equation generation without access to ground-truth equations during training.
• Use the generated equations to train a supervised MWP solver, enabling end-to-end learning from answers alone.
• Leverage the model’s equation generation capability to simulate supervised signals for downstream MWP solving.
• Fine-tune the equation generator using a combination of answer-aligned training objectives and sequence generation loss.
• Release a new dataset of 10,227 English math word problems with only answers annotated, suitable for weakly supervised training.

Experimental results

Research questions

• RQ1 Can a model learn to generate accurate equations for math word problems using only the final answer as supervision?
• RQ2 How effective are different weakly supervised techniques in generating equations from problem-text and answers?
• RQ3 To what extent can a weakly supervised model achieve competitive performance on standard MWP benchmarks without equation annotations?
• RQ4 Can a curated dataset of answer-annotated math word problems support effective training of weakly supervised MWP solvers?
• RQ5 Does the performance of the final MWP solver remain competitive when trained solely on answer-level supervision?

Key findings

• The proposed weakly supervised model achieves 56.0% accuracy on the standard Math23K benchmark without using any equation annotations.
• The model successfully learns to generate plausible equations from problem descriptions and final answers, enabling effective downstream MWP solving.
• Various weakly supervised techniques for equation generation yield competitive results, demonstrating the feasibility of answer-only supervision.
• The curated dataset of 10,227 English MWP instances with only answer annotations is released for public use in training weakly supervised models.
• The approach significantly reduces annotation cost while maintaining strong performance, proving viable for low-resource MWP training scenarios.
• The results show that answer-level supervision alone can yield a functional MWP solver, challenging the necessity of full equation-level annotations.