[Paper Review] Self-Consistency Improves Chain of Thought Reasoning in Language Models

Xuezhi Wang, Jason Lee+6 more|arXiv (Cornell University)|Mar 21, 2022

Topic Modeling674 citations

TL;DR

The paper introduces self-consistency, a decoding strategy that samples diverse chain-of-thought paths and aggregates the most consistent final answer, significantly boosting reasoning accuracy across arithmetic and commonsense tasks without extra training.

ABSTRACT

Chain-of-thought prompting combined with pre-trained large language models has achieved encouraging results on complex reasoning tasks. In this paper, we propose a new decoding strategy, self-consistency, to replace the naive greedy decoding used in chain-of-thought prompting. It first samples a diverse set of reasoning paths instead of only taking the greedy one, and then selects the most consistent answer by marginalizing out the sampled reasoning paths. Self-consistency leverages the intuition that a complex reasoning problem typically admits multiple different ways of thinking leading to its unique correct answer. Our extensive empirical evaluation shows that self-consistency boosts the performance of chain-of-thought prompting with a striking margin on a range of popular arithmetic and commonsense reasoning benchmarks, including GSM8K (+17.9%), SVAMP (+11.0%), AQuA (+12.2%), StrategyQA (+6.4%) and ARC-challenge (+3.9%).

Motivation & Objective

• Motivate improving reasoning in large language models beyond standard chain-of-thought prompting.
• Propose a decoding method that generates diverse reasoning paths to identify the most reliable answer.
• Demonstrate robustness and performance gains across multiple models and reasoning benchmarks.
• Show that the approach requires no additional supervision or fine-tuning.
• Explore potential for uncertainty estimation and rationale collection from model outputs.

Proposed method

• Prompt the model with chain-of-thought exemplars as in prior CoT prompting.
• Sample a diverse set of reasoning paths from the model's decoder using temperature/top-k/nucleus strategies.
• Marginalize over the sampled reasoning paths by aggregating the final answers to select the most consistent one (majority vote or weighted aggregation).
• Treat each sampled path as a latent variable linking reasoning steps to the final answer, without training any auxiliary models.
• Compare aggregation strategies (majority vote vs. weighted sums) and show most-consistent answers yield better performance.
• Demonstrate that self-consistency is unsupervised, model-agnostic, and does not require fine-tuning or extra annotations.

Experimental results

Research questions

• RQ1 Does introducing diversity in reasoning paths via sampling improve final answer accuracy beyond greedy chain-of-thought decoding?
• RQ2 How should final answers be aggregated across multiple sampled paths to maximize correctness?
• RQ3 Is the self-consistency approach robust across model scales, prompting strategies, and sampling parameters?
• RQ4 Can self-consistency provide uncertainty estimates or assist in scenarios where prompts are imperfect?
• RQ5 How does self-consistency compare to sample-and-rank, beam search, and traditional ensembles?

Key findings

• Self-consistency yields substantial accuracy gains over standard chain-of-thought prompting across arithmetic and commonsense tasks.
• Gains are larger with bigger models (e.g., LaMDA-137B, PaLM-540B, GPT-3) and reach new state-of-the-art levels on several benchmarks.
• Across tasks like GSM8K, SVAMP, AQuA, StrategyQA, and ARC-challenge, reported improvements include up to +17.9%, +11.0%, +12.2%, +6.4%, and +3.9% respectively.
• Aggregating via a normalized weighted sum or majority vote over many sampled paths outperforms unweighted approaches and single-path greedy decoding.
• Self-consistency is robust to sampling strategies and model prompts, and can improve performance even when chain-of-thought hurts standard prompting.
• Compared to sample-and-rank, beam search, and ensemble methods, self-consistency provides larger gains with a single model and no extra training.