[Paper Review] Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse

Ryan Liu, Jiayi Geng+4 more|arXiv (Cornell University)|Oct 27, 2024

Complex Systems and Decision Making4 citations

TL;DR

This paper identifies that chain-of-thought (CoT) prompting can degrade performance in large language and multimodal models on tasks where verbal deliberation harms human performance, particularly in implicit statistical learning, visual recognition, and exception-laden classification. It demonstrates that CoT reduces accuracy by up to 36.3% compared to zero-shot inference, offering a cognitive psychology-informed heuristic to predict such failure cases.

ABSTRACT

Chain-of-thought (CoT) prompting has become a widely used strategy for improving large language and multimodal model performance. However, it is still an open question under which settings CoT systematically reduces performance. In this paper, we seek to identify the characteristics of tasks where CoT reduces performance by drawing inspiration from cognitive psychology, focusing on six representative tasks from the psychological literature where deliberation hurts performance in humans. In three of these tasks, state-of-the-art models exhibit significant performance drop-offs with CoT (up to 36.3\% absolute accuracy for OpenAI o1-preview compared to GPT-4o), while in others, CoT effects are mixed, with positive, neutral, and negative changes. While models and humans do not exhibit perfectly parallel cognitive processes, considering cases where thinking has negative consequences for humans helps identify settings where it negatively impacts models. By connecting the literature on human verbal thinking and deliberation with evaluations of CoT, we offer a perspective for understanding the impact of inference-time reasoning.

Motivation & Objective

• To identify settings where chain-of-thought (CoT) prompting reduces model performance rather than improving it.
• To explore whether cognitive psychology findings on human performance degradation due to verbal thinking can predict similar failures in LLMs and VLMs.
• To test whether tasks where human performance declines under deliberation (condition i) and where human constraints generalize to models (condition ii) are also failure cases for CoT.
• To distinguish between cases where CoT harms models due to shared cognitive constraints and cases where model capabilities diverge from human limitations.

Proposed method

• Adapted six well-studied psychological tasks—three satisfying both (i) human performance drop under verbal thinking and (ii) constraint generalization to models, and three satisfying (i) but not (ii)—for evaluation on state-of-the-art LLMs and VLMs.
• Evaluated performance using zero-shot inference versus CoT prompting across three core failure tasks: implicit statistical learning, face recognition, and classification with exceptions.
• Used standardized benchmarks and controlled prompt engineering to isolate the impact of CoT on model accuracy, comparing to zero-shot baselines.
• Extended analysis to alternative reasoning techniques like tree-of-thought, confirming that performance degradation persists across inference-time reasoning methods.
• Applied a heuristic framework linking human cognitive psychology to model behavior, using known human failure cases as predictors for model failure cases.
• Conducted extensive ablation and prompt variation studies to rule out implementation-specific artifacts as the cause of performance differences.

Experimental results

Research questions

• RQ1 In which tasks does chain-of-thought prompting reduce model performance rather than improve it?
• RQ2 Can cases where verbal thinking impairs human performance predict similar negative effects in large language and multimodal models?
• RQ3 Are there shared cognitive constraints between humans and models that explain why CoT harms performance in certain tasks?
• RQ4 Do tasks where human and model priors or working memory limits differ show divergent responses to CoT prompting?
• RQ5 To what extent do alternative reasoning techniques like tree-of-thought replicate the negative effects seen with standard CoT?

Key findings

• CoT prompting reduced model accuracy by up to 36.3 percentage points in the implicit statistical learning task, with OpenAI o1-preview dropping from 94.00% (zero-shot) to 57.70% with CoT.
• In face recognition tasks, CoT caused a 21.8% absolute drop in accuracy for GPT-4o, from 82.5% (zero-shot) to 60.7% with CoT.
• For classification with exceptions, CoT reduced accuracy by 18.5% for LLaMA-3-70B, dropping from 85.2% (zero-shot) to 66.7% with CoT.
• In tasks where human and model constraints differ (e.g., working memory limits), CoT did not degrade performance and sometimes improved it, indicating the failure is not universal.
• Tree-of-thought prompting also failed to recover performance in implicit statistical learning, achieving only 64.55% accuracy versus 62.52% for standard CoT, still far below the 94.00% zero-shot baseline.
• The study found no evidence that prompt implementation differences alone explained the performance drop, as extensive prompt variations failed to eliminate the negative effect in the three core failure tasks.