[Paper Review] Will the Real Linda Please Stand up...to Large Language Models? Examining the Representativeness Heuristic in LLMs

Pengda Wang, Zilin Xiao+2 more|arXiv (Cornell University)|Apr 1, 2024

Artificial Intelligence in Law4 citations

TL;DR

This paper introduces ReHeAT, a benchmark dataset of 202 representativeness heuristic problems across six cognitive bias types, to evaluate large language models' susceptibility to heuristic reasoning errors. Despite possessing correct statistical knowledge, all four tested LLMs (GPT-3.5, GPT-4, PaLM 2, LLaMA 2) exhibited representativeness heuristic biases, failing to apply statistical reasoning when stereotypes were present—though performance improved significantly with prompting hints, revealing a unique cognitive trap distinct from data bias.

ABSTRACT

Although large language models (LLMs) have demonstrated remarkable proficiency in modeling text and generating human-like text, they may exhibit biases acquired from training data in doing so. Specifically, LLMs may be susceptible to a common cognitive trap in human decision-making called the representativeness heuristic. This is a concept in psychology that refers to judging the likelihood of an event based on how closely it resembles a well-known prototype or typical example, versus considering broader facts or statistical evidence. This research investigates the impact of the representativeness heuristic on LLM reasoning. We created ReHeAT (Representativeness Heuristic AI Testing), a dataset containing a series of problems spanning six common types of representativeness heuristics. Experiments reveal that four LLMs applied to ReHeAT all exhibited representativeness heuristic biases. We further identify that the model's reasoning steps are often incorrectly based on a stereotype rather than on the problem's description. Interestingly, the performance improves when adding a hint in the prompt to remind the model to use its knowledge. This suggests the uniqueness of the representativeness heuristic compared to traditional biases. It can occur even when LLMs possess the correct knowledge while falling into a cognitive trap. This highlights the importance of future research focusing on the representativeness heuristic in model reasoning and decision-making and on developing solutions to address it.

Motivation & Objective

• To investigate whether large language models (LLMs) exhibit the representativeness heuristic, a cognitive bias where likelihood is judged by resemblance to prototypes rather than statistical evidence.
• To address the gap in research on cognitive biases in LLMs beyond data distribution biases, particularly those rooted in reasoning shortcuts.
• To develop a comprehensive, standardized benchmark to evaluate LLMs' susceptibility to heuristic reasoning errors.
• To explore whether prompting strategies, such as hints, can mitigate these heuristic-based errors despite models possessing correct statistical knowledge.

Proposed method

• Constructed ReHeAT, a dataset of 202 problems spanning six types of representativeness heuristics: base rate fallacy, conjunction fallacy, disjunction fallacy, insensitivity to sample size, misconceptions of chance, and regression fallacy.
• Adapted problems from classic psychological studies (e.g., Tversky & Kahneman, 1983) to create scenarios where statistical reasoning conflicts with stereotypical intuition.
• Evaluated four LLMs—GPT-3.5, GPT-4, PaLM 2, and LLaMA 2—on ReHeAT using zero-shot, chain-of-thought, in-context learning, and self-consistency prompting techniques.
• Designed two parallel questions per instance: a Statistical Prototype Question (SPQ) expressing logic statistically, and a Representativeness Heuristic Question (RHQ) embedding the same logic in a stereotypical scenario.
• Used a hint-based prompting strategy to test whether reminding models of their knowledge could override heuristic bias.
• Analyzed model reasoning steps to identify whether decisions were based on stereotypes or statistical facts.

Experimental results

Research questions

• RQ1 Do large language models exhibit the representativeness heuristic in reasoning, similar to humans?
• RQ2 Can LLMs correctly apply statistical reasoning when the problem is framed in a statistically neutral way, but fail when a stereotypical description is introduced?
• RQ3 To what extent does providing a prompting hint that reminds models of their knowledge reduce heuristic bias in LLMs?
• RQ4 Is the representativeness heuristic distinct from other forms of bias in LLMs, such as data distribution bias?
• RQ5 How do different prompting strategies (e.g., chain-of-thought, in-context learning) affect LLMs’ susceptibility to representativeness heuristic errors?

Key findings

• All four evaluated LLMs—GPT-3.5, GPT-4, PaLM 2, and LLaMA 2—demonstrated significant representativeness heuristic bias, incorrectly ranking stereotypical outcomes as more likely than statistically valid ones.
• Despite correctly answering the Statistical Prototype Question (SPQ), models failed to apply the same reasoning to the Representativeness Heuristic Question (RHQ), indicating a failure to transfer knowledge under heuristic pressure.
• The addition of a prompting hint significantly improved model performance on RHQs, suggesting that the models possess the correct knowledge but are blocked by cognitive shortcuts.
• Model reasoning often relied on stereotypical associations (e.g., 'girls are more likely in arts') rather than statistical facts (e.g., 'most students are in arts clubs'), even when the latter was explicitly stated.
• The representativeness heuristic in LLMs is distinct from data bias, as models can fail even when trained on unbiased data and possessing correct knowledge.
• The study reveals a unique cognitive trap in LLMs: they can be misled by intuitive, prototype-based reasoning despite having the statistical tools to reason correctly.