[Paper Review] LLMs achieve adult human performance on higher-order theory of mind tasks

Winnie Street, John Oliver Siy+8 more|arXiv (Cornell University)|May 29, 2024

EEG and Brain-Computer Interfaces16 citations

TL;DR

The paper introduces MoToMQA to assess ToM in LLMs up to 6th order and shows GPT-4 and Flan-PaLM achieving adult-level or near-adult performance, with GPT-4 excelling at 6th order. It suggests model size and finetuning boost higher-order ToM abilities and discusses implications and risks.

ABSTRACT

This paper examines the extent to which large language models (LLMs) have developed higher-order theory of mind (ToM); the human ability to reason about multiple mental and emotional states in a recursive manner (e.g. I think that you believe that she knows). This paper builds on prior work by introducing a handwritten test suite -- Multi-Order Theory of Mind Q&A -- and using it to compare the performance of five LLMs to a newly gathered adult human benchmark. We find that GPT-4 and Flan-PaLM reach adult-level and near adult-level performance on ToM tasks overall, and that GPT-4 exceeds adult performance on 6th order inferences. Our results suggest that there is an interplay between model size and finetuning for the realisation of ToM abilities, and that the best-performing LLMs have developed a generalised capacity for ToM. Given the role that higher-order ToM plays in a wide range of cooperative and competitive human behaviours, these findings have significant implications for user-facing LLM applications.

Motivation & Objective

• Assess higher-order theory of mind (ToM) in large language models (LLMs) up to 6th order.
• Compare LLM ToM performance with adult human benchmarks using a handwritten test suite.
• Examine how ToM performance differs from factual recall tasks of similar complexity.
• Investigate the impact of model size, finetuning, and prompting on ToM capabilities.
• Explore potential ethical and practical implications of LLMs possessing higher-order ToM.

Proposed method

• Introduce MoToMQA benchmark based on the Imposing Memory Task (IMT) for adults.
• Create 7 short social stories with 20 true/false statements; 10 ToM (orders 2-6) and 10 factual statements.
• Collect adult human responses (n≈29,259) under controlled story/response conditions for comparability.
• Evaluate 5 LLMs (GPT-3.5 Turbo Instruct, GPT-4, LaMDA, PaLM, Flan-PaLM) using logprob-based scoring with multiple candidate tokens to derive true/false probabilities.
• Map human and LLM outcomes to binary responses; align units by collapsing human responses to single true/false per statement.
• Analyze results with Cochran’s Q, McNemar’s tests, and independent proportions; test for anchoring effects across prompt/response conditions.

Figure 1: Human, LaMDA, PaLM, Flan-PaLM, GPT-3.5 and GPT-4 performance on ToM tasks up to order 6

Experimental results

Research questions

• RQ1 To what extent can LLMs perform higher-order ToM reasoning (orders 2-6) relative to adult humans?
• RQ2 How does ToM performance vary with model size and finetuning (instruction tuning, RLHF) among contemporary LLMs?
• RQ3 How does ToM performance compare to performance on equivalent factual tasks of similar complexity?
• RQ4 Do prompting and response-order (anchoring) effects influence LLM and human ToM responses?
• RQ5 What factors (e.g., multimodality, training data) might underlie observed ToM capabilities in the best-performing models?

Key findings

• GPT-4 and Flan-PaLM achieve at-human or near-adult ToM performance across orders, with GPT-4 surpassing human performance on 6th-order inferences.
• Among models, GPT-4 and Flan-PaLM outperform others overall; GPT-4 and humans show comparable performance on orders 2-4 and 6, with GPT-4 exceeding humans on order 6.
• GPT-4 achieves the highest aggregate ToM accuracy (89%), and GPT-4 attains 93% on order 6 whereas humans reach 82% on the same order.
• Factual recall tasks are generally easier than ToM tasks for most models; some models (GPT-4, Flan-PaLM) maintain strong ToM performance despite this gap.
• There is evidence that larger, instruction-tuned models with potential multimodal or feedback-based training better realize ToM abilities than smaller, non-finetuned models.
• Anchoring (true-first vs false-first prompts) affects some models (PaLM, GPT-3.5) but not GPT-4, Flan-PaLM, or humans.