[Paper Review] Boosting Theory-of-Mind Performance in Large Language Models via Prompting

Shima Rahimi Moghaddam, Christopher J. Honey|arXiv (Cornell University)|Apr 22, 2023

Topic Modeling39 citations

TL;DR

The paper shows that prompting, especially in-context learning with two-shot chain-of-thought or step-by-step prompts, boosts theory-of-mind performance in RLHF-trained LLMs, with GPT-4 reaching 100% ToM accuracy in prompts and zero-shot GPT-4 near 80%; human accuracy is 87%.

ABSTRACT

Large language models (LLMs) excel in many tasks in 2023, but they still face challenges in complex reasoning. Theory-of-mind (ToM) tasks, which require understanding agents' beliefs, goals, and mental states, are essential for common-sense reasoning involving humans, making it crucial to enhance LLM performance in this area. This study measures the ToM performance of GPT-4 and three GPT-3.5 variants (Davinci-2, Davinci-3, GPT-3.5-Turbo), and investigates the effectiveness of in-context learning in improving their ToM comprehension. We evaluated prompts featuring two-shot chain of thought reasoning and step-by-step thinking instructions. We found that LLMs trained with Reinforcement Learning from Human Feedback (RLHF) (all models excluding Davinci-2) improved their ToM accuracy via in-context learning. GPT-4 performed best in zero-shot settings, reaching nearly 80% ToM accuracy, but still fell short of the 87% human accuracy on the test set. However, when supplied with prompts for in-context learning, all RLHF-trained LLMs exceeded 80% ToM accuracy, with GPT-4 reaching 100%. These results demonstrate that appropriate prompting enhances LLM ToM reasoning, and they underscore the context-dependent nature of LLM cognitive capacities.

Motivation & Objective

• Assess ToM performance of GPT-4 and three GPT-3.5 variants on ToM tasks.
• Evaluate the effect of in-context learning prompts on ToM accuracy.
• Examine differences between RLHF-trained models and non-RLHF baselines on ToM tasks.

Proposed method

• Evaluate ToM performance of GPT-4 and three GPT-3.5 variants (Davinci-2, Davinci-3, GPT-3.5-Turbo).
• Test zero-shot and in-context learning prompts including two-shot chain-of-thought and step-by-step thinking instructions.
• Compare RLHF-trained models against non-RLHF baselines regarding ToM accuracy.
• Measure ToM accuracy against human performance as a benchmark.

Experimental results

Research questions

• RQ1 How does prompting influence theory-of-mind accuracy in large language models?
• RQ2 Do RLHF-trained LLMs benefit from in-context learning for ToM tasks more than non-RLHF models?
• RQ3 What are the best prompting configurations (zero-shot vs. in-context, chain-of-thought vs. step-by-step) for ToM in LLMs?
• RQ4 How close do LLM ToM performances come to human accuracy on the test set?

Key findings

• GPT-4 achieves nearly 80% ToM accuracy in zero-shot settings.
• RLHF-trained models (excluding Davinci-2) improve ToM accuracy via in-context learning.
• With prompting, all RLHF-trained LLMs exceed 80% ToM accuracy; GPT-4 with prompts reaches 100%.
• GPT-4 in zero-shot approaches approaches but does not reach the human accuracy of 87% on the test set.