[Paper Review] Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models

Yue Zhang, Yafu Li+14 more|arXiv (Cornell University)|Sep 3, 2023

Topic ModelingComputer Science233 citations

TL;DR

A comprehensive survey of hallucination in large language models, covering definitions, benchmarks, sources, evaluation methods, and mitigation across the model life cycle.

ABSTRACT

While large language models (LLMs) have demonstrated remarkable capabilities across a range of downstream tasks, a significant concern revolves around their propensity to exhibit hallucinations: LLMs occasionally generate content that diverges from the user input, contradicts previously generated context, or misaligns with established world knowledge. This phenomenon poses a substantial challenge to the reliability of LLMs in real-world scenarios. In this paper, we survey recent efforts on the detection, explanation, and mitigation of hallucination, with an emphasis on the unique challenges posed by LLMs. We present taxonomies of the LLM hallucination phenomena and evaluation benchmarks, analyze existing approaches aiming at mitigating LLM hallucination, and discuss potential directions for future research.

Motivation & Objective

• Define LLM hallucination and classify its three main types (input-conflicting, context-conflicting, fact-conflicting).
• Summarize evaluation benchmarks and metrics used to study LLM hallucination.
• Identify sources of hallucination across the LLM life cycle and discuss mitigation strategies.
• Review mitigation approaches at different stages (pre-training, fine-tuning, RLHF, inference) and outline future directions.

Proposed method

• Present a taxonomy of LLM hallucination types with examples (input-, context-, fact-conflicting).
• Survey representative benchmarks and task formats (generation vs. discrimination).
• Analyze sources of hallucination from pre-training data, knowledge recall/reasoning limits, alignment, and generation strategy.
• Categorize mitigation by timing in the LLM life cycle (pre-training, alignment/fine-tuning, inference).
• Discuss evaluation challenges and emphasize the need for robust, automatic benchmarks.

Experimental results

Research questions

• RQ1 What are the canonical types of hallucination in LLM outputs and how can they be detected?
• RQ2 What benchmarks and metrics effectively evaluate LLM hallucination across tasks and formats?
• RQ3 What are the primary sources of hallucination in LLMs, and how can mitigation be effectively applied across the model life cycle?

Key findings

• Hallucination in LLMs is categorized into input-conflicting, context-conflicting, and fact-conflicting types with examples and discussion.
• A range of benchmarks exists for evaluating hallucination, with generation-based and discrimination-based formats across QA, TI, and text completion tasks.
• Hallucination sources span pre-training data quality, knowledge memorization limits, overconfidence, misalignment, and generation strategies like token-level optimization.
• Mitigation strategies are discussed at each life-cycle stage, highlighting data curation, alignment improvements, and prompting/inference-time techniques.
• The survey emphasizes the unique challenges of LLM hallucination due to scale, versatility, and the difficulty of automatic evaluation, and points to open-source resources for ongoing study.