[Paper Review] HALoGEN: Fantastic LLM Hallucinations and Where to Find Them

Abhilasha Ravichander, Shrusti Ghela+2 more|ArXiv.org|Jan 14, 2025

Mental Health and Psychiatry3 citations

TL;DR

HALOGEN presents a large-scale, multi-domain benchmark to quantify and categorize hallucinations in 14 LLMs across 9 tasks, with automatic atomic-unit verification and a new error taxonomy exploring training-data vs fabrication origins.

ABSTRACT

Despite their impressive ability to generate high-quality and fluent text, generative large language models (LLMs) also produce hallucinations: statements that are misaligned with established world knowledge or provided input context. However, measuring hallucination can be challenging, as having humans verify model generations on-the-fly is both expensive and time-consuming. In this work, we release HALoGEN, a comprehensive hallucination benchmark consisting of: (1) 10,923 prompts for generative models spanning nine domains including programming, scientific attribution, and summarization, and (2) automatic high-precision verifiers for each use case that decompose LLM generations into atomic units, and verify each unit against a high-quality knowledge source. We use this framework to evaluate ~150,000 generations from 14 language models, finding that even the best-performing models are riddled with hallucinations (sometimes up to 86% of generated atomic facts depending on the domain). We further define a novel error classification for LLM hallucinations based on whether they likely stem from incorrect recollection of training data (Type A errors), or incorrect knowledge in training data (Type B errors), or are fabrication (Type C errors). We hope our framework provides a foundation to enable the principled study of why generative models hallucinate, and advances the development of trustworthy large language models.

Motivation & Objective

• Measure the prevalence of hallucinations in long-form LLM generations across diverse domains.
• Decompose model outputs into atomic facts and verify each unit against reliable sources.
• Classify hallucination errors by potential origins in pretraining data or fabrication.
• Provide a multi-domain dataset and tooling to study why LLMs hallucinate and how to reduce it.

Proposed method

• Construct HALOGEN with nine tasks encompassing response-based and refusal-based settings.
• Automatically decompose model outputs into atomic facts using task-specific parsers or prompts.
• Verify each atomic unit against external sources or LLM-based classifiers (e.g., Package Index, Semantic Scholar, entailed/extrinsic checks).
• Define and compute hallucination, response ratio, and utility scores from the decomposition and verification results.
• Annotate hallucinations to assign error types (Type A, B, C) based on pretraining data relationships.
• Analyze attribution by tracing hallucinations to pretraining corpora (e.g., C4, OpenWebText) and contextual sources.

Experimental results

Research questions

• RQ1 What is the prevalence of hallucinations in state-of-the-art LLMs across diverse domains?
• RQ2 How do hallucinations vary by domain and task type (code, summarization, biographies, scientific attribution, etc.)?
• RQ3 Can automatic verifiers reliably decompose and judge the factuality of atomic units in generated text?
• RQ4 What are the primary sources of hallucinations in pretraining data vs fabrication (Type A vs Type B vs Type C)?
• RQ5 How do model size, architecture, and openness influence hallucination rates and refusal behavior?

Key findings

• LLMs exhibit substantial hallucinations across tasks, with rates ranging from 4% to 86% depending on domain and model.
• Hallucination patterns are domain-dependent; larger models generally hallucinate less on some response-based tasks but not necessarily on refusal-based tasks.
• GPT-3.5 and GPT-4 (closed models) show strongest factual performance across many tasks, with open models like Llama-2 and Mistral varying in different categories.
• A large share of code-package hallucinations trace to pretraining data (Type B), while some factual errors (Type A) arise from correct facts present in pretraining but misused.
• Content-grounded summarization tends to have intrinsic hallucinations (context processing errors) as the dominant source of errors among high-utility models.
• On summarization, 83% of hallucinations are intrinsic (context processing) while 17% are extrinsic (external facts introduced).