[Paper Review] LLMs Will Always Hallucinate, and We Need to Live With This

Sourav Banerjee, Ayushi Agarwal+1 more|arXiv (Cornell University)|Sep 9, 2024

Legal Education and Practice InnovationsSocial Sciences24 citations

TL;DR

The paper argues that hallucinations are an intrinsic, structural feature of LLMs rooted in their mathematical and logical foundations, and cannot be eliminated through architecture or data alone.

ABSTRACT

As Large Language Models become more ubiquitous across domains, it becomes important to examine their inherent limitations critically. This work argues that hallucinations in language models are not just occasional errors but an inevitable feature of these systems. We demonstrate that hallucinations stem from the fundamental mathematical and logical structure of LLMs. It is, therefore, impossible to eliminate them through architectural improvements, dataset enhancements, or fact-checking mechanisms. Our analysis draws on computational theory and Godel's First Incompleteness Theorem, which references the undecidability of problems like the Halting, Emptiness, and Acceptance Problems. We demonstrate that every stage of the LLM process-from training data compilation to fact retrieval, intent classification, and text generation-will have a non-zero probability of producing hallucinations. This work introduces the concept of Structural Hallucination as an intrinsic nature of these systems. By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated.

Motivation & Objective

• Motivate a critical examination of LLM limitations by framing hallucinations as intrinsic to LLMs.
• Define and formalize the concept of Structural Hallucinations as an inherent property of the models.
• Analyze how training, retrieval, interpretation, and generation stages contribute to non-zero hallucination risk.
• Propose and review mitigation strategies across the model life cycle (training, intent classification, information retrieval, and output generation).

Proposed method

• Present a formal view of LLM generation as a product of token-level likelihoods.
• Introduce and explain Structural Hallucinations as a consequence of undecidability results (e.g., Halting, Emptiness, Acceptance) in the context of LLMs.
• Survey architectural alternatives to attention (e.g., Mamba, Jamba, KANs) and discuss their implications for long-context modeling.
• Describe Transfer Learning and Parameter-Efficient Fine-Tuning approaches (Adapters, BitFit, LoRA) as practical mitigations.
• Explain Retrieval-Augmented Generation (RAG) and its role in grounding outputs.
• Discuss uncertainty quantification and faithfulness explanations as post-generation aids.

Experimental results

Research questions

• RQ1 Are hallucinations in LLMs avoidable, or are they an intrinsic property of the models’ mathematical and logical structure?
• RQ2 At which stages of the LLM pipeline (training, intent classification, information retrieval, output generation, post-generation checking) do hallucinations arise and how can they be mitigated?
• RQ3 What alternative architectures or training paradigms might reduce, but not eliminate, hallucinations?
• RQ4 Can retrieval, uncertainty quantification, and explainability methods meaningfully reduce the impact of hallucinations in practice?

Key findings

• Hallucinations are argued to be unavoidable due to the inherent undecidability and incompleteness in training data and in retrieval/classification/generation processes.
• Structural Hallucinations are proposed as a formal category describing hallucinations that arise from the core structure of LLMs, not just from data quality.
• A range of mitigation strategies are reviewed, including Chain-of-Thought prompting, self-consistency, uncertainty quantification, RAG, and PEFT methods, though none guarantees elimination of hallucinations.
• Alternative architectures and models (e.g., Mamba, Jamba, KANs) are discussed as potential avenues for efficiency and context handling, but do not remove the fundamental issue.
• Post-generation checks and faithful explanations are proposed to help users identify hallucinations, even if they cannot be fully prevented.