[Paper Review] Banishing LLM Hallucinations Requires Rethinking Generalization

Liujun Li, Saksham Consul+10 more|arXiv (Cornell University)|Jun 25, 2024

Hallucinations in medical conditions8 citations

TL;DR

The paper argues that memorization of random data by LLMs challenges traditional generalization views, proposes Lamini-1 with a massive memory-expert mixture to banish hallucinations, and analyzes computational trade-offs.

ABSTRACT

Despite their powerful chat, coding, and reasoning abilities, Large Language Models (LLMs) frequently hallucinate. Conventional wisdom suggests that hallucinations are a consequence of a balance between creativity and factuality, which can be mitigated, but not eliminated, by grounding the LLM in external knowledge sources. Through extensive systematic experiments, we show that these traditional approaches fail to explain why LLMs hallucinate in practice. Specifically, we show that LLMs augmented with a massive Mixture of Memory Experts (MoME) can easily memorize large datasets of random numbers. We corroborate these experimental findings with a theoretical construction showing that simple neural networks trained to predict the next token hallucinate when the training loss is above a threshold as it usually does in practice when training on internet scale data. We interpret our findings by comparing against traditional retrieval methods for mitigating hallucinations. We use our findings to design a first generation model for removing hallucinations -- Lamini-1 -- that stores facts in a massive mixture of millions of memory experts that are retrieved dynamically.

Motivation & Objective

• Question the sufficiency of generalization error as a predictor of hallucinations in LLMs.
• Demonstrate that LLMs can memorize random data without increasing generalization error.
• Propose a memory-based architecture (Lamini-1) to store and retrieve facts with near-zero training loss on key facts.
• Assess the computational and energy costs of removing hallucinations via extensive memorization.
• Highlight limitations of traditional retrieval and regularization approaches for mitigating hallucinations.

Proposed method

• Perform randomized labeling experiments showing LLMs memorize random tokens with near-zero training loss.
• Use regularization tests to compare behavior when trained on true versus random labels.
• Analyze information retrieval inspired architectures and their limitations for hallucination mitigation.
• Introduce Lamini-1 architecture with a massive mixture of memory experts (MoME) attached to a frozen backbone.
• Describe training protocol that memorizes facts by updating only a subset of memory experts with cross-attention-based routing.
• Provide computational cost estimates for achieving near-zero loss on key facts and compare with baseline scaling recipes.

Experimental results

Research questions

• RQ1 Can generalization error distinguish between LLMs that hallucinate and those that do not?
• RQ2 To what extent can LLMs memorize random data without loss in generalization performance?
• RQ3 Is a memory-based architecture like Lamini-1 capable of eliminating hallucinations for factual recall, and at what computational cost?
• RQ4 How do traditional retrieval and regularization explanations for hallucinations fare against randomization and memory-muning results?

Key findings

• LLMs can memorize large datasets of random labels with near-zero finetuning loss, while not suffering noticeable generalization degradation on standard tasks.
• Generalization error alone does not discriminate between models that hallucinate and those that don't; benchmarks like MMLU do not reliably predict factual recall capabilities.
• Targeted training to remove hallucinations via memorization incurs substantial computational costs under traditional scaling recipes (e.g., 100x more SGD epochs).
• A memory-based Lamini-1 architecture (MoME) can memorize and recall facts by retrieving from millions of memory experts, enabling near-zero loss on key facts and improved factual recall with specialized hardware kernels.
• Memory tuning reduces hallucinations by storing facts as explicit memories rather than encoding them solely in transformer parameters.