[Paper Review] Lynx: An Open Source Hallucination Evaluation Model

S. Ravi, Bartosz Mielczarek+3 more|arXiv (Cornell University)|Jul 11, 2024

Psychedelics and Drug Studies4 citations

TL;DR

Lynx is an open-source, state-of-the-art hallucination detection model fine-tuned on diverse, real-world RAG scenarios to identify faithless LLM outputs. It outperforms GPT-4o, Claude-3-Sonnet, and other closed/open-source models on HaluBench, a new 15k-sample benchmark spanning finance, medicine, and other domains, achieving 87.4% accuracy on hallucination detection.

ABSTRACT

Retrieval Augmented Generation (RAG) techniques aim to mitigate hallucinations in Large Language Models (LLMs). However, LLMs can still produce information that is unsupported or contradictory to the retrieved contexts. We introduce LYNX, a SOTA hallucination detection LLM that is capable of advanced reasoning on challenging real-world hallucination scenarios. To evaluate LYNX, we present HaluBench, a comprehensive hallucination evaluation benchmark, consisting of 15k samples sourced from various real-world domains. Our experiment results show that LYNX outperforms GPT-4o, Claude-3-Sonnet, and closed and open-source LLM-as-a-judge models on HaluBench. We release LYNX, HaluBench and our evaluation code for public access.

Motivation & Objective

• To address the critical challenge of hallucination in Retrieval-Augmented Generation (RAG) systems, where LLMs generate factually unsupported or contradictory outputs despite retrieved context.
• To overcome the limitations of existing LLM-as-a-judge models, which often lack transparency, are closed-source, or underperform on nuanced hallucination detection.
• To develop a high-accuracy, open-source hallucination detection model that matches or exceeds closed-source models like GPT-4o and Claude-3-Sonnet.
• To create a comprehensive, diverse, and real-world grounded benchmark—HaluBench—for evaluating hallucination detection in reference-free settings.
• To enable reproducible, cost-effective, and scalable evaluation of RAG system outputs in production environments, especially in high-stakes domains like finance and healthcare.

Proposed method

• Fine-tune Llama-3-70B-Instruct on a curated dataset of perturbed QA examples from real-world domains, focusing on hard-to-detect hallucinations through semantic perturbations.
• Generate challenging hallucination examples by introducing subtle semantic changes to LLM-generated answers, creating plausible but factually inconsistent responses.
• Construct HaluBench, a 15k-sample benchmark with Context-Question-Answer triplets annotated for faithfulness, covering diverse domains including medicine, finance, and general knowledge.
• Use a reference-free evaluation protocol where the model judges faithfulness based solely on context and question, without ground-truth answers.
• Train two variants: Lynx (70B) for high accuracy and Lynx (8B) for efficiency and low-cost inference.
• Evaluate performance using human-annotated faithfulness labels and compare against GPT-4o, Claude-3-Sonnet, and other LLM-as-a-judge baselines.

Experimental results

Research questions

• RQ1 Can an open-source LLM-as-a-judge model outperform closed-source models like GPT-4o and Claude-3-Sonnet in hallucination detection on real-world RAG tasks?
• RQ2 How effective is a fine-tuned Llama-3 model in detecting subtle, hard-to-detect hallucinations that are semantically plausible but factually inconsistent?
• RQ3 To what extent does HaluBench, as a diverse and real-world grounded benchmark, improve the reliability and generalization of hallucination evaluation compared to existing datasets?
• RQ4 What is the performance gap between open-source and closed-source LLMs in hallucination detection, especially in specialized domains like medicine and finance?
• RQ5 Can a lightweight 8B model variant (Lynx-8B) maintain high faithfulness detection accuracy while significantly reducing inference cost compared to larger models?

Key findings

• Lynx (70B) achieves 87.4% accuracy on HaluBench, outperforming GPT-4o and Claude-3-Sonnet, demonstrating state-of-the-art performance in reference-free hallucination detection.
• Lynx (70B) is 8.3% more accurate than GPT-4o in detecting hallucinations in medical QA from PubMedQA, highlighting its strength in specialized domains.
• Lynx (8B) achieves high performance at a fraction of the size and cost of closed-source models, making it suitable for scalable, low-cost deployment in production RAG systems.
• The fine-tuned Llama-3-70B model improves average accuracy by 7.8% over the base model, indicating strong gains from domain-specific, perturbation-based fine-tuning.
• Lynx (70B) outperforms GPT-3.5-Turbo by 27.6% in average accuracy across all tasks, closing the performance gap between open and closed-source models in hallucination detection.
• HaluBench shows high agreement with human annotations and contains a balanced, real-world distribution of faithful and hallucinated examples, making it a reliable benchmark for future evaluation.