[Paper Review] Hallucination Mitigation using Agentic AI Natural Language-Based Frameworks

Diego Gosmar, Deborah A. Dahl|ArXiv.org|Jan 19, 2025

Mental Health Research Topics3 citations

TL;DR

This paper presents a multi-agent, NLP-based workflow using the OVON framework to detect, flag, and refine hallucinations in LLM outputs across three reviewer levels, achieving reduced hallucination scores.

ABSTRACT

Hallucinations remain a significant challenge in current Generative AI models, undermining trust in AI systems and their reliability. This study investigates how orchestrating multiple specialized Artificial Intelligent Agents can help mitigate such hallucinations, with a focus on systems leveraging Natural Language Processing (NLP) to facilitate seamless agent interactions. To achieve this, we design a pipeline that introduces over three hundred prompts, purposefully crafted to induce hallucinations, into a front-end agent. The outputs are then systematically reviewed and refined by second- and third-level agents, each employing distinct large language models and tailored strategies to detect unverified claims, incorporate explicit disclaimers, and clarify speculative content. Additionally, we introduce a set of novel Key Performance Indicators (KPIs) specifically designed to evaluate hallucination score levels. A dedicated fourth-level AI agent is employed to evaluate these KPIs, providing detailed assessments and ensuring accurate quantification of shifts in hallucination-related behaviors. A core component of this investigation is the use of the OVON (Open Voice Network) framework, which relies on universal NLP-based interfaces to transfer contextual information among agents. Through structured JSON messages, each agent communicates its assessment of the hallucination likelihood and the reasons underlying questionable content, thereby enabling the subsequent stage to refine the text without losing context. The results demonstrate that employing multiple specialized agents capable of interoperating with each other through NLP-based agentic frameworks can yield promising outcomes in hallucination mitigation, ultimately bolstering trust within the AI community.

Motivation & Objective

• Address the challenge of hallucinations in Generative AI LLMs.
• Propose a multi-agent pipeline that iteratively refines outputs via specialized reviewers.
• Introduce novel KPIs to quantify hallucination mitigation and explainability.

Proposed method

• Inject 310 prompts designed to induce hallucinations into a front-end agent.
• Use second- and third-level agents (with GPT-4o) to detect, disclaim, and refine claims via OVON JSON messages.
• Employ a fourth agent to evaluate KPIs and quantify hallucination shifts (THS).
• Facilitate inter-agent communication through OVON’s Conversation Envelopes (utterance and whisper events).
• Base orchestration on an Autogen-based agentic framework for iterative improvement.

Figure 1: Multi-Agent Scenario for Hallucination Mitigation Evaluation

Experimental results

Research questions

• RQ1 Can a multi-agent, NLP-based framework reduce hallucination likelihood in LLM outputs?
• RQ2 How do OVON-based inter-agent communications influence information flow and refinement?
• RQ3 What KPIs best quantify factuality, grounding, and the explicit signaling of fictional content?
• RQ4 Does a third-level reviewer further improve factuality and disclaimer effectiveness beyond a second-level reviewer?

Key findings

• A multi-agent pipeline progressively lowers hallucination scores across stages (front-end → second-level → third-level reviewers).
• New KPIs (Factual Claim Density, Factual Grounding References, Fictional Disclaimer Frequency, Explicit Contextualization Score) effectively quantify hallucination mitigation.
• A dedicated fourth agent can provide a quantified assessment of KPI shifts and total hallucination scores (THS).
• OVON-based JSON messaging preserves context while enabling targeted refinements across agents, improving explainability and reliability.

Figure 2: THS results over 310 prompts with 3 agents