[Paper Review] Large Language Model Agent for Fake News Detection

Xinyi Li, Yongfeng Zhang+1 more|arXiv (Cornell University)|Apr 30, 2024

Misinformation and Its Impacts4 citations

TL;DR

This paper proposes FactAgent, an agentic framework that uses large language models (LLMs) to detect fake news through a structured, expert-designed workflow combining internal knowledge and external tools. Without requiring any training, FactAgent achieves high accuracy by decomposing verification into sub-steps, providing transparent, step-by-step reasoning and outperforming supervised models and standard prompting techniques.

ABSTRACT

In the current digital era, the rapid spread of misinformation on online platforms presents significant challenges to societal well-being, public trust, and democratic processes, influencing critical decision making and public opinion. To address these challenges, there is a growing need for automated fake news detection mechanisms. Pre-trained large language models (LLMs) have demonstrated exceptional capabilities across various natural language processing (NLP) tasks, prompting exploration into their potential for verifying news claims. Instead of employing LLMs in a non-agentic way, where LLMs generate responses based on direct prompts in a single shot, our work introduces FactAgent, an agentic approach of utilizing LLMs for fake news detection. FactAgent enables LLMs to emulate human expert behavior in verifying news claims without any model training, following a structured workflow. This workflow breaks down the complex task of news veracity checking into multiple sub-steps, where LLMs complete simple tasks using their internal knowledge or external tools. At the final step of the workflow, LLMs integrate all findings throughout the workflow to determine the news claim's veracity. Compared to manual human verification, FactAgent offers enhanced efficiency. Experimental studies demonstrate the effectiveness of FactAgent in verifying claims without the need for any training process. Moreover, FactAgent provides transparent explanations at each step of the workflow and during final decision-making, offering insights into the reasoning process of fake news detection for end users. FactAgent is highly adaptable, allowing for straightforward updates to its tools that LLMs can leverage within the workflow, as well as updates to the workflow itself using domain knowledge. This adaptability enables FactAgent's application to news verification across various domains.

Motivation & Objective

• To address the challenge of scalable and efficient fake news detection in the face of rapid misinformation spread on digital platforms.
• To overcome the limitations of supervised models that require costly, human-annotated datasets for training.
• To enhance interpretability and transparency in fake news detection by providing explicit reasoning at each step of the verification process.
• To enable flexible adaptation of the detection system across diverse domains by updating tools and workflows using domain expertise.
• To demonstrate that an LLM-based agentic system can outperform non-agentic prompting and existing fact-checking methods without fine-tuning.

Proposed method

• FactAgent employs a structured, multi-step workflow that emulates human expert fact-checking behavior, breaking down news verification into manageable sub-tasks.
• Each sub-step uses the LLM’s internal knowledge or external tools (e.g., search engines, knowledge graphs) to retrieve relevant evidence.
• The LLM compares findings from each tool against a domain-specific checklist to ensure consistency and completeness in reasoning.
• Final decision-making is based on the LLM’s synthesis of all observations, not a rigid majority vote, allowing for nuanced judgment.
• The workflow is designed using expert domain knowledge and can be updated easily to incorporate new tools or verification criteria.
• The system operates in a zero-shot manner, requiring no model fine-tuning or labeled data for training.

Experimental results

Research questions

• RQ1 Can an LLM-based agentic system outperform non-agentic prompting and supervised models in fake news detection without any training?
• RQ2 How does the inclusion of expert-designed workflows and external tools impact the accuracy and interpretability of LLM-based fact-checking?
• RQ3 Does allowing the LLM to reason over multiple evidence sources and a checklist lead to better performance than relying on a majority vote strategy?
• RQ4 To what extent can FactAgent be adapted across different news domains through workflow and tool updates?
• RQ5 How does the transparency of step-by-step reasoning in FactAgent improve user trust and understanding compared to black-box models?

Key findings

• FactAgent with an expert-designed workflow outperforms supervised learning models, standard prompting, chain-of-thought prompting, and single-aspect analysis in fake news detection.
• FactAgent achieves higher performance than HiSS, a prior LLM-based fact-checking system using external search, by integrating structured reasoning and domain-specific checklists.
• Using a majority vote strategy for final decisions led to inferior performance compared to instructing the LLM to compare findings with a checklist, indicating that flexible reasoning yields better outcomes.
• The system provides explicit, natural language explanations at each step, enhancing interpretability and user understanding of the verification process.
• FactAgent’s performance is highly dependent on the quality of the expert-designed workflow, highlighting the importance of domain knowledge in system design.
• The framework is highly adaptable, allowing seamless integration of new tools and workflow modifications without retraining.