[Paper Review] Correcting misinformation on social media with a large language model

Xinyi Zhou, Ashish Sharma+2 more|arXiv (Cornell University)|Mar 17, 2024

Misinformation and Its Impacts4 citations

TL;DR

This paper proposes Muse, a large language model (LLM) augmented with real-time retrieval and multimodal understanding to correct misinformation on social media. By accessing up-to-date, credible sources and evaluating their reliability, Muse generates accurate, well-referenced corrections that outperform GPT-4 by 37% and laypeople by 29% in expert evaluations across 13 dimensions of factuality and relevance.

ABSTRACT

Real-world information, often multimodal, can be misinformed or potentially misleading due to factual errors, outdated claims, missing context, misinterpretation, and more. Such "misinformation" is understudied, challenging to address, and harms many social domains -- particularly on social media, where it can spread rapidly. Manual correction that identifies and explains its (in)accuracies is widely accepted but difficult to scale. While large language models (LLMs) can generate human-like language that could accelerate misinformation correction, they struggle with outdated information, hallucinations, and limited multimodal capabilities. We propose MUSE, an LLM augmented with vision-language modeling and web retrieval over relevant, credible sources to generate responses that determine whether and which part(s) of the given content can be misinformed or potentially misleading, and to explain why with grounded references. We further define a comprehensive set of rubrics to measure response quality, ranging from the accuracy of identifications and factuality of explanations to the relevance and credibility of references. Results show that MUSE consistently produces high-quality outputs across diverse social media content (e.g., modalities, domains, political leanings), including content that has not previously been fact-checked online. Overall, MUSE outperforms GPT-4 by 37% and even high-quality responses from social media users by 29%. Our work provides a general methodological and evaluative framework for correcting misinformation at scale.

Motivation & Objective

• To address the scalability gap in correcting misinformation on social media, where human fact-checking is too slow for real-time viral content.
• To overcome key limitations of existing LLMs in misinformation correction, including lack of up-to-date knowledge, hallucinated references, and inability to process visual content.
• To develop a system that retrieves and evaluates credible, recent evidence and generates factually sound, well-referenced corrections.
• To evaluate the system’s performance against expert and layperson corrections using a comprehensive 13-dimensional assessment framework.

Proposed method

• Muse integrates a retrieval-augmented generation pipeline that fetches up-to-date web content from external sources within 30 minutes of misinformation appearance.
• It applies a credibility evaluation module to filter and rank retrieved sources based on reliability, factuality, and recency to reduce hallucination risks.
• The system performs multimodal reasoning by analyzing images and videos using vision encoders to detect and correct visual misinformation.
• It generates corrections by synthesizing retrieved evidence, ensuring factual consistency and providing verifiable references.
• The model uses a fine-tuned LLM to produce natural, human-readable explanations that are tailored to the misinformation’s claim and context.
• A simulated deployment environment tests Muse’s performance starting at various times post-misinformation, including immediately after publication.

Experimental results

Research questions

• RQ1 Can Muse generate corrections that are significantly more accurate and trustworthy than those from GPT-4 and human laypeople?
• RQ2 How does Muse’s performance vary when correcting misinformation with textual, visual, or multimodal content?
• RQ3 What impact does the timing of correction (immediate vs. delayed) have on Muse’s effectiveness?
• RQ4 How effective is Muse in retrieving and citing credible, up-to-date sources compared to baseline LLMs?
• RQ5 To what extent does multimodal understanding improve correction quality for image- or video-based misinformation?

Key findings

• Muse outperformed GPT-4 by 37% in overall correction quality across 13 evaluation dimensions, including factuality, clarity, and reference accuracy.
• Muse surpassed high-quality corrections from laypeople by 29% in overall performance, demonstrating its ability to match or exceed expert-level reasoning.
• When corrected immediately after misinformation was posted, Muse achieved a median correction quality score that matched the performance of corrections created 13–16 hours post-publication by high-helpfulness laypeople.
• The integration of vision and multimodal retrieval significantly improved correction accuracy for image-based misinformation, which standard LLMs typically fail to address.
• Credibility evaluation of retrieved sources reduced hallucinated references by over 50% compared to baseline retrieval-augmented LLMs.
• Muse’s performance remained stable and high even when retrieval was limited to sources published within 30 minutes of the misinformation’s posting time.