[Paper Review] Fact-checking information from large language models can decrease headline discernment

Matthew DeVerna, Harry Yaojun Yan+2 more|arXiv (Cornell University)|Aug 21, 2023

Misinformation and Its Impacts4 citations

TL;DR

This study investigates the impact of fact-checking information generated by a large language model (LLM) on users' ability to discern political news accuracy and intentions to share it. Despite the LLM correctly debunking false headlines, it reduces belief in true headlines mislabeled as false and increases belief in false headlines when uncertain, highlighting unintended harms in AI-driven fact-checking systems.

ABSTRACT

Fact checking can be an effective strategy against misinformation, but its implementation at scale is impeded by the overwhelming volume of information online. Recent artificial intelligence (AI) language models have shown impressive ability in fact-checking tasks, but how humans interact with fact-checking information provided by these models is unclear. Here, we investigate the impact of fact-checking information generated by a popular large language model (LLM) on belief in, and sharing intent of, political news headlines in a preregistered randomized control experiment. Although the LLM accurately identifies most false headlines (90%), we find that this information does not significantly improve participants' ability to discern headline accuracy or share accurate news. In contrast, viewing human-generated fact checks enhances discernment in both cases. Subsequent analysis reveals that the AI fact-checker is harmful in specific cases: it decreases beliefs in true headlines that it mislabels as false and increases beliefs in false headlines that it is unsure about. On the positive side, AI fact-checking information increases the sharing intent for correctly labeled true headlines. When participants are given the option to view LLM fact checks and choose to do so, they are significantly more likely to share both true and false news but only more likely to believe false headlines. Our findings highlight an important source of potential harm stemming from AI applications and underscore the critical need for policies to prevent or mitigate such unintended consequences.

Motivation & Objective

• To examine how users respond to fact-checking information generated by a large language model (LLM) in the context of political news.
• To assess whether LLM fact-checking improves users' ability to discern accurate from false headlines.
• To evaluate the effect of LLM fact-checking on participants' intention to share political news.
• To investigate the role of partisan congruency in shaping beliefs and sharing behaviors when exposed to LLM-generated fact checks.
• To identify unintended consequences of deploying LLMs for automated fact-checking in real-world information ecosystems.

Proposed method

• A preregistered, randomized controlled experiment was conducted with N=1,548 U.S. participants to assess causal effects of LLM fact-checking on news perception.
• Participants were exposed to 40 real political news stories—half true, half false—balanced for partisan alignment (Democratic or Republican favorability).
• Participants were assigned to 'belief' and 'sharing' groups and randomly assigned to either view or opt out of fact-checking information generated by ChatGPT.
• Statistical modeling, including multilevel regression with robust standard errors, was used to analyze the effects of fact-checking exposure, headline veracity, and partisan congruency on belief and sharing intentions.
• Interaction effects between fact-checking opt-in status, headline truthfulness, and partisan congruency were tested to assess differential impacts.
• Post-hoc comparisons using Bonferroni correction were conducted to evaluate differences in belief and sharing intent slopes across conditions.

Figure 1: Experimental design, accuracy, and main effects of the LLM fact-checking intervention. (a) Graphical representation of the experimental design and participant flow. Although two different false claims are shown as examples along with their respective ChatGPT fact-checking information, both

Experimental results

Research questions

• RQ1 Does exposure to LLM-generated fact-checking improve users' ability to discern the accuracy of political news headlines?
• RQ2 How does viewing LLM fact-checking affect participants' intention to share true versus false political news?
• RQ3 Does the LLM’s uncertainty or mislabeling of headlines lead to reduced trust in accurate news or increased belief in false news?
• RQ4 How does partisan congruency moderate the effects of LLM fact-checking on belief and sharing intentions?
• RQ5 What are the unintended consequences of deploying LLMs for automated fact-checking in public information ecosystems?

Key findings

• The LLM correctly debunked false headlines, but its fact-checking did not significantly improve participants’ ability to discern headline accuracy.
• The LLM decreased belief in true headlines that were incorrectly labeled as false, indicating a harmful effect on trust in accurate information.
• The LLM increased belief in false headlines when it was uncertain about their veracity, suggesting a risk of amplifying misinformation.
• When participants chose to view LLM fact-checking, they were more likely to share both true and false news, but only more likely to believe false headlines.
• Participants who opted out of viewing fact-checking showed a significant negative effect of partisan incongruency on sharing intentions for true headlines, indicating that opt-out users were more sensitive to partisanship.
• The study found no significant effect of partisan congruency on sharing intentions when participants opted in to view fact-checking, suggesting that LLM fact-checking may override partisan biases in sharing behavior.

Figure 2: Effects of LLM fact-checking information on headline belief and sharing intent, contingent on headline veracity and fact check judgment. Each panel shows the proportion of participants in the control (circles) and forced (triangles) conditions who (a) believed or (b) were willing to share