[Paper Review] Fake News Detectors are Biased against Texts Generated by Large Language Models

Jinyan Su, Terry Yue Zhuo+3 more|arXiv (Cornell University)|Sep 15, 2023

Misinformation and Its ImpactsSocial Sciences12 citations

TL;DR

This paper reveals that fake news detectors are biased toward classifying LLM-generated content as fake and proposes debiasing via adversarial training with LLM-paraphrased real news, backed by new GossipCop++ and PolitiFact++ datasets.

ABSTRACT

The spread of fake news has emerged as a critical challenge, undermining trust and posing threats to society. In the era of Large Language Models (LLMs), the capability to generate believable fake content has intensified these concerns. In this study, we present a novel paradigm to evaluate fake news detectors in scenarios involving both human-written and LLM-generated misinformation. Intriguingly, our findings reveal a significant bias in many existing detectors: they are more prone to flagging LLM-generated content as fake news while often misclassifying human-written fake news as genuine. This unexpected bias appears to arise from distinct linguistic patterns inherent to LLM outputs. To address this, we introduce a mitigation strategy that leverages adversarial training with LLM-paraphrased genuine news. The resulting model yielded marked improvements in detection accuracy for both human and LLM-generated news. To further catalyze research in this domain, we release two comprehensive datasets, exttt{GossipCop++} and exttt{PolitiFact++}, thus amalgamating human-validated articles with LLM-generated fake and real news.

Motivation & Objective

• Motivate realistic evaluation of fake news detectors on both human-written and LLM-generated misinformation.
• Characterize biases of detectors toward LLM-generated content.
• Investigate linguistic features (NELA) that may underlie detected biases.
• Develop debiasing methods using adversarial training with LLM-paraphrased real news.
• Provide new datasets (GossipCop++ and PolitiFact++) for benchmark and replication.

Proposed method

• Define Pre-LLM and LLM-era task formulations for fake news detection.
• Create PolitiFact++ and GossipCop++ datasets by generating LLM-synthesized content via Structured Mimicry Prompting (SMP) with ChatGPT.
• Evaluate multiple detectors (RoBERTa, BERT, ELECTRA, ALBERT, DeBERTa) on HR, HF, and MF subsets.
• Analyze detector behavior with NELA-content features and statistical tests (e.g., Tukey pairwise) to identify bias sources.
• Propose an adversarial training-based debiasing strategy using LLM-paraphrased real news.
• Assess debiasing impact through performance shifts across human- and LLM-generated content.

Figure 1: SMP : Prompting LLMs to generate fake news articles.

Experimental results

Research questions

• RQ1 RQ1: How well do fake news detectors perform on PolitiFact++ and GossipCop++ across human-written real, human-written fake, and LLM-generated fake content?
• RQ2 RQ2: Why do detectors show bias toward LLM-generated news, and what features correlate with this bias?
• RQ3 RQ3: Can bias be mitigated, and how does adversarial training with LLM-paraphrased real news affect detection across content types?

Key findings

• Detectors are biased toward LLM-generated content, often misclassifying truthful LLM outputs as fake.
• NELA-content features show significant differences between human-written fake and LLM-generated fake news, suggesting detectors may rely on language patterns unique to LLM outputs.
• Debiasing with regression-based models alters performance, improving detection for human-written fake news but sometimes reducing it for LLM-generated cases.
• Adversarial training using LLM-paraphrased real news improves generalization, enhancing detection for both human-written and LLM-generated content across datasets.
• Two enhanced datasets, GossipCop++ and PolitiFact++, pair original human-written articles with LLM-synthesized content to benchmark detectors.