[Paper Review] How Vulnerable Are Automatic Fake News Detection Methods to Adversarial Attacks?

Camille Koenders, Johannes Filla+2 more|arXiv (Cornell University)|Jul 16, 2021

Misinformation and Its Impacts26 references4 citations

TL;DR

This paper investigates the vulnerability of state-of-the-art fake news detection models to adversarial attacks using the TextAttack library. It demonstrates that 65.15% of fake news sentences can be automatically manipulated to evade detection by altering words or characters while preserving semantic meaning, with word-level attacks being significantly more effective than character-level ones.

ABSTRACT

As the spread of false information on the internet has increased dramatically in recent years, more and more attention is being paid to automated fake news detection. Some fake news detection methods are already quite successful. Nevertheless, there are still many vulnerabilities in the detection algorithms. The reason for this is that fake news publishers can structure and formulate their texts in such a way that a detection algorithm does not expose this text as fake news. This paper shows that it is possible to automatically attack state-of-the-art models that have been trained to detect Fake News, making these vulnerable. For this purpose, corresponding models were first trained based on a dataset. Then, using Text-Attack, an attempt was made to manipulate the trained models in such a way that previously correctly identified fake news was classified as true news. The results show that it is possible to automatically bypass Fake News detection mechanisms, leading to implications concerning existing policy initiatives.

Motivation & Objective

• To assess the vulnerability of automatic fake news detection models to adversarial attacks.
• To evaluate whether adversarial perturbations can successfully bypass trained fake news classifiers.
• To compare the robustness of different models—BERTweet, RoBERTa, and Flair—under various attack types.
• To analyze the effectiveness of word-level versus character-level attack recipes in manipulating detection outcomes.
• To inform policy decisions by highlighting risks of automated content moderation relying on AI classifiers.

Proposed method

• Trained three state-of-the-art fake news detection models: BERTweet, RoBERTa, and Flair on a dataset of fake news sentences.
• Applied TextAttack to generate adversarial examples by modifying input sentences through word and character-level perturbations.
• Used multiple attack recipes: TextFooler, IGAWang, PWWS, PSOZang, BAEGarg, DeepWordBug, TextBugger, Pruthi, and CheckList2020.
• Calculated success rate using the formula $ S_r = \frac{1}{a} \sum_{i=1}^{a} \frac{s_i}{s_i + f_i} $, where $ s_i $ is successful attacks and $ f_i $ is failed attacks per recipe.
• Excluded skipped instances (misclassified by models initially) from success rate calculations to focus on attacks targeting correctly labeled fake news.
• Evaluated model robustness by measuring how often perturbed sentences were reclassified as real news.

Experimental results

Research questions

• RQ1 How vulnerable are automatic fake news detection models to adversarial attacks using TextAttack?
• RQ2 Which attack type—word-level or character-level—yields higher success rates in evading detection?
• RQ3 How do different pre-trained models (BERTweet, RoBERTa, Flair) compare in their susceptibility to adversarial manipulation?
• RQ4 To what extent do adversarial perturbations preserve semantic meaning while evading detection?
• RQ5 What are the implications of these vulnerabilities for policy-driven content moderation strategies?

Key findings

• The overall success rate of adversarial attacks across all models was 65.15%, indicating that more than 60% of fake news sentences could be successfully reclassified as real.
• BERTweet showed the highest success rate at 72.45%, followed by RoBERTa at 68.22%, while Flair was the most robust with a success rate of 54.77%.
• Word-level attacks achieved an average success rate of 76.87%, significantly outperforming character-level and mixed attacks, which averaged 55.21%.
• TextFooler (81.17% success rate) and IGAWang (70%) were among the most effective attack recipes, while CheckList2020 and Pruthi performed poorly with 9.83% and 27.83% success rates, respectively.
• Flair’s use of contextual string embeddings made it more resilient to adversarial attacks compared to models relying on document-level embeddings.
• Despite high attack success, many adversarially modified sentences contained obvious errors (e.g., 'TrOmp hsut down American airportA on 4 Jul 201B'), suggesting human detection may still be effective.