[Paper Review] Categorising Fine-to-Coarse Grained Misinformation: An Empirical Study of COVID-19 Infodemic

Ye Jiang, Xingyi Song+3 more|arXiv (Cornell University)|Jun 22, 2021

Misinformation and Its Impacts8 citations

TL;DR

This paper introduces a fine-grained, manually annotated dataset of 1,722 COVID-19 misinformation tweets with eight categories, including related debunking, comments, and questions, enabling both misinformation detection and social behavior analysis. The study reveals that misinformation and debunking tweets spread at similar rates, and that social behaviors like questions and comments strongly correlate with misinformation; however, state-of-the-art NLP models show significantly degraded performance under 'leave claim out' validation, indicating poor generalization to unseen misinformation.

ABSTRACT

The spreading COVID-19 misinformation over social media already draws the attention of many researchers. According to Google Scholar, about 26000 COVID-19 related misinformation studies have been published to date. Most of these studies focusing on 1) detect and/or 2) analysing the characteristics of COVID-19 related misinformation. However, the study of the social behaviours related to misinformation is often neglected. In this paper, we introduce a fine-grained annotated misinformation tweets dataset including social behaviours annotation (e.g. comment or question to the misinformation). The dataset not only allows social behaviours analysis but also suitable for both evidence-based or non-evidence-based misinformation classification task. In addition, we introduce leave claim out validation in our experiments and demonstrate the misinformation classification performance could be significantly different when applying to real-world unseen misinformation.

Motivation & Objective

• To address the gap in understanding social behaviors (e.g., questions, comments, debunking) associated with misinformation during the COVID-19 infodemic.
• To develop a fine-grained, manually annotated dataset of 1,722 tweets with eight categories to support both evidence-based and non-evidence-based misinformation classification.
• To investigate the correlation between misinformation spread and related social behaviors such as questions, comments, and debunking.
• To evaluate the robustness of state-of-the-art NLP models in classifying misinformation under realistic 'leave claim out' cross-validation settings.

Proposed method

• An information retrieval pipeline was used to extract tweets related to IFCN-fact-checked misinformation claims by using the claims as queries.
• A fine-grained annotation scheme was developed with eight categories: Related Debunk, Related Misinformation, Comment, Question, Relevant Other, Irrelevant, and two additional classes for context.
• Each tweet was triple-annotated by independent annotators, and inter-annotator agreement was measured using Krippendorff’s alpha (0.67, substantial agreement).
• Benchmark experiments were conducted using state-of-the-art NLP models (e.g., BERT_Pair) on three classification tasks: non-evidence-based, evidence-based, and fine-grained classification.
• A novel 'leave claim out' cross-validation strategy was applied to simulate real-world deployment, where unseen claims are excluded during training.
• Quantitative analysis was performed over a 10-month period to study the evolution and types of misinformation.

Experimental results

Research questions

• RQ1 How many social media posts are questioning or commenting about a misinformation claim, and do these behaviors correlate with the spread of misinformation?
• RQ2 Does the volume of debunking tweets correlate with the volume of misinformation tweets?
• RQ3 What are the distinct categories and characteristics of misinformation spreading on social media?
• RQ4 How do state-of-the-art NLP models perform under realistic 'leave claim out' validation compared to standard cross-validation?
• RQ5 To what extent do social behaviors like comments and questions co-occur with misinformation and debunking content?

Key findings

• Misinformation and debunking tweets exhibit similar spread rates, indicating that fact-checking efforts may not be sufficient to counter misinformation at scale.
• Questions and comments about misinformation show a strong positive correlation with both misinformation and debunking tweets, suggesting active public engagement with false claims.
• A significant number of misinformation tweets contain URLs from high-credibility sources, indicating potential manipulation or misrepresentation of reliable information.
• The 'leave claim out' validation setting caused a dramatic drop in F1 scores—especially for the 'Comment' class (F1 < 0.25)—revealing that models overfit to training claims and fail to generalize to unseen misinformation.
• The 'Misinformation' class maintained relatively stable performance (F1 ~0.43) under both settings, likely due to its high sample size, while other classes like 'Question' and 'Comment' suffered from poor generalization.
• Confusion matrices revealed that models frequently misclassified 'Related Debunk' as 'Debunk' and 'Comment' as 'Misinformation', due to semantic similarity and inclusion of misinformation within comment texts.