[Paper Review] Detecting Propagators of Disinformation on Twitter Using Quantitative Discursive Analysis

Mark M. Bailey|arXiv (Cornell University)|Oct 11, 2022

Misinformation and Its Impacts4 citations

TL;DR

This paper proposes a quantitative discursive analysis method to detect Russian disinformation bots on Twitter during the 2016 U.S. election using centering resonance analysis and Clauset-Newman-Moore community detection. It achieves a Matthews Correlation Coefficient (MCC) of 0.9070 in classifying bots with high sensitivity, demonstrating strong discursive similarity among propagators, though it struggles to identify non-bots due to discursive dissimilarity in the control set.

ABSTRACT

Efforts by foreign actors to influence public opinion have gained considerable attention because of their potential to impact democratic elections. Thus, the ability to identify and counter sources of disinformation is increasingly becoming a top priority for government entities in order to protect the integrity of democratic processes. This study presents a method of identifying Russian disinformation bots on Twitter using centering resonance analysis and Clauset-Newman-Moore community detection. The data reflect a significant degree of discursive dissimilarity between known Russian disinformation bots and a control set of Twitter users during the timeframe of the 2016 U.S. Presidential Election. The data also demonstrate statistically significant classification capabilities (MCC = 0.9070) based on community clustering. The prediction algorithm is very effective at identifying true positives (bots), but is not able to resolve true negatives (non-bots) because of the lack of discursive similarity between control users. This leads to a highly sensitive means of identifying propagators of disinformation with a high degree of discursive similarity on Twitter, with implications for limiting the spread of disinformation that could impact democratic processes.

Motivation & Objective

• To identify propagators of disinformation on Twitter during the 2016 U.S. Presidential Election.
• To address the challenge of distinguishing disinformation bots from legitimate users using discursive patterns.
• To develop a method that leverages linguistic and network structures to detect coordinated disinformation campaigns.
• To evaluate the effectiveness of quantitative discursive analysis in classifying bot behavior with high sensitivity.

Proposed method

• Centering resonance analysis is applied to model discourse coherence and focus in user-generated content.
• Clauset-Newman-Moore community detection is used to identify clusters of users with similar discursive patterns.
• Discursive similarity between known Russian disinformation bots and a control set of non-bot users is quantified.
• The method compares discursive trajectories across users to detect communities of high similarity indicative of coordinated bot activity.
• A classification model is trained on community clusters to predict bot status based on discursive structure.
• The approach relies on statistical analysis of linguistic coherence and network clustering to isolate propagators of disinformation.

Experimental results

Research questions

• RQ1 Can discursive similarity patterns effectively distinguish Russian disinformation bots from non-bot users on Twitter during the 2016 U.S. election?
• RQ2 To what extent do known disinformation bots exhibit statistically significant discursive coherence compared to a control set of users?
• RQ3 How effective is community clustering based on discursive patterns in classifying bot accounts?
• RQ4 Why does the model achieve high sensitivity for true positives but fail to resolve true negatives in the control set?
• RQ5 What implications do these findings have for detecting and mitigating coordinated disinformation campaigns?

Key findings

• The method achieved a Matthews Correlation Coefficient (MCC) of 0.9070, indicating strong classification performance.
• Significant discursive dissimilarity was observed between known Russian disinformation bots and the control set of non-bot users.
• The model demonstrated high sensitivity in identifying true positives (i.e., actual bots) due to strong discursive similarity within bot clusters.
• The inability to resolve true negatives stems from the lack of discursive similarity among control users, which limits negative classification.
• The results suggest that discursive coherence in coordinated bot networks enables highly effective detection of propagators of disinformation.
• The study confirms that quantitative discursive analysis can serve as a powerful tool for identifying disinformation spreaders with minimal false negatives in high-similarity clusters.