[Paper Review] TexTrolls: Identifying Russian Trolls on Twitter from a Textual Perspective

Bilal Ghanem, Davide Buscaldi+1 more|arXiv (Cornell University)|Oct 3, 2019

Misinformation and Its Impacts11 citations

TL;DR

This paper proposes a text-based machine learning model to detect Russian Internet Research Agency (IRA) trolls on Twitter using thematic and profiling features derived from tweet content. By applying unsupervised topic modeling and Natural Language Inference (NLI) features, the model achieves a macro F1 score of 0.94, demonstrating that linguistic style and thematic patterns are highly effective for identifying coordinated disinformation campaigns from textual cues alone.

ABSTRACT

The online new emerging suspicious users, that usually are called trolls, are one of the main sources of hate, fake, and deceptive online messages. Some agendas are utilizing these harmful users to spread incitement tweets, and as a consequence, the audience get deceived. The challenge in detecting such accounts is that they conceal their identities which make them disguised in social media, adding more difficulty to identify them using just their social network information. Therefore, in this paper, we propose a text-based approach to detect the online trolls such as those that were discovered during the US 2016 presidential elections. Our approach is mainly based on textual features which utilize thematic information, and profiling features to identify the accounts from their way of writing tweets. We deduced the thematic information in a unsupervised way and we show that coupling them with the textual features enhanced the performance of the proposed model. In addition, we find that the proposed profiling features perform the best comparing to the textual features.

Motivation & Objective

• To detect Russian trolls on Twitter using only textual features, without relying on social network metadata.
• To investigate whether thematic information extracted via unsupervised topic modeling improves detection performance.
• To analyze how IRA trolls use emotional language and linguistic styles to manipulate public opinion.
• To evaluate the effectiveness of profiling features—such as sentiment, assertion, and irony markers—in distinguishing troll accounts from legitimate users.

Proposed method

• The authors apply unsupervised LDA-based topic modeling to extract thematic features from tweets, capturing underlying ideological themes.
• They extract a comprehensive set of profiling features, including sentiment (e.g., anger, fear), assertion strength, and linguistic markers (e.g., hedging, certainty, negation).
• Natural Language Inference (NLI) features are used to model rhetorical and logical structures in tweets, such as contradiction and entailment, to detect manipulative writing patterns.
• A machine learning classifier is trained on a combination of thematic and profiling features, using a dataset of 2,023 IRA troll accounts and 94,643 legitimate accounts.
• Cross-validation is used to evaluate performance, with macro F1 as the primary metric to assess balanced detection across both classes.
• False positive cases are manually analyzed to identify common failure modes, such as link-only posts, vague questions, and content from deleted links.

Experimental results

Research questions

• RQ1 RQ1: Does the inclusion of thematic information extracted via unsupervised topic modeling improve the detection performance of IRA trolls?
• RQ2 RQ2: Can IRA trolls be effectively identified using only textual features, without relying on network or metadata signals?
• RQ3 RQ3: How do IRA trolls utilize emotional language and linguistic strategies to influence public opinion?

Key findings

• The proposed model achieved a macro F1 score of 0.94 in cross-validation, demonstrating high effectiveness in distinguishing IRA trolls from legitimate users.
• Thematic features extracted via unsupervised topic modeling significantly boosted model performance when combined with surface-level textual features.
• Profiling features—particularly those related to sentiment, assertion, and certainty—performed better than thematic features alone in identifying troll accounts.
• The model showed strong performance in detecting irony and manipulative rhetorical strategies, which were frequently used by IRA trolls.
• False positives were primarily caused by link-only tweets, vague questions, and external links to deleted content, suggesting the need for link-content analysis in future detection pipelines.
• Manual analysis revealed that irony was frequently employed by trolls, indicating that irony detection could be a valuable addition to future detection systems.