[Paper Review] Online User Profiling to Detect Social Bots on Twitter

Maryam Heidari, James H. Jones+1 more|arXiv (Cornell University)|Mar 9, 2022

Misinformation and Its Impacts18 citations

TL;DR

This paper proposes a novel machine learning approach to detect social bots on Twitter by constructing online user profiles from users' textual posts, extracting personal attributes such as age, gender, education, and personality. By leveraging similarities in personal information across posts as a signal, the model achieves high accuracy in distinguishing bots from humans, introducing a new public dataset of over 6,900 Twitter accounts with profile annotations from the Cresci 2017 dataset.

ABSTRACT

Social media platforms can expose influential trends in many aspects of everyday life. However, the movements they represent can be contaminated by disinformation. Social bots are one of the significant sources of disinformation in social media. Social bots can pose serious cyber threats to society and public opinion. This research aims to develop machine learning models to detect bots based on the extracted user's profile from a Tweet's text. Online users' profile shows the user's personal information, such as age, gender, education, and personality. In this work, the user's profile is constructed based on the user's online posts. This work's main contribution is three-fold: First, we aim to improve bot detection through machine learning models based on the user's personal information generated by the user's online comments. When comparing two online posts, the similarity of personal information makes it difficult to differentiate a bot from a human user. However, this research turns personal information similarity among two online posts into an advantage for the new bot detection model. The new proposed model for bot detection creates user profiles based on personal information such as age, personality, gender, education from users' online posts and introduces a machine learning model to detect social bots with high prediction accuracy based on personal information. Second, create a new public data set that shows the user's profile for more than 6900 Twitter accounts in the Cresci 2017 data set.

Motivation & Objective

• To improve social bot detection by modeling user profiles based on personal information derived from online posts.
• To address the challenge of distinguishing bots from humans when personal information is highly similar across accounts.
• To create a new public dataset annotating user profiles for over 6,900 Twitter accounts from the Cresci 2017 benchmark.
• To develop a machine learning model that leverages profile similarity as a discriminative signal for bot detection.
• To demonstrate the effectiveness of profile-based features in enhancing bot detection accuracy.

Proposed method

• User profiles are constructed by extracting personal attributes—such as age, gender, education, and personality—from users' Twitter posts using natural language processing techniques.
• The model treats similarity in personal information across multiple posts as a signal to identify bot-like behavior, assuming bots often replicate or mimic such attributes.
• A supervised machine learning classifier is trained on profile features to distinguish between human and bot accounts.
• The approach uses the Cresci 2017 dataset as the foundation, with profile annotations generated via NLP pipelines on textual content.
• The method incorporates both linguistic and demographic features to represent user profiles in a structured, learnable format.
• A new public dataset is released, containing profile annotations for 6,900 Twitter accounts, enabling reproducibility and benchmarking.

Experimental results

Research questions

• RQ1 Can personal information extracted from user posts improve the accuracy of social bot detection?
• RQ2 How does the similarity of personal attributes across multiple posts serve as a distinguishing signal between bots and humans?
• RQ3 To what extent can profile-based features enhance bot detection performance compared to traditional methods?
• RQ4 What is the feasibility and reliability of constructing user profiles from unstructured Twitter text?
• RQ5 How does the proposed model compare to existing bot detection approaches using profile-level features?

Key findings

• The proposed model achieves high prediction accuracy in detecting social bots by leveraging user profile features derived from online posts.
• Profile similarity across posts is shown to be a strong indicator for identifying bot accounts, especially when combined with demographic and personality traits.
• The study introduces a new public dataset of over 6,900 Twitter accounts with annotated user profiles, enhancing reproducibility and future research.
• The integration of personal attributes such as age, gender, education, and personality into the model significantly improves detection performance.
• The method outperforms baseline approaches by treating profile consistency as a discriminative feature rather than noise.
• The results demonstrate that even when bots mimic human-like personal information, subtle inconsistencies or patterns in profile similarity can be exploited for detection.