[Paper Review] Online Engagement with Retracted Articles: Who, When, and How?

Henry K. Dambanemuya, Rod Abhari+2 more|arXiv (Cornell University)|Mar 8, 2022

Misinformation and Its Impacts5 citations

TL;DR

This study analyzes Twitter engagement with over 3,800 retracted English-language articles using user type classification and engagement metrics, revealing that retracted articles receive significantly higher attention and engagement—especially from the public and bots—primarily before retraction. The findings highlight non-experts' central role in spreading retracted research online and suggest social media platforms can aid early detection of problematic science.

ABSTRACT

Retracted research discussed on social media can spread misinformation. Yet we lack an understanding of how retracted articles are mentioned by academic and non-academic users. This is especially relevant on Twitter due to the platform's prominent role in science communication. Here, we analyze the pre- and post-retraction differences in Twitter attention and engagement metrics for over 3,800 retracted English-language articles alongside comparable non-retracted articles. We subset these findings according to five user types detected by our supervised learning classifier: members of the public, academics, bots, science practitioners, and science communicators. We find that retracted articles receive greater user attention (tweet count) and engagement (likes, retweets, and replies) than non-retracted articles, especially among members of the public and bots, with the majority of user engagement happening before retraction. Our results highlight the prominent role of non-experts in discussions of retracted research and suggest an opportunity for social media platforms to contribute towards early detection of problematic scientific research online.

Motivation & Objective

• To understand how different user types on Twitter engage with retracted scientific articles compared to non-retracted ones.
• To investigate whether retracted articles receive more attention and engagement than non-retracted articles, and how this varies across user types.
• To examine the timing of engagement relative to retraction events, particularly before and after retraction.
• To assess how keyword-related content (e.g., reasons for retraction) influences engagement patterns on Twitter.
• To inform social media platform design by identifying opportunities to support early detection of flawed scientific research through user engagement patterns.

Proposed method

• Utilized the Retraction Watch database to identify 3,800 retracted English-language articles and matched non-retracted articles from the same journals.
• Leveraged Altmetric and Twitter’s Research API to collect social media engagement metrics (tweets, likes, retweets, replies) for each article.
• Applied a supervised machine learning classifier to categorize Twitter users into five types: academics, science communicators, science practitioners, bots, and members of the public using profile descriptions.
• Conducted time-series analysis to compare pre- and post-retraction engagement levels across user types.
• Used TF-IDF and keyword-based analysis to examine how retraction-related content (e.g., 'fraud', 'data issues') correlates with engagement levels.
• Controlled for article promotion by matching non-retracted articles based on tweet volume to ensure comparable visibility.

Experimental results

Research questions

• RQ1 Does attention to and engagement with retracted articles on Twitter differ from non-retracted articles?
• RQ2 How does attention and engagement with (non)-retracted articles vary across different user types on Twitter?
• RQ3 How does engagement with (non)-retracted articles differ across user types before and after retraction?
• RQ4 How does engagement on Twitter vary with different keyword-related content (e.g., reasons for retraction) associated with retracted articles?

Key findings

• Retracted articles received significantly higher user attention (measured by tweet count) and engagement (likes, retweets, replies) than non-retracted articles.
• The majority of engagement with retracted articles occurred before retraction, indicating that attention peaks during the pre-retraction phase.
• Members of the public and bots exhibited the highest levels of engagement with retracted articles, particularly in terms of tweet volume and retweets.
• Academics and science communicators showed lower engagement levels compared to non-academic users, despite their potential role in correcting misinformation.
• Engagement with retracted articles was strongly associated with keywords related to retraction reasons (e.g., 'fraud', 'data issues'), suggesting that retraction-related content drives visibility.
• The study found that retracted articles like the Lancet’s hydroxychloroquine study attracted over 29,000 mentions on Twitter, demonstrating the scale of post-publication discussion even after retraction.