[Paper Review] A Multi-dimensional Investigation of the Effects of Publication Retraction on Scholarly Impact

Xin Shuai, Isabelle Moulinier+4 more|arXiv (Cornell University)|Feb 29, 2016

Academic integrity and plagiarism16 references5 citations

TL;DR

This study investigates the scholarly impact of retracted papers using Web of Science data, comparing retracted and non-retracted articles, authors, and institutions. It finds that retracted papers and their authors suffer significant, localized reputational damage—especially when misconduct is involved—while the negative effects do not propagate to institutions, co-authors, or related research topics.

ABSTRACT

Over the past few decades, the rate of publication retractions has increased dramatically in academia. In this study, we investigate retractions from a quantitative perspective, aiming to answer two fundamental questions. One, how do retractions influence the scholarly impact of retracted papers, authors, and institutions? Two, does this influence propagate to the wider academic community through scholarly associations? Specifically, we analyzed a set of retracted articles indexed in Thomson Reuters Web of Science (WoS), and ran multiple experiments to compare changes in scholarly impact against a control set of non-retracted articles, authors, and institutions. We further applied the Granger Causality test to investigate whether different scientific topics are dynamically affected by retracted papers occurring within those topics. Our results show two key findings: first, the scholarly impact of retracted papers and authors significantly decreases after retraction, and the most severe impact decrease correlates to retractions based on proven purposeful scientific misconduct; second, this retraction penalty does not seem to spread through the broader scholarly social graph, but instead has a limited and localized effect. Our findings may provide useful insights for scholars or science committees to evaluate the scholarly value of papers, authors, or institutions related to retractions.

Motivation & Objective

• To understand how publication retractions affect the scholarly impact of retracted papers, their authors, and their institutions.
• To investigate whether the negative impact of retractions spreads through scholarly networks, including co-authorship and topic-based associations.
• To assess whether retraction rates influence the future popularity of scientific research topics.
• To develop a systematic framework for categorizing retractions by reason and measuring their downstream effects.
• To evaluate the long-term consequences of retractions on citation behavior and academic reputation beyond the directly involved parties.

Proposed method

• Constructed a dataset of retracted articles from Thomson Reuters Web of Science, categorized by retraction reason using a custom coding schema.
• Performed comparative statistical experiments between retracted articles and non-retracted control articles published in the same journal and on the same date.
• Tracked changes in citation counts and scholarly reputation for retracted authors and their institutions pre- and post-retraction.
• Applied Granger Causality testing to examine temporal relationships between retraction rates and the popularity of scientific topics.
• Analyzed co-authorship and topic-based citation networks to detect propagation of retraction effects to related scholars and papers.
• Used longitudinal citation data to measure reputation decay and recovery patterns for retracted authors and institutions.

Experimental results

Research questions

• RQ1 How does retraction affect the citation impact of retracted papers compared to non-retracted controls?
• RQ2 What is the effect of retraction on the scholarly reputation of first authors, especially those involved in scientific misconduct?
• RQ3 Does the negative impact of retraction propagate to sponsoring institutions, co-authors, or topic-related papers?
• RQ4 Is there a causal relationship between retraction rates and the future popularity of specific scientific research topics?
• RQ5 How do different retraction reasons (e.g., falsification, plagiarism, error) differentially affect scholarly impact?

Key findings

• Retracted papers experience a significant and sustained decrease in citations after retraction, with the most severe decline observed in cases of proven scientific misconduct.
• First authors of retracted papers—particularly those guilty of data fabrication or falsification—face a sharp and lasting drop in scholarly reputation, as measured by citation trends.
• Institutions sponsoring retracted authors show no measurable reputational damage, indicating that institutional impact is not systematically penalized by retraction.
• No propagation effect was found in co-author networks or topic-based citation networks, suggesting that the negative impact of retraction is highly localized.
• Granger Causality analysis revealed no significant dynamic relationship between retraction rates and the future popularity of scientific topics.
• Citing retracted papers does not appear to lead to academic consequences for citing authors, suggesting contextual factors in citation behavior may mitigate reputational risk.