[Paper Review] On Good and Bad Intentions behind Anomalous Citation Patterns among Journals in Computer Sciences

Joyita Chakraborty, Dinesh K. Pradhan+3 more|arXiv (Cornell University)|Jul 27, 2018

Complex Network Analysis Techniques3 citations

TL;DR

This paper investigates anomalous citation patterns—such as mutual citations, citation chains, triangles, and cartels—among 2,500+ computer science journals from 1990 to 2012 using a weighted directed graph model. It distinguishes between 'good' (legitimate) and 'bad' (manipulative) intentions behind these patterns, revealing that sudden impact factor inflation and citation surges are often linked to unethical practices like citation stacking and self-citation rings, challenging the reliability of current bibliometric indicators.

ABSTRACT

Scientific journals are an important choice of publication venue for most authors. Publishing in prestigious journal plays a decisive role for authors in hiring and promotions. In last decade, citation pressure has become intact for all scientific entities more than ever before. Unethical publication practices has started to manipulate widely used performance metric such as "impact factor" for journals and citation based indices for authors. This threatens the integrity of scientific quality and takes away deserved credit of legitimate authors and their authentic publications. In this paper we extract all possible anomalous citation patterns between journals from a Computer Science bibliographic dataset which contains more than 2,500 journals. Apart from excessive self-citations, we mostly focus on finding several patterns between two or more journals such as bi-directional mutual citations, chains, triangles, mesh, cartel relationships. On a macroscopic scale, the motivation is to understand the nature of these patterns by modeling how journals mutually interact through citations. On microscopic level, we differentiate between possible intentions (good or bad) behind such patterns. We see whether such patterns prevail for long period or during any specific time duration. For abnormal citation behavior, we study the nature of sudden inflation in impact factor of journals on a time basis which may occur due to addition of irrelevant and superfluous citations in such closed pattern interaction. We also study possible influences such as abrupt increase in paper count due to the presence of self-referential papers or duplicate manuscripts, author self-citation, author co-authorship network, author-editor network, publication houses etc. The entire study is done to question the reliability of existing bibliometrics, and hence, it is an urgent need to curtail their usage or redefine them.

Motivation & Objective

• To identify and classify anomalous citation patterns such as mutual citations, citation chains, triangles, meshes, and cartels in computer science journals.
• To differentiate between legitimate (good) and manipulative (bad) intentions behind these citation patterns.
• To investigate whether such patterns are time-specific or persistent over long periods.
• To analyze the impact of citation anomalies on journal impact factors and paper counts, especially sudden inflation due to superfluous citations.
• To question the reliability of existing bibliometric indicators like impact factor and advocate for their redefinition or restricted use.

Proposed method

• Constructed a weighted directed graph from a Microsoft Academic Search dataset of 2,500+ CS journals and 490,249 papers (1990–2012).
• Identified anomalous patterns using graph-based analysis: mutual citations, citation chains, triangles, meshes, self-loops, and citation cartels.
• Tracked temporal dynamics of impact factor inflation and citation surges to detect sudden anomalies.
• Analyzed contributing factors such as self-citations, duplicate manuscripts, author co-authorship networks, author-editor networks, and publisher influence.
• Used citation network topology to detect closed-loop citation behaviors indicative of citation cartels or stacking.
• Applied macroscopic (network-level) and microscopic (intent-based) analysis to assess legitimacy of citation patterns.

Experimental results

Research questions

• RQ1 Can we distinguish between legitimate and manipulative citation patterns using network-level analysis of citation graphs?
• RQ2 Are anomalous citation patterns like citation cartels or mutual citation rings persistent over time or confined to specific periods?
• RQ3 To what extent do sudden increases in impact factor correlate with superfluous or self-referential citations?
• RQ4 How do factors such as author self-citation, co-authorship networks, and publisher influence contribute to citation anomalies?
• RQ5 To what extent do current bibliometric indicators like impact factor remain reliable in the presence of such manipulative behaviors?

Key findings

• A 60% increase in publication rate and 74% rise in citation rate were observed from 1990 to 2012, amplifying risks of citation manipulation.
• Anomalous patterns such as citation cartels, mutual citations, and citation chains were prevalent, particularly in journals with sudden impact factor inflation.
• Sudden spikes in impact factors were strongly correlated with the presence of excessive one-way or reciprocal citations among a small group of journals.
• Self-citation and self-referential papers were significant contributors to inflated paper counts and citation surges, especially in journals with low external citation diversity.
• Author co-authorship and author-editor networks were found to facilitate coordinated citation behavior, suggesting institutional or collaborative collusion.
• The study concludes that current bibliometric indicators like impact factor are vulnerable to manipulation and calls for urgent re-evaluation or redesign of such metrics.