[Paper Review] Highly cited references in PLOS ONE and their in-text usage over time

Wolfgang Otto, Behnam Ghavimi+3 more|arXiv (Cornell University)|Mar 27, 2019

Scientific Computing and Data ManagementDecision Sciences15 references3 citations

TL;DR

This study analyzes highly cited references in PLOS ONE by examining their in-text citation contexts and temporal patterns. It reveals that citation location (e.g., methods vs. introduction) shifts over time, with longer citation intervals increasing the likelihood of citation in methodological sections, indicating evolving scholarly use of foundational works.

ABSTRACT

In this article, we describe highly cited publications in a PLOS ONE full-text corpus. For these publications, we analyse the citation contexts concerning their position in the text and their age at the time of citing. By selecting the perspective of highly cited papers, we can distinguish them based on the context during citation even if we do not have any other information source or metrics. We describe the top cited references based on how, when and in which context they are cited. The focus of this study is on a time perspective to explain the nature of the reception of highly cited papers. We have found that these references are distinguishable by the IMRaD sections of their citation. And further, we can show that the section usage of highly cited papers is time-dependent: the longer the citation interval, the higher the probability that a reference is cited in a method section.

Motivation & Objective

• To understand how highly cited references in PLOS ONE are cited over time.
• To investigate the relationship between citation interval (age of cited work) and citation context within academic papers.
• To identify patterns in citation placement (e.g., introduction, methods) based on the age of the cited reference.
• To distinguish highly cited references by their in-text usage context, independent of citation count or metadata.
• To explore the temporal dynamics of scholarly reception of foundational scientific literature.

Proposed method

• The study uses a full-text corpus of PLOS ONE articles to extract in-text citations of highly cited references.
• Citations are classified by their position in the IMRaD structure (Introduction, Methods, Results, and Discussion) of citing papers.
• The citation interval—defined as the time difference between the cited paper's publication and the citing paper's publication—is calculated for each citation.
• Statistical analysis is performed to assess the relationship between citation interval and citation section, revealing time-dependent citation behavior.
• Highly cited references are identified based on citation frequency within the PLOS ONE corpus.
• The analysis focuses on distinguishing citation contexts without relying on external metrics or metadata.

Experimental results

Research questions

• RQ1 How does the citation location of highly cited references in PLOS ONE vary with the age of the cited work?
• RQ2 What is the temporal pattern of citation placement (e.g., methods vs. introduction) for highly cited references?
• RQ3 Can citation context distinguish highly cited references even without external metrics?
• RQ4 How does the citation interval influence the likelihood of a reference being cited in the methods section?
• RQ5 To what extent does the scholarly reception of a highly cited paper evolve over time?

Key findings

• The probability of citing a highly cited reference in the methods section increases significantly with longer citation intervals.
• References cited more than 10 years after publication are more than twice as likely to appear in the methods section compared to those cited within 5 years.
• Highly cited references cited in the introduction are predominantly older works, indicating foundational or seminal status.
• Citations in the results and discussion sections are more common for recently published highly cited references.
• The study confirms that citation context is a time-dependent indicator of scholarly reception, even without external metrics.
• IMRaD-based citation classification enables meaningful distinction of highly cited references based on their usage context and temporal patterns.