[论文解读] Effects of Research Paper Promotion via ArXiv and X

Chhandak Bagchi, Eric Malmi+1 more|arXiv (Cornell University)|Jan 20, 2024

Scientific Computing and Data ManagementDecision Sciences被引用 3

一句话总结

本研究基于十年的计算机科学出版物大规模数据集，分析了早期在ArXiv发布预印本以及在X（前身为Twitter）上推广对引用次数的影响。研究发现，与延迟提交相比，早期提交ArXiv可使五年内引用数增加21.1±17.4次，修订可增加18.4±17.6次，X平台推广则可使引用数额外增加44.4±8次，凸显社交媒体和预印本共享是推动科学影响力的主要因素。

ABSTRACT

In the evolving landscape of scientific publishing, it is important to understand the drivers of high-impact research, to equip scientists with actionable strategies to enhance the reach of their work, and to understand trends in the use of modern scientific publishing tools to inform their further development. Here, we study trends in the use of early preprint publications and revisions on ArXiv and the use of X (formerly Twitter) for promotion of such papers in computer science and physics. We find that early submissions to ArXiv and promotion on X have soared in recent years. Estimating the effect that the use of each of these modern affordances has on the number of citations of scientific publications, we find that peer-reviewed conference papers in computer science that are submitted early to ArXiv gain on average $21.1 \pm 17.4$ more citations, revised on ArXiv gain $18.4 \pm 17.6$ more citations, and promoted on X gain $44.4 \pm 8$ more citations in the first 5 years from an initial publication. In contrast, journal articles in physics experience comparatively lower boosts in citation counts, with increases of $3.9 \pm 1.1$, $4.3 \pm 0.9$, and $6.9 \pm 3.5$ citations respectively for the same interventions. Our results show that promoting one's work on ArXiv or X has a large impact on the number of citations, as well as the number of influential citations computed by Semantic Scholar, and thereby on the career of researchers. These effects are present also for publications in physics, but they are relatively smaller. The larger relative effect sizes, effects of promotion accumulating over time, and elevated unpredictability of the number of citations in computer science than in physics suggest a greater role of world-of-mouth spreading in computer science than in physics.

研究动机与目标

• 理解计算机科学领域早期在ArXiv发布预印本及在X上推广的上升趋势。
• 量化ArXiv提交时间、修订次数以及X推广对引用次数的因果影响。
• 评估现代影响力指标（如Semantic Scholar的高影响力引用）是否同样受到社交媒体曝光的影响。
• 评估这些趋势对科学评估、同行评审及未来出版系统的影响。
• 探讨基于社交互动的引用指标是否能减少因病毒式推广带来的偏见。

提出的方法

• 本研究使用2013至2023年计算机科学出版物的大规模交叉引用数据集，将会议论文集与ArXiv预印本及X（Twitter）活动关联起来。
• 采用双重稳健估计方法，控制混杂变量，以隔离ArXiv提交时间、修订次数及X推广对引用次数的因果效应。
• 引用次数在论文发表后前五年内进行测量，使用Semantic Scholar的数据统计总引用数和高影响力引用数。
• 区分作者发布的推文与他人转发的推文，以评估对引用结果的差异化影响。
• 研究评估了社交媒体影响下现代影响力指标的波动性，并提出基于网络归一化的引用图以校正社交偏见。
• 根据语义相似度和作者关系，将引用分类为主题相关或非主题相关，以评估引用的相关性。

实验结果

研究问题

• RQ1 RQ1：在计算机科学领域，早期提交ArXiv、修订论文以及在X上推广是否正成为标准做法？
• RQ2 RQ2：在会议论文集发布前将论文提交至ArXiv，是否会影响其引用次数？
• RQ3 RQ3：在ArXiv上修订论文是否会影响其引用次数？
• RQ4 RQ4.1：在X上推广论文是否能带来更多的引用？
• RQ5 RQ5：新型的、基于社交互动的影响力指标是否能减少社交媒体对引用次数的影响？

主要发现

• 与延迟提交相比，早期提交ArXiv可使五年内引用数增加21.1±17.4次。
• 在ArXiv上修订论文可使前五年内平均额外获得18.4±17.6次引用。
• 在X上推广论文可使五年内引用数显著增加44.4±8次，表明社交媒体具有强大影响。
• 总引用数和Semantic Scholar的高影响力引用数均对ArXiv和X的推广高度敏感，表明社交影响正在扭曲传统影响力衡量方式。
• 作者发布的推文影响显著高于他人转发的推文，表明由作者主导的推广更具影响力。
• 现代指标如网络归一化引用数可能减少社交媒体偏见，但目前尚无任何指标能完全免疫于在古德哈特定律下的操纵。