Skip to main content
Nubint
QUICK REVIEW

[论文解读] Dataset Artefacts are the Hidden Drivers of the Declining Disruptiveness in Science

Vincent Holst, Andres Algaba|arXiv (Cornell University)|Feb 7, 2024
Artificial Intelligence in Healthcare and Education被引用 8
一句话总结

作者表明随时间推移报道的科学与技术的颠覆性下降，是由零引用的异常项（CD5=1）驱动的；一旦排除这些伪迹或对其进行适当控制，下降基本消失。

ABSTRACT

Park et al. [1] reported a decline in the disruptiveness of scientific and technological knowledge over time. Their main finding is based on the computation of CD indices, a measure of disruption in citation networks [2], across almost 45 million papers and 3.9 million patents. Due to a factual plotting mistake, database entries with zero references were omitted in the CD index distributions, hiding a large number of outliers with a maximum CD index of one, while keeping them in the analysis [1]. Our reanalysis shows that the reported decline in disruptiveness can be attributed to a relative decline of these database entries with zero references. Notably, this was not caught by the robustness checks included in the manuscript. The regression adjustment fails to control for the hidden outliers as they correspond to a discontinuity in the CD index. Proper evaluation of the Monte-Carlo simulations reveals that, because of the preservation of the hidden outliers, even random citation behaviour replicates the observed decline in disruptiveness. Finally, while these papers and patents with supposedly zero references are the hidden drivers of the reported decline, their source documents predominantly do make references, exposing them as pure dataset artefacts.

研究动机与目标

  • 在大型引用数据集（论文和专利）中复现 Park 等人的颠覆性（CD）分析。
  • 识别是否零引用条目驱动了观测到的 CD5 值随时间的下降。
  • 评估 Park 等人的控制方法（回归分析和蒙特卡洛模拟）对数据伪迹的鲁棒性。
  • 提出处理零引用条目以避免由伪迹驱动的结论的正确方法。

提出的方法

  • 在时序有向引用网络中定义 CDt 指标，用于在一个窗口内对前向引用进行分类（CDt）。
  • 证明零引用论文/专利会在 CDt 中产生不连续性（当存在前向引用时，CDt=1）。
  • 在 Park 等人的回归中扩展一个零引用虚拟变量以控制不连续性并评估模型拟合度（R2）。
  • 复现蒙特卡洛重新连线分析，以检验在保持度分布的随机网络下观测到的下降是否仍然存在。
  • 使用多数据源（Web of Science、PatentsView、SciSciNet）来验证伪迹驱动的效应。
  • 提供补充分析，展示跨数据源的零引用伪迹。
Figure 1: Distribution of the $\mathbf{\mathrm{CD}_{5}}$ index with vs without the hidden outliers and its impact on the apparent decline of disruptive science and technology. This figure shows that $\mathrm{CD}_{5}=1$ papers and patents are driving the reported decline in the disruptiveness of scie
Figure 1: Distribution of the $\mathbf{\mathrm{CD}_{5}}$ index with vs without the hidden outliers and its impact on the apparent decline of disruptive science and technology. This figure shows that $\mathrm{CD}_{5}=1$ papers and patents are driving the reported decline in the disruptiveness of scie

实验结果

研究问题

  • RQ1在正确考虑零引用条目后，随时间推移观测到的平均 CD5 的下降是否仍然存在？
  • RQ2在回归中包含零引用虚拟变量的控制是否充分解决 CD5 的不连续性？
  • RQ3当保留或移除零引用伪迹时，蒙特卡洛重新连线结果是否仍然呈现下降？
  • RQ4零引用条目是不是主要是元数据伪迹，而不是指示真实颠覆性的指标？
  • RQ5观测到的下降是否在多数据源和多前向引用窗口中保持一致？

主要发现

  • 来自零引用条目的 CD5=1 的隐藏离群值对表观下降的贡献很大。
  • 排除零引用条目或对其进行适当控制，在很大程度上消除了论文和专利的 CD5 随时间的下降。
  • 在回归模型中加入零引用虚拟变量显著改善拟合度（专利：R2 从 0.10/0.15 提升到 0.52；论文：0.95）。
  • 在随机重新连线的网络中，当保持零引用对应关系时也显示出类似的下降，这表明这是伪迹而非真实的颠覆趋势。
  • 跨数据源，绝大多数 CD5=1 且零引用的条目在其PDF中仍包含引用，证实元数据错误是伪迹的来源。
  • 总体而言，随时间的颠覆性下降归因于数据质量的改进和伪迹，而非真正的科学或技术进步。
Figure 2: The reason why the robustness checks in Park et al. [ 1 ] failed to detect the consequences of the hidden outliers. This figure displays how the Park et al. [ 1 ] regression adjustment (models $4$ and $8$ in Supplementary Table $1$ in [ 1 ] ) fails to control for the discontinuous effect o
Figure 2: The reason why the robustness checks in Park et al. [ 1 ] failed to detect the consequences of the hidden outliers. This figure displays how the Park et al. [ 1 ] regression adjustment (models $4$ and $8$ in Supplementary Table $1$ in [ 1 ] ) fails to control for the discontinuous effect o

更好的研究，从现在开始

从论文设计到论文写作，大幅缩短您的研究时间。

无需绑定信用卡

本解读由 AI 生成，并经人工编辑审核。