[論文レビュー] Overcoming Barriers to Computational Reproducibility

Roman Hornung, László Németh+14 more|arXiv (Cornell University)|Jan 27, 2026

Scientific Computing and Data Management被引用数 1

ひとこと要約

The paper synthesizes benefits of computational reproducibility, identifies barriers, and offers pragmatic guidelines and a harmonised standards concept for journals to promote reproducible analyses across disciplines.

ABSTRACT

Computational reproducibility, the possibility for independent researchers to exactly reproduce published empirical results, is fundamental to science. Despite its importance, the proportion of research articles aiming for reproducibility remains low and uneven across disciplines. Barriers include a perceived lack of incentives for researchers and journals, practical challenges in preparing reproducible materials, and the absence of harmonised standards of reproducibility processes and requirements by journals. Existing guidance is often highly technical, reaching mainly those already engaged with reproducible research. In this paper, we first synthesize evidence on the benefits of reproducibility for both authors and journals. Drawing on our extensive experience in reproducibility checking at various journals, we then put forward concise, pragmatic guidelines for creating reproducible analyses across disciplines. We further review current reproducibility policies of selected journals, illustrating the substantial heterogeneity in requirements and procedures. Motivated by the latter, we propose conceptual foundations for a harmonised multi-tier system of reproducibility standards that could support transparent, consistent assessment across journals and research communities. Our goal as journal (reproducibility) editors and contributors to the MaRDI initiative is to encourage broader adoption of reproducibility practices, in particular by lowering practical barriers for authors and journals.

研究の動機と目的

• Summarize evidence on the benefits of reproducibility for authors and journals.
• Identify key barriers that reduce reproducibility uptake among researchers and journals.
• Provide concise, actionable guidelines for creating reproducible analyses across disciplines.
• Review current reproducibility policies of selected journals to illustrate heterogeneity.
• Propose conceptual foundations for a harmonised multi-tier system of reproducibility standards.

提案手法

• Synthesize existing literature and practical experiences from reproducibility checking at multiple journals.
• Draw on authors' long-standing roles as reproducibility editors to inform guidelines.
• Present concise, pragmatic guidelines accessible to readers with varying computational expertise.
• Analyze current journal policies to highlight heterogeneity in requirements and procedures.
• Propose conceptual foundations for harmonised reproducibility standards to support cross-journal assessment.

実験結果

リサーチクエスチョン

• RQ1 What are the benefits of reproducible research for authors and journals?
• RQ2 What practical guidelines can lower barriers to reproducibility for a broad audience?
• RQ3 How do current journal reproducibility policies vary across disciplines and journals?
• RQ4 What conceptual framework could support harmonised, multi-tier reproducibility standards?

主な発見

• Reproducibility practices provide practical workflow and future-work benefits for researchers, including easier reuse and increased confidence in results.
• Publicly available code and data are associated with higher citation counts and greater methodological impact.
• Journals benefit from reproducibility policies as quality signals and by improving submission quality and transparency.
• There is substantial heterogeneity in current journal reproducibility policies, underscoring the need for harmonised standards.
• Even lightweight reproducibility requirements can meaningfully enhance transparency and scientific rigour.