[Paper Review] Data management to support reproducible research

Brian A. Wandell, Ariel Rokem+3 more|arXiv (Cornell University)|Feb 21, 2015

Scientific Computing and Data ManagementDecision Sciences9 references20 citations

TL;DR

This paper presents a scientific data management (SDM) system designed to support reproducible research by integrating data and computational workflow management from the outset of research projects. Deployed at an MRI center for over two years, the system enhances data integrity, enables method reuse, and ensures transparency by tracking data and code throughout the research lifecycle, demonstrating that early adoption of SDM tools significantly improves reproducibility and collaboration in quantitative biology.

ABSTRACT

We describe the current state and future plans for a set of tools for scientific data management (SDM) designed to support scientific transparency and reproducible research. SDM has been in active use at our MRI Center for more than two years. We designed the system to be used from the beginning of a research project, which contrasts with conventional end-state databases that accept data as a project concludes. A number of benefits accrue from using scientific data management tools early and throughout the project, including data integrity as well as reuse of the data and of computational methods.

Motivation & Objective

• To address the lack of reproducibility in scientific research by implementing a data management system from the start of projects.
• To improve data integrity and transparency by tracking data and computational workflows throughout the research lifecycle.
• To enable reuse of data and computational methods across studies by maintaining structured, versioned records.
• To shift from end-of-project data archiving to continuous, integrated data management during active research.
• To support collaborative, transparent, and reproducible research in quantitative biology and neuroscience.

Proposed method

• The SDM system is designed to be implemented at the beginning of a research project, not as a post-hoc archive.
• It integrates data collection, processing, and version control with computational workflows to ensure full provenance tracking.
• The system supports both raw and processed data, with metadata captured at every stage of the pipeline.
• It enables researchers to reconstruct analyses exactly as performed, ensuring reproducibility.
• The system is built to be extensible and adaptable to different research domains, particularly in neuroimaging and quantitative biology.
• It uses a centralized, version-controlled infrastructure to maintain data lineage and facilitate collaboration.

Experimental results

Research questions

• RQ1 How can data management systems be integrated early in the research process to improve reproducibility?
• RQ2 What benefits does continuous data management offer compared to end-of-project data archiving?
• RQ3 To what extent can SDM tools enhance data integrity and method reuse in scientific research?
• RQ4 How does early deployment of SDM tools affect collaboration and transparency in quantitative research?
• RQ5 Can a unified data and workflow management system support reproducible research across diverse scientific projects?

Key findings

• The SDM system has been actively used at the MRI center for over two years, demonstrating long-term viability.
• By implementing data management from the start, the system ensures data integrity and full reproducibility of analyses.
• The system enables reuse of both data and computational methods across multiple studies, reducing redundancy.
• Researchers can exactly reproduce results due to complete provenance tracking of data and code.
• The approach shifts the paradigm from reactive data archiving to proactive, integrated data management.
• The system supports transparency and collaboration by maintaining a complete, auditable record of all research steps.