[Paper Review] ReproServer: Making Reproducibility Easier and Less Intensive

Rémi Rampin, Fernando Chirigati+2 more|arXiv (Cornell University)|Aug 4, 2018

Scientific Computing and Data Management3 references4 citations

TL;DR

ReproServer is a web-based platform that enables one-click reproduction of computational experiments using ReproZip bundles, automatically building and executing isolated Docker environments from shared .rpz files. It eliminates dependency management and local setup, allowing researchers and reviewers to run, inspect, and extend experiments directly in a browser with no software installation.

ABSTRACT

Reproducibility in the computational sciences has been stymied because of the complex and rapidly changing computational environments in which modern research takes place. While many will espouse reproducibility as a value, the challenge of making it happen (both for themselves and testing the reproducibility of others' work) often outweigh the benefits. There have been a few reproducibility solutions designed and implemented by the community. In particular, the authors are contributors to ReproZip, a tool to enable computational reproducibility by tracing and bundling together research in the environment in which it takes place (e.g. one's computer or server). In this white paper, we introduce a tool for unpacking ReproZip bundles in the cloud, ReproServer. ReproServer takes an uploaded ReproZip bundle (.rpz file) or a link to a ReproZip bundle, and users can then unpack them in the cloud via their browser, allowing them to reproduce colleagues' work without having to install anything locally. This will help lower the barrier to reproducing others' work, which will aid reviewers in verifying the claims made in papers and reusing previously published research.

Motivation & Objective

• To address the persistent challenge of reproducibility in computational research by reducing the technical and operational burden of sharing and reusing experiments.
• To eliminate the need for researchers to manually manage software dependencies, environment configurations, and installation procedures during sharing or peer review.
• To enable seamless, interactive reproduction of experiments in a web browser without local software installation, using ReproZip-compressed bundles.
• To integrate with existing research workflows by supporting data repositories like Open Science Framework and figshare for persistent, citable links to reproducible experiments.
• To improve peer review by allowing reviewers to instantly access and execute experiments via a single URL, enhancing transparency and trust in published results.

Proposed method

• Users upload a ReproZip bundle (.rpz file) or provide a link to a repository-hosted .rpz file through the ReproServer web interface.
• The ReproServer back-end extracts metadata (input/output files, command lines, environment specs) from the .rpz bundle and builds a Docker image from the captured dependencies.
• The Docker image is cached in a private registry to accelerate future reproductions and detect malformed bundles early.
• Upon user request, ReproServer runs the experiment in the isolated Docker environment using specified input files and parameters, enforcing time and memory limits to prevent abuse.
• Execution logs are displayed in real time, and output files are stored on an Amazon S3-compatible storage service for download after completion.
• A permanent, shareable URL is generated for each reproduction, enabling citation, embedding in publications, and reuse with modified inputs.

Experimental results

Research questions

• RQ1 Can a web-based system significantly reduce the effort required to reproduce computational experiments across diverse environments?
• RQ2 How effectively can ReproServer automate the setup and execution of experiments from ReproZip bundles without requiring local software installation?
• RQ3 To what extent does ReproServer improve the reproducibility of research during peer review by enabling instant, interactive access to experiments?
• RQ4 Can ReproServer be deployed and scaled efficiently in production environments using container orchestration (e.g., Kubernetes) and cloud infrastructure?
• RQ5 How does integrating with existing data repositories (e.g., Open Science Framework) enhance discoverability and persistence of reproducible research outputs?

Key findings

• ReproServer enables one-click reproduction of experiments via a web browser, eliminating the need for local software installation, dependency management, or environment configuration.
• The system successfully unpacks and executes experiments from ReproZip bundles across different operating systems, ensuring portability and isolation through Docker containers.
• ReproServer generates persistent, shareable URLs for each reproduction, which can be embedded in papers or shared for collaborative validation and extension.
• By caching Docker images, ReproServer improves performance and detects malformed bundles early, reducing failure rates during reproduction.
• The integration with repositories like Open Science Framework ensures long-term persistence and citability of reproducible experiments without relying on ReproServer’s own storage.
• The system supports interactive modification of input files and parameters during reproduction, enabling sensitivity analysis and validation under varied conditions.