[Paper Review] WorkflowHub: a registry for computational workflows

Ove Gustafsson, Sean R. Wilkinson+13 more|Research Explorer (The University of Manchester)|Oct 9, 2024

Scientific Computing and Data ManagementDecision Sciences3 citations

TL;DR

WorkflowHub is a global, FAIR-compliant registry that aggregates computational workflows from diverse repositories, enabling discovery, versioning, and persistent identification via DOIs. By integrating with Git, RO-Crates, and external registries, it enhances workflow reproducibility, reusability, and scholarly credit, with over 700 workflows registered across 12 countries and a 2027 data retention policy including archival via Zenodo or Figshare.

ABSTRACT

The rising popularity of computational workflows is driven by the need for repetitive and scalable data processing, sharing of processing know-how, and transparent methods. As both combined records of analysis and descriptions of processing steps, workflows should be reproducible, reusable, adaptable, and available. Workflow sharing presents opportunities to reduce unnecessary reinvention, promote reuse, increase access to best practice analyses for non-experts, and increase productivity. In reality, workflows are scattered and difficult to find, in part due to the diversity of available workflow engines and ecosystems, and because workflow sharing is not yet part of research practice. WorkflowHub provides a unified registry for all computational workflows that links to community repositories, and supports both the workflow lifecycle and making workflows findable, accessible, interoperable, and reusable (FAIR). By interoperating with diverse platforms, services, and external registries, WorkflowHub adds value by supporting workflow sharing, explicitly assigning credit, enhancing FAIRness, and promoting workflows as scholarly artefacts. The registry has a global reach, with hundreds of research organisations involved, and more than 700 workflows registered.

Motivation & Objective

• To address the challenge of scattered, hard-to-find computational workflows across diverse scientific domains and workflow ecosystems.
• To promote the FAIR principles (Findable, Accessible, Interoperable, Reusable) for computational workflows as scholarly artifacts.
• To establish a centralized registry that supports workflow sharing, versioning, and long-term preservation with persistent identifiers.
• To enable credit attribution and scholarly recognition for workflow creators through DOI minting and metadata curation.
• To support interoperability with existing workflow platforms, repositories, and FAIR data infrastructures like DataCite and Zenodo.

Proposed method

• WorkflowHub uses Git-based version control to store and manage workflow repositories, enabling versioning, snapshots, and change tracking.
• It ingests workflows via direct Git import or RO-Crate submission, ensuring structured, machine-readable metadata and provenance.
• The registry integrates with external platforms (e.g., GitHub, GitLab, Bitbucket) and FAIR data infrastructures (e.g., DataCite for DOIs) to enhance discoverability and persistence.
• It implements a knowledge graph of registered workflow RO-Crates, published on Zenodo, to enable semantic interlinking and advanced discovery.
• A formal End-of-Life policy ensures long-term preservation by archiving workflows as RO-Crates in public repositories (e.g., Zenodo, Figshare) with updated DOIs.
• The system is built on the FAIRDOM-SEEK platform, leveraging existing FAIR software engineering practices and CRediT-compliant contributor tracking.

Figure 1: WorkflowHub connects to platforms, services, and resources that support a workflow’s life cycle [ 28 ] . A researcher initially needs to Plan & Find , where they either plan for a particular analysis and find existing workflows (i.e. using a registry), or Develop a new workflow. WorkflowHu

Experimental results

Research questions

• RQ1 How can computational workflows be effectively discovered and shared across diverse scientific communities and workflow ecosystems?
• RQ2 What technical and organizational mechanisms enable the long-term preservation and FAIR compliance of computational workflows?
• RQ3 How can workflow creators be credited and recognized as scholarly contributors through persistent identifiers and metadata?
• RQ4 To what extent can a centralized registry interoperate with existing workflow platforms and FAIR data infrastructures?
• RQ5 What infrastructure and governance models are required to sustain a global, community-driven workflow registry?

Key findings

• WorkflowHub has registered over 700 computational workflows from more than 120 research organizations across 12 countries, demonstrating broad international adoption.
• The registry supports persistent identification of workflows through DOIs issued via DataCite, ensuring long-term accessibility and citability.
• All registered workflows are archived as RO-Crates and will be preserved beyond 2027 through public repositories such as Zenodo or Figshare, with DOI redirection to archived versions.
• The system enables versioned workflow management via Git, allowing for snapshotting and reproducible execution across different workflow engine environments.
• The knowledge graph of registered workflow RO-Crates, published on Zenodo, enables semantic discovery and interlinking of workflows and their metadata.
• WorkflowHub’s integration with FAIRDOM-SEEK and support for CRediT contributor roles enhances scholarly attribution and transparency in workflow development.

Figure 2: Workflow types registered with WorkflowHub.