[Paper Review] AlabOS: A Python-based Reconfigurable Workflow Management Framework for Autonomous Laboratories

Yuxing Fei, Bernardus Rendy+11 more|arXiv (Cornell University)|May 22, 2024

Scientific Computing and Data ManagementDecision Sciences3 citations

TL;DR

AlabOS is a Python-based, reconfigurable workflow management framework for autonomous laboratories that enables flexible, modular, and concurrent execution of experimental workflows in materials research. It supports complex task dependencies, parallelization, and dynamic reconfiguration, and has successfully orchestrated over 2,500 samples in a prototype lab with peak throughput of 149 samples in a single day.

ABSTRACT

The recent advent of autonomous laboratories, coupled with algorithms for high-throughput screening and active learning, promises to accelerate materials discovery and innovation. As these autonomous systems grow in complexity, the demand for robust and efficient workflow management software becomes increasingly critical. In this paper, we introduce AlabOS, a general-purpose software framework for orchestrating experiments and managing resources, with an emphasis on automated laboratories for materials synthesis and characterization. AlabOS features a reconfigurable experiment workflow model and a resource reservation mechanism, enabling the simultaneous execution of varied workflows composed of modular tasks while eliminating conflicts between tasks. To showcase its capability, we demonstrate the implementation of AlabOS in a prototype autonomous materials laboratory, A-Lab, with around 3,500 samples synthesized over 1.5 years.

Motivation & Objective

• To address the growing need for flexible, extensible, and robust workflow management in autonomous laboratories for materials research.
• To overcome limitations of existing frameworks that enforce rigid, sequential workflows and lack support for complex dependencies and parallel execution.
• To enable researchers to define and execute diverse, reconfigurable experimental workflows with minimal low-level programming.
• To support seamless integration of physical hardware, data management, and human-in-the-loop operations in self-driving laboratories.
• To provide a platform-agnostic, user-friendly system that scales to high-throughput, multi-sample experimental campaigns.

Proposed method

• AlabOS uses a task-based, actor-model architecture where each experimental operation is encapsulated as an independent task actor with defined inputs, outputs, and resource requirements.
• It implements a resource occupation mechanism to prevent conflicts between concurrent tasks, ensuring safe and efficient scheduling of shared laboratory resources.
• The framework provides high-level Python APIs for defining workflows, enabling users to compose complex, tree-like, or parallel task sequences without low-level system programming.
• It includes a dashboard and notification system for real-time monitoring and human-machine interaction, especially useful during maintenance and error recovery.
• Workflows are managed through a modular, extensible design that allows dynamic reconfiguration and supports both automated and manual intervention points.
• The system integrates with hardware controllers and data storage, maintaining a centralized database to track sample states and task progress across multiple experiments.

Figure 1 : Schematic representation of lab states managed within the AlabOS system. The lab status is divided into four collections: (a) The sample collection tracks the sample’s position and identifies the task reserving that position; (b) the device collection records the status of the device and

Experimental results

Research questions

• RQ1 How can a workflow management system support complex, reconfigurable, and concurrent experimental workflows in autonomous materials laboratories?
• RQ2 What architectural patterns enable safe, scalable, and flexible orchestration of heterogeneous laboratory tasks with shared resources?
• RQ3 How can a software framework balance automation with human oversight and intervention in self-driving laboratory environments?
• RQ4 To what extent can a general-purpose, platform-independent framework reduce the development overhead for building autonomous laboratories?
• RQ5 What performance and throughput metrics can be achieved with a modular, task-based workflow engine in a real-world autonomous lab setting?

Key findings

• AlabOS successfully managed the execution of over 2,500 distinct samples in a prototype autonomous laboratory over approximately one year.
• The system achieved a peak daily throughput of 149 samples on February 9, 2024, demonstrating high scalability and efficiency.
• The framework supported multiple complex workflows, including parallel XRD and SEM/EDS analysis, manual heating with operator notification, and external sample processing via a Starting task.
• The maximum number of completed tasks per task type ranged from 1 (Ending) to 1,865 (RecoverPowder), indicating robust handling of high-volume operations.
• The resource occupation mechanism effectively prevented conflicts between concurrent tasks, enabling safe execution of parallel and dependent workflows.
• The dashboard and notification system enabled effective human-machine collaboration, particularly during maintenance and error recovery, enhancing system reliability.

Figure 2 : An architecture diagram for AlabOS. (a) Human operators submit experiments, monitor the laboratory state, and manage user requests. (b) The physical laboratory performs these tasks via manager processes. These include (c) the dashboard server that hosts the user interface and APIs for mon