[Paper Review] MONAI Label: A framework for AI-assisted Interactive Labeling of 3D Medical Images

Andres Diaz‐Pinto, Sachidanand Alle+15 more|arXiv (Cornell University)|Mar 23, 2022

Radiomics and Machine Learning in Medical ImagingMedicine42 references41 citations

TL;DR

MONAI Label is an open-source framework that enables AI-assisted interactive and non-interactive labeling of 3D medical images, featuring DeepGrow, DeepEdit, scribbles-based methods, active learning, and GUI integrations (3D Slicer, OHIF).

ABSTRACT

The lack of annotated datasets is a major bottleneck for training new task-specific supervised machine learning models, considering that manual annotation is extremely expensive and time-consuming. To address this problem, we present MONAI Label, a free and open-source framework that facilitates the development of applications based on artificial intelligence (AI) models that aim at reducing the time required to annotate radiology datasets. Through MONAI Label, researchers can develop AI annotation applications focusing on their domain of expertise. It allows researchers to readily deploy their apps as services, which can be made available to clinicians via their preferred user interface. Currently, MONAI Label readily supports locally installed (3D Slicer) and web-based (OHIF) frontends and offers two active learning strategies to facilitate and speed up the training of segmentation algorithms. MONAI Label allows researchers to make incremental improvements to their AI-based annotation application by making them available to other researchers and clinicians alike. Additionally, MONAI Label provides sample AI-based interactive and non-interactive labeling applications, that can be used directly off the shelf, as plug-and-play to any given dataset. Significant reduced annotation times using the interactive model can be observed on two public datasets.

Motivation & Objective

• Address the bottleneck of scarce expert-annotated 3D medical datasets by enabling AI-assisted, efficient labeling workflows.
• Provide an open, modular SDK to develop and deploy labeling apps for clinical evaluation.
• Offer interactive and non-interactive segmentation approaches integrated with standard UI frontends.
• Incorporate active learning to prioritize annotation of uncertain or informative samples.
• Facilitate data management and deployment through a RESTful server and DICOMWeb compatibility.

Proposed method

• Provide three high-level MONAI Label modules: client, server, and datastore for annotations and models.
• Include interactive annotation models (DeepGrow, DeepEdit) and a scribbles-based segmentation method, plus a non-interactive automated segmentation option.
• Implement an active learning loop with uncertainty estimation (epistemic via dropout, optional aleatoric via TTA) and common acquisition functions to rank unlabeled volumes.
• Support GUI integrations (3D Slicer, OHIF) and DICOMWeb communications (WADO-RS, STOW-RS, QIDO-RS).
• Offer a framework to develop apps by inheriting MONAILabelApp, defining inferencing, training, data selection, and custom transforms.

Experimental results

Research questions

• RQ1 How can AI-assisted interactive labeling reduce manual annotation time for 3D medical images while maintaining or improving segmentation accuracy?
• RQ2 What is the impact of active learning strategies on annotation efficiency and model performance in MONAI Label?
• RQ3 How can MONAI Label be instantiated with different frontends (3D Slicer, OHIF) and integrated with DICOM-based workflows?
• RQ4 What are the benefits and limitations of DeepGrow, DeepEdit, and scribbles-based approaches within this framework?

Key findings

• Interactive DeepGrow and scribbles-based methods substantially reduce annotation time on spleen segmentation tasks compared to traditional tools.
• DeepGrow (2D+3D) enables 1–2.5 minute annotations per 3D volume in later stages, indicating up to ~10x faster performance versus contour-based methods.
• Scribbles-based online likelihood segmentation markedly decreases manual labeling effort, averaging around 2 minutes per sample versus paintbrush or contour-based approaches.
• DeepEdit supports continuous training and interactive refinement, enabling practitioners to start model-based annotation after labeling only a subset of volumes, significantly speeding up the workflow.
• Experiments on MSD datasets demonstrate time savings and competitive Dice scores; applications include telestroke brain hemorrhage labeling and neurosurgical brain atlas development.