[Paper Review] ARTEMIS: AI-driven Robotic Triage Labeling and Emergency Medical Information System

Senthilkumaran, Revanth Krishna, Prashanth, Mridu+4 more|arXiv (Cornell University)|Sep 16, 2023

Disaster Response and ManagementHealth Professions3 citations

TL;DR

ARTEMIS is an AI-driven robotic system that uses computer vision and contact-based vital sign sensors to autonomously assess and assign triage labels (Critical to Minor) to victims in mass casualty incidents, achieving 92% classification accuracy in a controlled simulation, thereby enhancing first responder efficiency and real-time decision-making.

ABSTRACT

Mass casualty incidents (MCIs) pose a significant challenge to emergency medical services by overwhelming available resources and personnel. Effective victim assessment is the key to minimizing casualties during such a crisis. We introduce ARTEMIS, an AI-driven Robotic Triage Labeling and Emergency Medical Information System, to aid first responders in MCI events. It leverages speech processing, natural language processing, and deep learning to help with acuity classification. This is deployed on a quadruped that performs victim localization and preliminary injury severity assessment. First responders access victim information through a Graphical User Interface that is updated in real-time. To validate our proposed algorithmic triage protocol, we used the Unitree Go1 quadruped. The robot identifies humans, interacts with them, gets vitals and information, and assigns an acuity label. Simulations of an MCI in software and a controlled environment outdoors were conducted. The system achieved a triage-level classification precision of over 74% on average and 99% for the most critical victims, i.e. level 1 acuity, outperforming state-of-the-art deep learning-based triage labeling systems. In this paper, we showcase the potential of human-robot interaction in assisting medical personnel in MCI events.

Motivation & Objective

• Address the critical challenge of triage overload during mass casualty incidents (MCIs) by automating victim assessment using robotics and AI.
• Reduce response time and human error in emergency triage by deploying autonomous robots equipped with sensors for real-time vital sign collection.
• Improve coordination between robots and first responders through a real-time, accessible web-based interface for dynamic emergency management.
• Develop a robust, scalable triage classification model trained on clinical data to ensure high accuracy and reliability in life-or-death scenarios.
• Overcome data scarcity and skew in triage datasets by creating and releasing a balanced, augmented dataset (MIMIC+) for future research.

Proposed method

• Deploy a pre-trained SSD MobileNetV1 model on an unmanned ground robot for real-time human detection and victim identification in MCI environments.
• Integrate contact-based sensors to collect vital signs (heart rate, SpO2, temperature) from identified victims for triage classification.
• Train a deep neural network on a public dataset from Beth Israel Deaconess Medical Center (BIDMC), augmented with domain-aware data synthesis to address class imbalance.
• Implement a real-time web-based graphical user interface (GUI) that updates with robot-collected triage data and is accessible to first responders.
• Use a one-vs-all ROC analysis and precision-recall metrics to evaluate multi-class triage performance across five acuity levels (Critical to Minor).
• Conduct statistical validation using the Wilcoxon signed-rank test to compare neural network and decision tree models, confirming superior performance of the former.

Experimental results

Research questions

• RQ1 Can an AI-driven robotic system achieve high-accuracy, real-time triage labeling in simulated mass casualty incidents using contact-based vital signs?
• RQ2 How does data augmentation impact the performance of triage classification models when trained on imbalanced clinical datasets?
• RQ3 To what extent does a neural network outperform a decision tree in classifying critical (acuity level 1) patients with high precision and low error margin?
• RQ4 How effective is real-time data sharing via a web GUI in enhancing human-robot collaboration during emergency response operations?
• RQ5 What are the key sensor and system-level limitations affecting the deployment of robotic triage systems in real-world MCI scenarios?

Key findings

• The ARTEMIS system achieved a 92% accuracy rate in classifying triage levels across five categories (Critical to Minor) during a controlled MCI simulation.
• The neural network model outperformed the decision tree model in precision, with a statistically significant p-value of 2.23e-06 from the Wilcoxon signed-rank test.
• The model demonstrated high precision (0.91) for acuity level 1 (Critical), indicating reliable identification of the most urgent patients.
• Despite high precision for critical cases, recall for levels 1 and 2 was low, suggesting overlapping clinical criteria that challenge clear distinction.
• Temperature and other vital signs showed limited correlation with acuity levels, prompting the need for data augmentation to improve model robustness.
• The researchers released the augmented and balanced dataset, MIMIC+, to address the scarcity of publicly available triage data and support future research.