[Paper Review] Dynamic Mortality Risk Predictions in Pediatric Critical Care Using Recurrent Neural Networks

Melissa Aczon, David Ledbetter+5 more|arXiv (Cornell University)|Jan 23, 2017

Machine Learning in Healthcare19 references71 citations

TL;DR

The study trains a three-layer LSTM-based RNN on 12,020 pediatric ICU encounters to continuously predict in-ICU mortality, outperforming static models like PIM 2 and PRISM 3 in AUROC (~93.4%).

ABSTRACT

Viewing the trajectory of a patient as a dynamical system, a recurrent neural network was developed to learn the course of patient encounters in the Pediatric Intensive Care Unit (PICU) of a major tertiary care center. Data extracted from Electronic Medical Records (EMR) of about 12000 patients who were admitted to the PICU over a period of more than 10 years were leveraged. The RNN model ingests a sequence of measurements which include physiologic observations, laboratory results, administered drugs and interventions, and generates temporally dynamic predictions for in-ICU mortality at user-specified times. The RNN's ICU mortality predictions offer significant improvements over those from two clinically-used scores and static machine learning algorithms.

Motivation & Objective

• Motivate improved mortality prediction in pediatric critical care beyond static scoring systems.
• Develop a dynamic, time-aware model that continuously updates mortality risk as new data arrive.
• Leverage long short-term memory networks to integrate heterogeneous temporal clinical data.
• Assess whether dynamic RNN predictions outperform existing clinical scores and static ML baselines.

Proposed method

• Model: three-layer LSTM RNN using inputs X_V (vitals), X_L (labs), X_I (interventions), X_D (drugs), and Δt for future horizon.
• Data: anonymized EMR from PICU at Children’s Hospital Los Angeles (Dec 2002–Mar 2016); 12,020 patients, 16,559 encounters; 75% train, 25% holdout; ensure encounters from the same patient are in one set.
• Preprocessing: variable aggregation to ~300 features, Z-normalization, imputation (zero for drugs/interventions; forward-fill for physiologic/lab vars).
• Output: probability of survival at t_n + Δt_n; input includes Δt_n to control/flex time-to-prediction.
• Architecture: Keras-based three-layer LSTM network implemented to predict in-ICU mortality.
• Evaluation: ROC/AUC comparisons against LR, MLP, PIM 2, and PRISM 3 on holdout data.

Experimental results

Research questions

• RQ1 Does a dynamic RNN outperform static clinical scores (PIM 2, PRISM 3) and static ML baselines in predicting in-ICU mortality for pediatric patients?
• RQ2 How does predictive accuracy evolve with increasing observation time and data availability?
• RQ3 What is the impact of modeling temporal trends via recurrence on predictive performance?
• RQ4 Can the model leverage diverse EMR features (vitals, labs, interventions, drugs) to improve discrimination?

Key findings

• RNN achieves AUROC of 93.4% on 2849 holdout encounters with both PIM 2 and PRISM 3 scores.
• Static ML baselines: MLP 88.8%, LR 86.1%; PIM 2 86.3%, PRISM 3 88.0% (p-values < .01 or < .003).
• RNN predictions from the 12th hour outperform PRISM 3 that uses 12 hours of data; with more observation time, RNN accuracy continues to improve.
• RNN provides dynamic, continuously updated mortality risk as new data arrive; static scores do not incorporate ongoing temporal history.
• Inclusion of approximately 300 variables and dynamic integration of temporal trends drive performance gains; potential enhancements include additional monitor data and fluid balance.