[Paper Review] Clinical Intervention Prediction and Understanding using Deep Networks

Harini Suresh, Nathan Hunt+4 more|arXiv (Cornell University)|May 23, 2017

Machine Learning in Healthcare15 references78 citations

TL;DR

The paper develops forward-facing LSTM and CNN models to predict onset and weaning of multiple ICU interventions using multimodal MIMIC-III data, achieving state-of-the-art AUCs and providing interpretability through feature occlusion and activation analyses.

ABSTRACT

Real-time prediction of clinical interventions remains a challenge within intensive care units (ICUs). This task is complicated by data sources that are noisy, sparse, heterogeneous and outcomes that are imbalanced. In this paper, we integrate data from all available ICU sources (vitals, labs, notes, demographics) and focus on learning rich representations of this data to predict onset and weaning of multiple invasive interventions. In particular, we compare both long short-term memory networks (LSTM) and convolutional neural networks (CNN) for prediction of five intervention tasks: invasive ventilation, non-invasive ventilation, vasopressors, colloid boluses, and crystalloid boluses. Our predictions are done in a forward-facing manner to enable "real-time" performance, and predictions are made with a six hour gap time to support clinically actionable planning. We achieve state-of-the-art results on our predictive tasks using deep architectures. We explore the use of feature occlusion to interpret LSTM models, and compare this to the interpretability gained from examining inputs that maximally activate CNN outputs. We show that our models are able to significantly outperform baselines in intervention prediction, and provide insight into model learning, which is crucial for the adoption of such models in practice.

Motivation & Objective

• Integrate heterogeneous ICU data (vitals, labs, notes, demographics) to predict onset and weaning of invasive interventions.
• Develop forward-facing, real-time prediction models with six-hour gap windows.
• Compare LSTM and CNN architectures for multi-task intervention prediction.
• Explore model interpretability via feature occlusion (LSTM) and input activation analysis (CNN).

Proposed method

• Combine static variables, time-varying vitals/labs, and notes into hourly feature vectors including topic distributions from notes.
• Represent notes with 50-topic LDA distributions replicated over time.
• Use physiological words by discretizing z-scored vitals to capture missingness and reduce noise.
• Train two deep architectures (LSTM with two 512-neuron layers; CNN with multi-granularity temporal convolutions) and a logistic regression baseline.
• Evaluate with per-class AUC and macro AUC, using 70/10/20 train/validation/test splits and early stopping on validation AUC.

Experimental results

Research questions

• RQ1 Can a deep learning model predict onset, weaning, and ongoing status of invasive and non-invasive interventions in the ICU using all available data modalities?
• RQ2 Do LSTM and CNN architectures offer complementary strengths for multi-task ICU intervention prediction?
• RQ3 How do different data modalities (vitals/labs vs. notes vs. static demographics) contribute to predictive performance for each intervention?
• RQ4 Is interpretability achievable for these models via feature occlusion (LSTM) and activation analysis (CNN)?

Key findings

• Deep architectures achieve state-of-the-art AUCs for predicting five ICU interventions (onset, wean, stay on, stay off) across tasks.
• Physiological-word representation improves onset predictions for low-prevalence events (e.g., ventilation onset from 0.61 to 0.75, colloid bolus onset from 0.52 to 0.72).
• LSTM occlusion shows vitals/labs drive predictions for invasive interventions while notes/topics dominate less invasive tasks.
• CNN analyses reveal clinically plausible short-term trajectories and hallucinatory inputs that maximize specific outputs, aiding interpretability.
• Weaning predictions exhibit less AUC variance, suggesting greater clinical consistency in decisions to discontinue interventions.