[Paper Review] Predicting Clinical Deterioration of Outpatients Using Multimodal Data Collected by Wearables

Dingwen Li, Jay Vaidya+5 more|arXiv (Cornell University)|Mar 12, 2018

Blood Pressure and Hypertension Studies34 references3 citations

TL;DR

This study proposes a machine learning framework using multimodal wearable data (step count, heart rate, and sleep) from Fitbit Charge HR wristbands to predict clinical deterioration—defined as readmission or death—in heart failure outpatients. The system achieved 88% patient compliance, high data yield (80% step data), and a weighted one-class SVM model reached 96.35% accuracy in early deterioration prediction, significantly outperforming the LACE index.

ABSTRACT

Hospital readmission rate is high for heart failure patients. Early detection of deterioration will help doctors prevent readmissions, thus reducing health care cost and providing patients with just-in-time intervention. Wearable devices (e.g., wristbands and smart watches) provide a convenient technology for continuous outpatient monitoring. In the paper, we explore the feasibility of monitoring outpatients using Fitbit Charge HR wristbands and the potential of machine learning models to predicting clinical deterioration (readmissions and death) among outpatients discharged from the hospital. We developed and piloted a data collection system in a clinical study which involved 25 heart failure patients recently discharged from a hospital. The results from the clinical study demonstrated the feasibility of continuously monitoring outpatients using wristbands. We observed high levels of patient compliance in wearing the wristbands regularly and satisfactory yield, latency and reliability of data collection from the wristbands to a cloud-based database. Finally, we explored a set of machine learning models to predict deterioration based on the Fitbit data. Through 5-fold cross validation, K nearest neighbor achieved the highest accuracy of 0.8800 for identifying patients at risk of deterioration using the health data from the beginning of the monitoring. Machine learning models based on multimodal data (step, sleep and heart rate) significantly outperformed the traditional clinical approach based on LACE index. Moreover, our proposed weighted samples one class SVM model can reach high accuracy (0.9635) for predicting the deterioration happening in the future using data collected by a sliding window, which indicates the potential for allowing timely intervention.

Motivation & Objective

• To evaluate the feasibility of continuous outpatient monitoring using wearable devices in heart failure patients.
• To assess data yield, reliability, and patient compliance in collecting multimodal health data (step count, heart rate, sleep) via Fitbit wristbands.
• To develop and evaluate machine learning models that predict clinical deterioration (readmission or death) using wearable sensor data.
• To compare the predictive performance of machine learning models using multimodal data against the traditional LACE index.
• To investigate the earliest point at which clinical deterioration can be predicted using longitudinal wearable data.

Proposed method

• Deployed a cloud-based data collection system to passively gather per-minute step count, heart rate, and sleep duration/quality data from Fitbit Charge HR wristbands.
• Conducted a clinical study with 25 heart failure patients recently discharged from hospital to assess real-world feasibility of wearable monitoring.
• Applied 5-fold cross-validation to evaluate multiple machine learning models, including K-Nearest Neighbors (KNN), one-class SVM, and random forest, using multimodal sensor features.
• Used a sliding window approach to train models on increasing data lengths (5-, 10-, 15-, 20-day windows) to assess early prediction capability.
• Implemented a weighted samples one-class SVM to improve prediction of rare events (deterioration), optimizing for high sensitivity and specificity.
• Fixed sensitivity at ~0.95 in model evaluation to prioritize early detection of high-risk patients.

Experimental results

Research questions

• RQ1 Can wearable devices like Fitbit Charge HR reliably collect multimodal health data (step count, heart rate, sleep) from heart failure outpatients in a real-world outpatient setting?
• RQ2 What is the level of patient compliance and data reliability in long-term wearable monitoring of heart failure patients post-discharge?
• RQ3 Can machine learning models trained on multimodal wearable data predict clinical deterioration (readmission or death) more accurately than the standard LACE index?
• RQ4 How early can clinical deterioration be predicted using only the first few days of wearable data?
• RQ5 Which combination of sensor modalities (step, heart rate, sleep) yields the highest predictive performance for clinical deterioration?

Key findings

• 88% of patients wore the Fitbit Charge HR wristband regularly, indicating high compliance in outpatient monitoring.
• The system collected 80% of per-minute step data from 84% of participants, demonstrating high data yield for activity tracking.
• The median gap in heart rate data was only 4 minutes, indicating acceptable data reliability despite lower yield than step data.
• The K-Nearest Neighbors (KNN) model achieved the highest accuracy of 0.8800 when using combined step, heart rate, and sleep data for risk prediction.
• The weighted samples one-class SVM model reached 96.35% accuracy in predicting future deterioration using data from a sliding window, indicating strong potential for early warning.
• Models using multimodal data (step, heart rate, sleep) significantly outperformed models using single modalities and the traditional LACE index, with the best model achieving 86.67% accuracy when all features were combined.