[Paper Review] Unsupervised Representation for EHR Signals and Codes as Patient Status Vector

Sajad Darabi, Mohammad Kachuee+1 more|arXiv (Cornell University)|Oct 4, 2019

Machine Learning in Healthcare28 references4 citations

TL;DR

This paper proposes a two-step unsupervised representation learning method that combines clinical codes and vital signals into a patient status vector (PSV) using autoencoders and forecasting fine-tuning. The approach improves generalization on mortality and readmission prediction for both short- and long-duration ICU stays on the eICU dataset, outperforming prior methods by leveraging both single-visit and multi-visit patient data.

ABSTRACT

Effective modeling of electronic health records presents many challenges as they contain large amounts of irregularity most of which are due to the varying procedures and diagnosis a patient may have. Despite the recent progress in machine learning, unsupervised learning remains largely at open, especially in the healthcare domain. In this work, we present a two-step unsupervised representation learning scheme to summarize the multi-modal clinical time series consisting of signals and medical codes into a patient status vector. First, an auto-encoder step is used to reduce sparse medical codes and clinical time series into a distributed representation. Subsequently, the concatenation of the distributed representations is further fine-tuned using a forecasting task. We evaluate the usefulness of the representation on two downstream tasks: mortality and readmission. Our proposed method shows improved generalization performance for both short duration ICU visits and long duration ICU visits.

Motivation & Objective

• To address the challenge of limited labeled data in EHR modeling by developing an unsupervised representation learning method.
• To integrate multi-modal EHR data—clinical codes and vital signals—into a unified patient status vector for downstream tasks.
• To improve model generalization for both short-duration and long-duration ICU stays, which exhibit different clinical dynamics.
• To leverage both single-visit and multi-visit patient data in pre-training to enhance representation quality.
• To demonstrate that unsupervised pre-training on diverse EHR data improves performance on mortality and readmission prediction.

Proposed method

• A two-step unsupervised training pipeline is used: first, an autoencoder compresses sparse clinical codes and time-series vital signals into distributed representations.
• The concatenated representations from the autoencoder are then fine-tuned via a forecasting autoencoder task to capture sequential dependencies.
• The method uses a single-visit autoencoding step to learn from individual ICU stays and a multi-visit forecasting step to model longitudinal patterns.
• The final patient status vector (PSV) is formed by combining code and signal embeddings, enabling use in downstream classification tasks.
• The model is pre-trained in an unsupervised manner and then frozen for downstream fine-tuning on mortality and readmission prediction.
• The approach is evaluated on the eICU dataset, using periodic vital signals and ICD-9 codes as input features.

Experimental results

Research questions

• RQ1 Can a two-step unsupervised learning framework effectively combine clinical codes and vital signals into a unified patient status vector?
• RQ2 Does pre-training on both single-visit and multi-visit ICU data improve downstream performance on mortality and readmission prediction?
• RQ3 How does the proposed method generalize across short-duration and long-duration ICU stays, which have distinct clinical dynamics?
• RQ4 To what extent do code-only or signal-only representations limit performance compared to joint modeling?
• RQ5 Can unsupervised pre-training reduce reliance on costly labeled data while improving prediction accuracy in EHR analytics?

Key findings

• The PSV model with both code and signal representations achieved a PR-AUC of 62.46 (±0.20) and ROC of 90.06 (±0.12) on the 1h-24h mortality task, outperforming code-only and signal-only variants.
• For long-duration stays (24h-720h), the PSV model achieved a PR-AUC of 48.88 (±0.13) and ROC of 85.90 (±0.09), showing improved generalization over baselines.
• The semi-supervised variant of the PSV model achieved a PR-AUC of 65.40 (±0.35) and ROC of 89.10 (±0.59) on the 1h-24h mortality task, indicating strong performance with limited labels.
• The model significantly outperformed Seq2Seq and Transformer baselines, which reported PR-AUC values below 10 on the 1h-24h mortality task.
• Performance dropped substantially for long-stay patients, suggesting that current EHR data may not fully capture the complex dynamics of prolonged ICU care.
• Ablation studies confirmed that combining code and signal representations yields superior results, with signal-only representations performing poorly on readmission prediction (PR-AUC 30.47 ±0.13).