[Paper Review] Learning the Joint Representation of Heterogeneous Temporal Events for Clinical Endpoint Prediction

Luchen Liu, Jianhao Shen+3 more|arXiv (Cornell University)|Mar 13, 2018

Machine Learning in Healthcare14 citations

TL;DR

This paper proposes HE-LSTM, a deep learning model that learns joint representations of heterogeneous temporal events in electronic health records (EHRs) by introducing an event gate and event filter to model irregular sampling rates and nonlinear dependencies across diverse clinical events. Evaluated on real-world EHR data, HE-LSTM outperforms baselines in predicting death and abnormal lab tests, achieving up to 7.9% AUC improvement in early training and robust performance across varying sequence lengths.

ABSTRACT

The availability of a large amount of electronic health records (EHR) provides huge opportunities to improve health care service by mining these data. One important application is clinical endpoint prediction, which aims to predict whether a disease, a symptom or an abnormal lab test will happen in the future according to patients' history records. This paper develops deep learning techniques for clinical endpoint prediction, which are effective in many practical applications. However, the problem is very challenging since patients' history records contain multiple heterogeneous temporal events such as lab tests, diagnosis, and drug administrations. The visiting patterns of different types of events vary significantly, and there exist complex nonlinear relationships between different events. In this paper, we propose a novel model for learning the joint representation of heterogeneous temporal events. The model adds a new gate to control the visiting rates of different events which effectively models the irregular patterns of different events and their nonlinear correlations. Experiment results with real-world clinical data on the tasks of predicting death and abnormal lab tests prove the effectiveness of our proposed approach over competitive baselines.

Motivation & Objective

• To address the challenge of modeling irregular, multi-scale temporal patterns in heterogeneous clinical events from EHRs.
• To learn joint representations that capture complex nonlinear dependencies between diverse event types such as lab tests, diagnoses, and medications.
• To develop a scalable deep learning model suitable for large-scale EHR data with thousands of event types.
• To improve clinical endpoint prediction accuracy for outcomes like death and abnormal lab tests.

Proposed method

• Extends Phased LSTM by introducing an event gate controlled by event type embeddings and timestamps to handle asynchronous, multi-scale sampling rates.
• Introduces an event filter mechanism within the event gate to enhance adaptability to varying event frequencies and improve representation learning.
• Uses learned embeddings for each event type and its attributes to represent heterogeneous events in a shared latent space.
• Employs a gated recurrent unit architecture with time-gated memory updates to model long-term temporal dependencies across irregular event sequences.
• Trains the model end-to-end using cross-entropy and AUC as optimization objectives for clinical endpoint prediction.
• Applies multi-scale input sequence evaluation to assess temporal dependency modeling performance across varying history lengths.

Experimental results

Research questions

• RQ1 Can a deep learning model effectively capture the irregular visiting patterns of heterogeneous clinical events in EHRs?
• RQ2 How do nonlinear correlations between diverse event types (e.g., lab tests and diagnoses) impact clinical endpoint prediction?
• RQ3 To what extent does modeling joint representations of heterogeneous events improve prediction performance compared to modeling events independently?
• RQ4 How robust is the proposed model to variations in input sequence length and initial parameter settings?
• RQ5 Does the integration of an event filter and event gate enhance generalization and convergence speed in clinical prediction tasks?

Key findings

• HE-LSTM achieved a 7.9% improvement in AUC for death prediction during the first training epoch compared to the phase gate baseline.
• The event gate and event filter improved AUC by 0.5% and AP by 2.8% on average over the phase gate baseline for death prediction.
• HE-LSTM maintained stable performance gains as input sequence length increased, with AP rising from 0.7551 to 0.7687 and AUC from 0.9482 to 0.9516 over sequences of 20 to 1000 events.
• The model with the event filter showed 4.1% to 6.6% average performance improvement over the reduced HE-LSTM without the filter across different initial period settings.
• HE-LSTM outperformed all baselines in both death and abnormal lab test prediction tasks, demonstrating superior scalability and representation learning on real-world EHR data.
• The model’s performance gains were most pronounced in longer sequences, confirming its effectiveness in modeling long-term temporal dependencies in heterogeneous event streams.