[Paper Review] Temporal-Clustering Invariance in Irregular Healthcare Time Series

Mohammad Taha Bahadori, Zachary C. Lipton|arXiv (Cornell University)|Apr 27, 2019

Machine Learning in Healthcare45 references14 citations

TL;DR

This paper proposes temporal-clustering invariance as a regularizing invariance in irregular healthcare time series, introducing a stochastic data augmentation method that coarsens time series by merging nearby events with higher probability. The MultiResolution Ensemble (MRE) model improves mortality prediction accuracy by ensembling predictions across multiple coarsened resolutions, achieving a 53.92% mAP on MIMIC-III, a 2.39 percentage point improvement over the baseline.

ABSTRACT

Electronic records contain sequences of events, some of which take place all at once in a single visit, and others that are dispersed over multiple visits, each with a different timestamp. We postulate that fine temporal detail, e.g., whether a series of blood tests are completed at once or in rapid succession should not alter predictions based on this data. Motivated by this intuition, we propose models for analyzing sequences of multivariate clinical time series data that are invariant to this temporal clustering. We propose an efficient data augmentation technique that exploits the postulated temporal-clustering invariance to regularize deep neural networks optimized for several clinical prediction tasks. We introduce two techniques to temporally coarsen (downsample) irregular time series: (i) grouping the data points based on regularly-spaced timestamps; and (ii) clustering them, yielding irregularly-paced timestamps. Moreover, we propose a MultiResolution Ensemble (MRE) model, improving predictive accuracy by ensembling predictions based on inputs sequences transformed by different coarsening operators. Our experiments show that MRE improves the mAP on the benchmark mortality prediction task from 51.53% to 53.92%.

Motivation & Objective

• To investigate whether temporal-clustering invariance exists in irregular clinical time series data.
• To develop data augmentation techniques that exploit this invariance to regularize deep learning models.
• To improve model robustness and predictive accuracy in clinical time series tasks using multi-resolution ensembling.
• To address the challenge of inconsistent temporal clustering due to variations in healthcare data collection practices.

Proposed method

• Proposes a stochastic coarsening operator that randomly merges adjacent events, with higher probability for events closer in time.
• Introduces a deterministic cluster&count operator that groups events into irregularly spaced clusters and averages values, appending cluster size as a feature.
• Develops a MultiResolution Ensemble (MRE) model that processes the same input through multiple coarsening operators and ensembles predictions using attention.
• Uses the same neural network architecture across all resolutions, sharing weights to limit parameter growth.
• Applies the coarsening operators during data preprocessing and persists clusters to speed up training.
• Employs attention-based averaging to combine predictions from multiple resolutions at test time.

Experimental results

Research questions

• RQ1 Does temporal-clustering invariance exist in multivariate clinical time series, such that predictions should remain stable regardless of whether events are clustered or dispersed in time?
• RQ2 Can data augmentation based on temporal coarsening improve generalization and robustness in deep learning models for clinical prediction?
• RQ3 Does ensembling predictions across multiple resolutions of coarsened time series lead to improved predictive accuracy?
• RQ4 How does the proposed method compare to standard baselines in terms of robustness to data distribution shifts and adversarial perturbations?

Key findings

• The MultiResolution Ensemble (MRE) model improves the mean average precision (mAP) on the mortality prediction task from 51.53% to 53.92% on the MIMIC-III benchmark.
• The proposed data augmentation technique enhances model robustness to adversarial perturbations and reduces overfitting.
• The MRE-Grid model with a GRU weak-learner converges faster per training iteration than the individual weak learner.
• The MRE model achieves improved performance on the length-of-stay (LoS) prediction task, with lower MAE and RMSE than prior work, though results are sensitive to dataset preprocessing bias.
• The coarsening operators provide a degree of invariance to small measurement errors and time jitter, similar to time deformation invariance.
• The method is computationally efficient, with minimal training overhead, as coarsening is precomputed and stored.