[Paper Review] Clairvoyance: A Pipeline Toolkit for Medical Time Series

Daniel Jarrett, Jinsung Yoon+4 more|arXiv (Cornell University)|Jan 1, 2021

Machine Learning in Healthcare87 references13 citations

TL;DR

Clairvoyance is a unified, end-to-end, autoML-friendly pipeline toolkit for medical time-series learning that addresses engineering, evaluation, and optimization challenges in clinical ML. It enables reproducible, transparent experimentation across personalized prediction, treatment-effect estimation, and active monitoring using real-world datasets from outpatient, general ward, and ICU settings, demonstrating the first viable, comprehensive, and automatable pipeline for clinical time-series ML.

ABSTRACT

Time-series learning is the bread and butter of data-driven *clinical decision support*, and the recent explosion in ML research has demonstrated great potential in various healthcare settings. At the same time, medical time-series problems in the wild are challenging due to their highly *composite* nature: They entail design choices and interactions among components that preprocess data, impute missing values, select features, issue predictions, estimate uncertainty, and interpret models. Despite exponential growth in electronic patient data, there is a remarkable gap between the potential and realized utilization of ML for clinical research and decision support. In particular, orchestrating a real-world project lifecycle poses challenges in engineering (i.e. hard to build), evaluation (i.e. hard to assess), and efficiency (i.e. hard to optimize). Designed to address these issues simultaneously, Clairvoyance proposes a unified, end-to-end, autoML-friendly pipeline that serves as a (i) software toolkit, (ii) empirical standard, and (iii) interface for optimization. Our ultimate goal lies in facilitating transparent and reproducible experimentation with complex inference workflows, providing integrated pathways for (1) personalized prediction, (2) treatment-effect estimation, and (3) information acquisition. Through illustrative examples on real-world data in outpatient, general wards, and intensive-care settings, we illustrate the applicability of the pipeline paradigm on core tasks in the healthcare journey. To the best of our knowledge, Clairvoyance is the first to demonstrate viability of a comprehensive and automatable pipeline for clinical time-series ML.

Motivation & Objective

• Address the translational gap between state-of-the-art ML methods and real-world clinical decision support by unifying complex, composite workflows in medical time-series learning.
• Overcome the engineering burden in clinical ML projects, where over 95% of effort is spent on software implementation rather than scientific inquiry.
• Provide a standardized, realistic benchmarking environment to fairly evaluate component methods (e.g., imputation, prediction) in context, accounting for interdependencies.
• Enable automated pipeline configuration and hyperparameter optimization through an interface designed for autoML and optimization algorithms.
• Facilitate transparent, reproducible, and collaborative research by offering a composable, modular software toolkit with integrated pathways for clinical inference tasks.

Proposed method

• Design a modular, composable pipeline architecture that integrates data preprocessing, missing value imputation, feature selection, model prediction, uncertainty estimation, calibration, and interpretability into a single unified interface.
• Structure the pipeline around three core clinical pathways: (1) personalized prediction of outcomes (e.g., ICU admission, mortality), (2) treatment-effect estimation using counterfactual modeling, and (3) active sensing for optimal measurement timing.
• Implement standardized, end-to-end workflows using real-world datasets (e.g., MIMIC-III, UKCF, Ronald Reagan Medical Center), ensuring consistent data handling and evaluation protocols.
• Support automated hyperparameter optimization and pipeline configuration via an interface that accounts for component interdependencies and temporal distribution shifts.
• Leverage established deep learning and probabilistic models (e.g., GANs, RNNs, survival models) within the pipeline, with plug-and-play components for easy experimentation.
• Provide a public Python software repository (GitHub) with reusable, well-documented code to promote collaboration, reproducibility, and integration into clinical research pipelines.

Experimental results

Research questions

• RQ1 Can a single, unified pipeline framework effectively integrate and streamline the full lifecycle of clinical time-series ML, from data preprocessing to model interpretation?
• RQ2 To what extent does using a standardized, end-to-end pipeline improve the fairness, transparency, and reproducibility of method comparisons in medical time-series research?
• RQ3 How effective is the pipeline in enabling automated configuration and hyperparameter optimization across complex, interdependent components in real-world clinical settings?
• RQ4 Can the pipeline support diverse clinical inference tasks—personalized prediction, treatment-effect estimation, and active monitoring—within a single, consistent framework?
• RQ5 Does the integration of component interdependencies (e.g., imputation affecting prediction performance) lead to more realistic and reliable benchmarking than isolated component evaluation?

Key findings

• Clairvoyance is the first framework to demonstrate the viability of a comprehensive, structured, and automatable pipeline for clinical time-series machine learning, integrating all stages from data preprocessing to model interpretation.
• The toolkit enables significant reduction in engineering overhead, allowing researchers to focus on scientific questions rather than software plumbing, with over 95% of project time saved on infrastructure.
• Standardized, end-to-end pipelines provide a realistic benchmarking environment that reflects real-world interdependencies, leading to fairer and more transparent evaluation of new methods.
• The pipeline supports automated hyperparameter optimization and stepwise component selection, enabling efficient exploration of complex model configurations across diverse clinical tasks.
• Empirical evaluations on real-world datasets (e.g., MIMIC-III, UKCF, general ward data) show that the framework consistently supports accurate personalized prediction, treatment-effect estimation, and optimal monitoring strategies.
• The integration of uncertainty estimation, calibration, and interpretability components within the pipeline enhances model trustworthiness and clinical usability across ICU, inpatient, and outpatient settings.