[Paper Review] A Deep Learning Pipeline for Patient Diagnosis Prediction Using Electronic Health Records

Leopold Franz, Yash Raj Shrestha+1 more|arXiv (Cornell University)|Jun 23, 2020

Machine Learning in HealthcareComputer Science30 references19 citations

TL;DR

This paper presents a deep learning pipeline for multi-diagnosis prediction using electronic health records (EHRs), introducing two models—ClinicalBERT_Multi and DeepObserver—that leverage clinical notes and structured data, respectively. ClinicalBERT_Multi achieves the highest predictive performance with early diagnosis at 3 days post-admission and provides interpretable attention visualizations, while the FHIR-based data transformation tool enables standardized EHR integration across diverse datasets.

ABSTRACT

Augmentation of disease diagnosis and decision-making in healthcare with machine learning algorithms is gaining much impetus in recent years. In particular, in the current epidemiological situation caused by COVID-19 pandemic, swift and accurate prediction of disease diagnosis with machine learning algorithms could facilitate identification and care of vulnerable clusters of population, such as those having multi-morbidity conditions. In order to build a useful disease diagnosis prediction system, advancement in both data representation and development of machine learning architectures are imperative. First, with respect to data collection and representation, we face severe problems due to multitude of formats and lack of coherency prevalent in Electronic Health Records (EHRs). This causes hindrance in extraction of valuable information contained in EHRs. Currently, no universal global data standard has been established. As a useful solution, we develop and publish a Python package to transform public health dataset into an easy to access universal format. This data transformation to an international health data format facilitates researchers to easily combine EHR datasets with clinical datasets of diverse formats. Second, machine learning algorithms that predict multiple disease diagnosis categories simultaneously remain underdeveloped. We propose two novel model architectures in this regard. First, DeepObserver, which uses structured numerical data to predict the diagnosis categories and second, ClinicalBERT_Multi, that incorporates rich information available in clinical notes via natural language processing methods and also provides interpretable visualizations to medical practitioners. We show that both models can predict multiple diagnoses simultaneously with high accuracy.

Motivation & Objective

• To address the challenge of fragmented and non-interoperable electronic health record (EHR) data formats that hinder machine learning applications in healthcare.
• To develop a universal data representation framework by transforming the MIMIC-III EHR dataset into FHIR format for broader research compatibility.
• To design and evaluate novel deep learning architectures capable of predicting multiple diagnosis codes simultaneously with high accuracy.
• To enhance model interpretability for clinicians through attention visualization in the ClinicalBERT_Multi model.
• To enable early diagnosis prediction (within 3 days of ICU admission) using both clinical notes and structured numerical data.

Proposed method

• The authors developed a Python package to map the MIMIC-III EHR dataset into a flat FHIR (Fast Healthcare Interoperability Resources) format, enabling standardized, interoperable data access.
• The DeepObserver model uses a 1D convolutional neural network (CNN) to process binned numerical time-series data from the Chartevents table, predicting multiple diagnosis codes via multi-label classification.
• The ClinicalBERT_Multi model fine-tunes ClinicalBERT on clinical notes, combining embeddings from multiple text chunks using a probability combination strategy based on Equation 3 from the ClinicalBERT paper.
• Model attention weights are visualized to interpret predictions, with high attention scores on clinically relevant tokens (e.g., 'failure' in 'acute respiratory failure').
• Diagnosis prediction is performed at three time points: 3 days, 6 days, and 12 days post-admission, with early predictions evaluated for performance.
• The models are trained and evaluated on the MIMIC-III dataset, with performance measured using AU-PR and AUC metrics across multiple CCS diagnosis categories.

Experimental results

Research questions

• RQ1 Can a standardized FHIR-based data transformation pipeline improve interoperability and usability of heterogeneous EHR datasets for machine learning research?
• RQ2 Can a deep learning model that processes both structured EHR data and clinical notes achieve high accuracy in multi-diagnosis prediction?
• RQ3 Can early diagnosis prediction (within 3 days of ICU admission) be reliably achieved using deep learning on EHR data?
• RQ4 To what extent can attention visualization in ClinicalBERT_Multi improve interpretability for clinical decision support?
• RQ5 How do the performance characteristics of a CNN-based model (DeepObserver) compare to a transformer-based model (ClinicalBERT_Multi) in multi-label diagnosis prediction?

Key findings

• ClinicalBERT_Multi achieved the highest predictive performance among all models, with the highest AU-PR score, particularly excelling in early diagnosis prediction at 3 days post-admission.
• The model demonstrated strong performance in predicting multiple diagnosis codes simultaneously, leveraging rich textual information from clinical notes.
• Attention visualization revealed that the model correctly focused on clinically relevant terms such as 'failure' in 'acute respiratory failure', enhancing interpretability.
• DeepObserver achieved a high AU-PR score using only structured numerical data from the Chartevents table, without relying on clinical notes.
• The FHIR transformation package successfully converted the MIMIC-III dataset into a standardized, accessible format, facilitating integration with other clinical datasets.
• Both models showed limitations in capturing high-resolution temporal patterns due to binning of observations into 8-hour intervals, suggesting a need for higher-resolution modeling in future work.