[Paper Review] SACDNet: Towards Early Type 2 Diabetes Prediction with Uncertainty for Electronic Health Records

Tayyab Nasir, Muhammad Kamran Malik|arXiv (Cornell University)|Jan 12, 2023

Artificial Intelligence in Healthcare4 citations

TL;DR

This paper proposes SACDNet, a deep learning model combining multi-headed self-attention and dense layers to predict Type 2 Diabetes Mellitus (T2DM) from electronic health records (EHRs), achieving 89.3% accuracy and 89.1% F1-score. It enhances reliability by integrating Monte Carlo Dropout for uncertainty quantification, enabling the model to flag low-confidence predictions, and introduces a large, diverse real-world EHR dataset of 185,891 patients to support robust evaluation and fairness analysis.

ABSTRACT

Type 2 diabetes mellitus (T2DM) is one of the most common diseases and a leading cause of death. The problem of early diagnosis of T2DM is challenging and necessary to prevent serious complications. This study proposes a novel neural network architecture for early T2DM prediction using multi-headed self-attention and dense layers to extract features from historic diagnoses, patient vitals, and demographics. The proposed technique is called the Self-Attention for Comorbid Disease Net (SACDNet), achieving an accuracy of 89.3% and an F1-Score of 89.1%, having a 1.6% increased accuracy and 1.3% increased f1-score compared to the baseline techniques. Monte Carlo (MC) Dropout is applied to the SACDNet to get a bayesian approximation. A T2DM prediction framework based on the MC Dropout SACDNet is proposed to quantize the uncertainty associated with the predictions. A T2DM prediction dataset is also built as part of this study which is based on real-world routine Electronic Health Record (EHR) data comprising 4,124 diabetic and 181,767 non-diabetic examples, collected from 295 different EHR systems running in different parts of the United States of America. This dataset is further used to evaluate 7 different machine learning and 3 deep learning-based models. Finally, a detailed analysis of the fairness of every technique against different patient demographic groups is performed to validate the unbiased generalization of the techniques and the diversity of the data.

Motivation & Objective

• To develop a deep learning model for early prediction of Type 2 Diabetes Mellitus (T2DM) using real-world electronic health record (EHR) data.
• To address the limitations of existing datasets, such as small size, lack of diversity, and non-routine features, by constructing a large, real-world EHR-based T2DM dataset from 295 U.S. EHR systems.
• To improve model reliability by integrating uncertainty estimation into predictions using Monte Carlo Dropout and entropy-based confidence scoring.
• To evaluate model fairness across demographic groups (age, gender, race) to ensure unbiased generalization and validate dataset diversity.
• To provide a deployable framework for integration into existing EHR systems that supports early diagnosis and reduces overconfident mispredictions.

Proposed method

• SACDNet employs multi-headed self-attention mechanisms to extract relational patterns from historical diagnoses in EHR data.
• Dense feedforward layers process patient vitals and demographic features, capturing non-sequential patterns in these tabular inputs.
• The model fuses attention-based and dense features through shared fully connected layers to produce a final T2DM prediction.
• Monte Carlo Dropout is applied during inference to approximate Bayesian uncertainty, enabling probabilistic confidence estimation.
• Uncertainty is quantified using entropy of output probabilities, allowing the model to flag low-confidence predictions as uncertain.
• A novel EHR-based T2DM dataset is constructed from 295 real-world systems, containing 4,124 diabetic and 181,767 non-diabetic cases, with rigorous preprocessing to ensure clinical relevance and diversity.

Figure 1: Proposed framework for T2DM prediction with uncertainty.

Experimental results

Research questions

• RQ1 Can a deep learning model using multi-headed self-attention and dense layers outperform baseline models in early T2DM prediction using real-world EHR data?
• RQ2 To what extent does integrating Monte Carlo Dropout with SACDNet improve prediction reliability by quantifying uncertainty?
• RQ3 How does the proposed model perform across diverse demographic groups (age, gender, race), and does the dataset support fair generalization?
• RQ4 Can the proposed framework effectively identify and abstain from making high-confidence predictions on uncertain cases, reducing overconfident errors?
• RQ5 Does the constructed EHR-based dataset offer sufficient volume and diversity to support robust and unbiased model training and evaluation?

Key findings

• SACDNet achieved 89.3% accuracy and 89.1% F1-score, representing a 1.6% improvement in accuracy and 1.3% in F1-score over baseline models.
• The MC-Dropout-enhanced SACDNet (MC-SACDNet) identified more misclassified examples as uncertain by assigning them higher entropy, improving reliability.
• The model demonstrated consistent performance across all racial and gender groups, with no significant bias observed in evaluation metrics.
• Performance drops in children and preschoolers were due to data scarcity (only non-diabetic cases), not model bias, as accuracy remained high for these groups.
• The uncertainty analysis confirmed that MC-SACDNet assigns significantly higher entropy to misclassified examples than the standard SACDNet, reducing overconfident errors.
• The proposed EHR-based dataset of 185,891 patients from 295 real-world systems supports diverse, representative, and generalizable model training and evaluation.

Figure 2: Results of 5 folds cross-validation.