[Paper Review] AttenGluco: Multimodal Transformer-Based Blood Glucose Forecasting on AI-READI Dataset

Ebrahim Farahmand, Reza Rahimi Azghan+7 more|ArXiv.org|Feb 14, 2025

Artificial Intelligence in Healthcare3 citations

TL;DR

AttenGluco is a multimodal Transformer model that fuses CGM and activity data via cross-attention and multi-scale attention to forecast blood glucose over long horizons, outperforming a multimodal LSTM baseline on AI-READI data. It also analyzes cohort generalization and forgetting behavior.

ABSTRACT

Diabetes is a chronic metabolic disorder characterized by persistently high blood glucose levels (BGLs), leading to severe complications such as cardiovascular disease, neuropathy, and retinopathy. Predicting BGLs enables patients to maintain glucose levels within a safe range and allows caregivers to take proactive measures through lifestyle modifications. Continuous Glucose Monitoring (CGM) systems provide real-time tracking, offering a valuable tool for monitoring BGLs. However, accurately forecasting BGLs remains challenging due to fluctuations due to physical activity, diet, and other factors. Recent deep learning models show promise in improving BGL prediction. Nonetheless, forecasting BGLs accurately from multimodal, irregularly sampled data over long prediction horizons remains a challenging research problem. In this paper, we propose AttenGluco, a multimodal Transformer-based framework for long-term blood glucose prediction. AttenGluco employs cross-attention to effectively integrate CGM and activity data, addressing challenges in fusing data with different sampling rates. Moreover, it employs multi-scale attention to capture long-term dependencies in temporal data, enhancing forecasting accuracy. To evaluate the performance of AttenGluco, we conduct forecasting experiments on the recently released AIREADI dataset, analyzing its predictive accuracy across different subject cohorts including healthy individuals, people with prediabetes, and those with type 2 diabetes. Furthermore, we investigate its performance improvements and forgetting behavior as new cohorts are introduced. Our evaluations show that AttenGluco improves all error metrics, such as root mean square error (RMSE), mean absolute error (MAE), and correlation, compared to the multimodal LSTM model. AttenGluco outperforms this baseline model by about 10% and 15% in terms of RMSE and MAE, respectively.

Motivation & Objective

• Motivate accurate long-term blood glucose forecasting for type 2 diabetes using multimodal data (CGM + activity).
• Develop a Transformer-based architecture capable of fusing heterogeneous time-series with different sampling rates.
• Evaluate performance across diverse cohorts (healthy, prediabetes, T2DM on oral meds, T2DM on insulin) and under continual learning scenarios.
• Analyze robustness across prediction horizons and investigate forgetting when new cohorts are introduced.

Proposed method

• Propose AttenGluco, a Transformer-based forecasting model using two attention mechanisms: cross-attention to fuse CGM with activity signals, and multi-scale attention to capture temporal dependencies at multiple downsampling rates.
• Process inputs through embedding and positional encoding, then apply two cross-attention branches (CGM queries with WS and WI as keys/values).
• Combine CA outputs, downsample into three scales (factors 1, 2, 4), and apply multi-scale attention to capture both fine- and long-term patterns.
• Use a shared Add & Norm and feedforward network, culminating in a linear projection to predict m CGM values (every 5 minutes, horizon m).
• Train and evaluate on the AI-READI flagship dataset with baseline comparison to a 1D-CNN + LSTM multimodal model.
• Provide an algorithmic workflow (Algorithm 1) detailing data preprocessing, CA, MA, and output generation.

Experimental results

Research questions

• RQ1 Can AttenGluco improve RMSE, MAE, and correlation over a multimodal LSTM baseline across healthy, prediabetes, and diabetes cohorts?
• RQ2 How does cross-attention and multi-scale attention contribute to learning from irregularly sampled multimodal time-series in blood glucose forecasting?
• RQ3 What is AttenGluco’s robustness when forecasting at short (5 min) vs longer (30, 60 min) horizons, and how does performance change with cohort-specific fine-tuning?
• RQ4 Does continual fine-tuning on new cohorts lead to forgetting of previously learned cohorts, and how does AttenGluco compare to the baseline in this regard?

Key findings

• AttenGluco outperforms the baseline across all cohorts and metrics, with RMSE reductions around 10–13% in isolated-subject and cohort-wise fine-tuning scenarios.
• In the cohort-wise fine-tuning setup, AttenGluco achieves higher accuracy across healthy, pre-T2DM, oral, and insulin cohorts (e.g., Healthy RMSE 15.45 vs 17.79; Insulin RMSE 25.04 vs 28.22).
• Across prediction horizons, AttenGluco maintains more stable RMSE as horizon increases (5, 30, 60 minutes) compared to baseline.
• Continual fine-tuning on new cohorts reduces forgetting relative to the baseline, with AttenGluco showing lower test errors on new cohorts while retaining prior knowledge better.
• AttenGluco demonstrates strongest gains at longer horizons (60 minutes) and across diverse subject cohorts, indicating improved long-term forecasting capabilities.