[Paper Review] Time Series Modeling for Heart Rate Prediction: From ARIMA to Transformers

Haowei Ni, Shuchen Meng+6 more|arXiv (Cornell University)|Jun 18, 2024

Non-Invasive Vital Sign Monitoring4 citations

TL;DR

This study compares traditional time series models (ARIMA, Prophet) with deep learning architectures—including LSTM, TCN, TSMixer, and transformers—for heart rate prediction using the MIT-BIH database. Transformer-based models, especially PatchTST, significantly outperform all others in accuracy, capturing complex temporal dependencies and non-linear patterns in physiological data with lower error metrics.

ABSTRACT

Cardiovascular disease (CVD) is a leading cause of death globally, necessitating precise forecasting models for monitoring vital signs like heart rate, blood pressure, and ECG. Traditional models, such as ARIMA and Prophet, are limited by their need for manual parameter tuning and challenges in handling noisy, sparse, and highly variable medical data. This study investigates advanced deep learning models, including LSTM, and transformer-based architectures, for predicting heart rate time series from the MIT-BIH Database. Results demonstrate that deep learning models, particularly PatchTST, significantly outperform traditional models across multiple metrics, capturing complex patterns and dependencies more effectively. This research underscores the potential of deep learning to enhance patient monitoring and CVD management, suggesting substantial clinical benefits. Future work should extend these findings to larger, more diverse datasets and real-world clinical applications to further validate and optimize model performance.

Motivation & Objective

• To address the limitations of traditional time series models like ARIMA and Prophet in handling noisy, sparse, and highly variable medical time series data.
• To evaluate the performance of deep learning models—including LSTM, TCN, TSMixer, and transformer-based architectures—on heart rate forecasting from the MIT-BIH database.
• To identify the most effective model architecture for predicting heart rate dynamics with high accuracy and robustness in clinical-relevant time series settings.
• To provide empirical evidence for the superiority of transformer-based models in capturing complex temporal dependencies in physiological time series.
• To establish a benchmark for future research in biomedical time series forecasting and support clinical applications in patient monitoring and early intervention.

Proposed method

• The study uses heart rate time series data from four individuals in the MIT-BIH Arrhythmia Database, with sequences ranging from 7 to 15 minutes.
• Traditional models (ARIMA, Prophet) are applied with standard hyperparameter configurations, relying on manual tuning and seasonal decomposition.
• Deep learning models include LSTM, TCN, TSMixer, and transformer-based architectures such as PatchTST, iTransformer, and TimesNet, trained end-to-end on raw time series.
• Model evaluation is conducted using standard metrics: RMSE, MAE, and MAPE, with a train/val/test split preserving temporal order.
• The self-attention mechanism in transformer models enables dynamic weighting of temporal dependencies, improving long-range pattern recognition.
• PatchTST uses patchify-based input representation and a simplified transformer encoder, enhancing efficiency and performance on sequential data.

Figure 1 : T1 Heart Rate time series.

Experimental results

Research questions

• RQ1 How do traditional statistical models like ARIMA and Prophet perform on short, noisy heart rate time series from clinical databases?
• RQ2 To what extent do deep learning models such as LSTM, TCN, and TSMixer improve prediction accuracy over traditional models in heart rate forecasting?
• RQ3 Do transformer-based architectures like PatchTST, iTransformer, and TimesNet significantly outperform both traditional and non-transformer deep learning models in heart rate prediction?
• RQ4 What is the impact of architectural design—particularly self-attention and patch-based processing—on model performance in physiological time series forecasting?
• RQ5 Can transformer-based models generalize to complex, non-linear dynamics in real-world patient monitoring scenarios?

Key findings

• PatchTST achieved the lowest RMSE and MAE across all models, demonstrating superior performance in predicting heart rate dynamics.
• iTransformer closely followed PatchTST in accuracy, with predictions that closely matched the true values across all test sequences.
• TCN and TSMixer showed strong performance, though with slightly higher error margins than the top transformer models.
• LSTM exhibited higher variability in predictions compared to other deep learning models, yet still outperformed ARIMA and Prophet.
• Traditional models (ARIMA and Prophet) showed the highest error rates, with significant deviations from true heart rate values, especially in non-stationary or noisy segments.
• Visual comparisons confirmed that transformer-based models, particularly PatchTST and iTransformer, produced smoother and more accurate forecasts aligned with ground truth.

Figure 2 : Prophet fit over the entire T1 dataset. The actual fit is depicted in light blue, the trend function is represented by a red line, and the trend changepoints are marked by red dashed vertical lines.