[Paper Review] Comparison of Deep learning models on time series forecasting : a case study of Dissolved Oxygen Prediction

Hongqian Qin|arXiv (Cornell University)|Nov 17, 2019

Hydrological Forecasting Using AI47 references19 citations

TL;DR

This study compares multi-step deep learning models—LSTM, GRU, BiRNN, TCN, and CNN—for forecasting dissolved oxygen (DO) levels in the Yangtze River using high-frequency real-time data from 2012–2016. Using walk-forward validation, GRU outperforms all other models in RMSE, MAE, and R² across all time steps, with prediction accuracy declining non-linearly after 2.5–4 hours.

ABSTRACT

Deep learning has achieved impressive prediction performance in the field of sequence learning recently. Dissolved oxygen prediction, as a kind of time-series forecasting, is suitable for this technique. Although many researchers have developed hybrid models or variant models based on deep learning techniques, there is no comprehensive and sound comparison among the deep learning models in this field currently. Plus, most previous studies focused on one-step forecasting by using a small data set. As the convenient access to high-frequency data, this paper compares multi-step deep learning forecasting by using walk-forward validation. Specifically, we test Convolutional Neural Network (CNN), Temporal Convolutional Network (TCN), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Bidirectional Recurrent Neural Network (BiRNN) based on the real-time data recorded automatically at a fixed observation point in the Yangtze River from 2012 to 2016. By comparing the average accumulated statistical metrics of root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination in each time step, We find for multi-step time series forecasting, the average performance of each time step does not decrease linearly. GRU outperforms other models with significant advantages.

Motivation & Objective

• To conduct a comprehensive, fair comparison of deep learning models for multi-step time series forecasting of dissolved oxygen (DO) in rivers.
• To evaluate model performance using walk-forward validation on real, high-frequency DO data from 2012–2016.
• To investigate whether prediction accuracy degrades linearly over time steps or exhibits non-linear decay.
• To identify the most robust deep learning architecture for univariate DO forecasting under real-world data conditions.

Proposed method

• Employed walk-forward validation with an expanding window to simulate real-time forecasting, using 5-hour historical DO data to predict the next 5 hours.
• Trained and evaluated five deep learning models: CNN, TCN, LSTM, GRU, and BiRNN on the same dataset and experimental setup.
• Used root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R²) as primary evaluation metrics.
• Applied data preprocessing to remove zero values and addressed data quality issues, though extreme outliers remained unresolved.
• Optimized hyperparameters via empirical tuning, selecting large batch sizes, small units, and filters to improve training speed.
• Computed average performance metrics across all time steps (1 to 10) to assess multi-step forecasting reliability.

Experimental results

Research questions

• RQ1 Which deep learning model achieves the highest forecasting accuracy for multi-step dissolved oxygen prediction in real-time river monitoring?
• RQ2 How does prediction performance degrade over time steps, and is the decay linear or non-linear?
• RQ3 Does GRU outperform LSTM and other RNN variants in capturing long-term dependencies in DO time series?
• RQ4 How do different architectures (CNN, TCN, RNNs) compare in handling high-frequency, non-stationary water quality data?

Key findings

• GRU achieved the lowest average RMSE and MAE and the highest average R² across all time steps, indicating superior overall performance.
• LSTM had the highest RMSE and MAE and the lowest R², making it the least accurate model among the six tested.
• Prediction accuracy for all models declined non-linearly, with a sharp drop in performance after the 2.5-hour and 4-hour time steps.
• The performance decay was not uniform, suggesting that error rates increase abruptly beyond certain time horizons, especially after 2.5 hours.
• The results were robust across different hyperparameter configurations, indicating consistent model ranking regardless of tuning choices.
• Despite data quality issues such as extreme values and measurement errors, deep learning models, especially GRU, showed resilience in handling noisy, real-world time series.