[Paper Review] Recurrent Neural Networks for Time Series Forecasting

Gábor Petneházi|arXiv (Cornell University)|Jan 1, 2019

Time Series Analysis and ForecastingComputer Science40 references76 citations

TL;DR

The paper proposes an end-to-end RNN-based framework for time series forecasting, emphasizing feature engineering, feature importance, point and interval predictions, and empirical comparison of LSTM and GRU models with bootstrapped prediction intervals.

ABSTRACT

Time series forecasting is difficult. It is difficult even for recurrent neural networks with their inherent ability to learn sequentiality. This article presents a recurrent neural network based time series forecasting framework covering feature engineering, feature importances, point and interval predictions, and forecast evaluation. The description of the method is followed by an empirical study using both LSTM and GRU networks.

Motivation & Objective

• Provide an end-to-end framework for forecasting time series using recurrent neural networks (RNNs).
• Illustrate how feature engineering and feature importances can improve RNN forecasts.
• Develop point forecasts and prediction intervals for time series using bootstrapping and ensemble ideas.
• Empirically compare LSTM and GRU architectures on a real dataset and analyze performance and uncertainties.

Proposed method

• Describe an RNN-based forecasting framework that integrates feature engineering (lags, trend, seasonality, dummy indicators) and normalization.
• Use permutation-based feature importance (mean decrease accuracy) to assess input relevance for RNNs.
• Support one-step and iterative multi-step forecasts, with bootstrapped ensembles to obtain prediction intervals.
• Construct prediction intervals via bootstrapping by training bootstrap models and averaging predictions; estimate residual variance with a secondary network.
• Validate using regression and directional accuracy metrics, plus interval evaluation metrics (PICP, MPIW, NMPIW, CWC).
• Compare one-layer LSTM and GRU models with 32 units, trained with MSE loss and Adam optimizer on a Bike Sharing dataset.

Experimental results

Research questions

• RQ1 Can an end-to-end RNN forecasting framework improve accuracy for time series data?
• RQ2 What input features (lags, seasonality, trends, holidays) most strongly influence RNN forecasts?
• RQ3 How do LSTM and GRU compare in one-step and multi-step time series forecasting tasks?
• RQ4 Can bootstrapped ensembles provide reliable prediction intervals for neural network forecasts?
• RQ5 Do feature importances align with intuitive drivers of the time series?

Key findings

• LSTM and GRU models show similar forecasting performance on the studied dataset.
• Recent lag features and intraday seasonality (hour of day, holidays/working hours) are among the most important predictors.
• One-step forecasts achieve higher accuracy (R^2 around 0.95; RMSE ~31-33) than multi-step forecasts (R^2 around 0.8; RMSE ~90-105).
• Bootstrapped bagging improves predictive accuracy over individual models and yields broader but well-calibrated prediction intervals (often exceeding the nominal 90% coverage).
• Prediction intervals constructed via bootstrapping demonstrate higher coverage than nominal targets, with interval widths indicating uncertainty grows for multi-step forecasts.