[Paper Review] Forecasting Across Time Series Databases using Recurrent Neural Networks on Groups of Similar Series: A Clustering Approach

Kasun Bandara, Christoph Bergmeir+1 more|arXiv (Cornell University)|Oct 9, 2017

Stock Market Forecasting Methods4 citations

TL;DR

This paper proposes a clustering-based framework that trains Long Short-Term Memory (LSTM) networks on groups of similar time series to improve forecasting accuracy in large time series databases. By clustering similar series and training LSTMs on each subgroup, the method outperforms both global LSTM models and traditional univariate methods, achieving state-of-the-art results on the CIF2016 competition dataset and consistently improving over baseline LSTMs on NN5.

ABSTRACT

With the advent of Big Data, nowadays in many applications databases containing large quantities of similar time series are available. Forecasting time series in these domains with traditional univariate forecasting procedures leaves great potentials for producing accurate forecasts untapped. Recurrent neural networks (RNNs), and in particular Long Short-Term Memory (LSTM) networks, have proven recently that they are able to outperform state-of-the-art univariate time series forecasting methods in this context when trained across all available time series. However, if the time series database is heterogeneous, accuracy may degenerate, so that on the way towards fully automatic forecasting methods in this space, a notion of similarity between the time series needs to be built into the methods. To this end, we present a prediction model that can be used with different types of RNN models on subgroups of similar time series, which are identified by time series clustering techniques. We assess our proposed methodology using LSTM networks, a widely popular RNN variant. Our method achieves competitive results on benchmarking datasets under competition evaluation procedures. In particular, in terms of mean sMAPE accuracy, it consistently outperforms the baseline LSTM model and outperforms all other methods on the CIF2016 forecasting competition dataset.

Motivation & Objective

• To address the degradation in forecasting accuracy when applying global LSTM models to heterogeneous time series databases.
• To leverage cross-series similarity by grouping similar time series into clusters for more accurate, subgroup-specific forecasting.
• To develop a scalable, automatic framework that integrates clustering with recurrent neural networks for improved forecasting in Big Data time series applications.
• To evaluate the method on benchmark datasets under standardized competition evaluation protocols to validate performance gains.

Proposed method

• Time series are clustered using unsupervised clustering techniques to identify groups of similar series based on shape, trend, and seasonality.
• A separate LSTM model is trained for each cluster, allowing the model to learn patterns specific to similar series while benefiting from shared information within the group.
• The method uses a two-stage training process: first clustering the time series, then training individual LSTMs on each cluster’s data.
• The approach is evaluated using both fixed-origin and rolling-origin forecasting setups to ensure robustness across evaluation protocols.
• The framework is tested with LSTM variants, including a global LSTM (LSTM.All) and a cluster-based LSTM (LSTM.Cluster), to compare performance.

Experimental results

Research questions

• RQ1 Can clustering similar time series improve forecasting accuracy when using recurrent neural networks in large-scale databases?
• RQ2 How does the performance of a cluster-based LSTM model compare to a global LSTM model trained across all series?
• RQ3 Does the proposed method outperform state-of-the-art univariate forecasting methods on benchmark datasets with diverse time series?
• RQ4 Is the improvement from subgroup modeling statistically significant compared to baseline models?

Key findings

• On the CIF2016 forecasting competition dataset, the LSTM.Cluster model outperformed all other methods in terms of mean sMAPE, achieving the best rank.
• On the NN5 dataset, the LSTM.Cluster model ranked 6th overall in mean sMAPE, consistently outperforming the global LSTM_All baseline across all error metrics.
• The paired Wilcoxon signed-rank test showed that the difference between LSTM.Cluster and LSTM_All was highly significant (p = 7.22 × 10⁻⁴), confirming the benefit of clustering.
• Statistical testing using the aligned Friedman test yielded a p-value of 6.59 × 10⁻¹¹, indicating strong overall significance of the results.
• The LSTM.Cluster model did not perform significantly worse than the exponential smoothing benchmarks (ETS and ES), while the global LSTM_All model performed significantly worse than the control method.
• The method demonstrated consistent improvements in median sMAPE, mean MASE, and rank-based metrics across both fixed-origin and rolling-origin evaluations.