[Paper Review] Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting

Rajat Sen, Hsiang‐Fu Yu+1 more|arXiv (Cornell University)|May 9, 2019

Stock Market Forecasting Methods181 citations

TL;DR

The paper introduces DeepGLO, a hybrid global-local forecaster that combines a low-rank global temporal convolution model with a local temporal convnet, enabling reliable training on highly high-dimensional time series without apriori normalization and achieving strong empirical gains.

ABSTRACT

Forecasting high-dimensional time series plays a crucial role in many applications such as demand forecasting and financial predictions. Modern datasets can have millions of correlated time-series that evolve together, i.e they are extremely high dimensional (one dimension for each individual time-series). There is a need for exploiting global patterns and coupling them with local calibration for better prediction. However, most recent deep learning approaches in the literature are one-dimensional, i.e, even though they are trained on the whole dataset, during prediction, the future forecast for a single dimension mainly depends on past values from the same dimension. In this paper, we seek to correct this deficiency and propose DeepGLO, a deep forecasting model which thinks globally and acts locally. In particular, DeepGLO is a hybrid model that combines a global matrix factorization model regularized by a temporal convolution network, along with another temporal network that can capture local properties of each time-series and associated covariates. Our model can be trained effectively on high-dimensional but diverse time series, where different time series can have vastly different scales, without a priori normalization or rescaling. Empirical results demonstrate that DeepGLO can outperform state-of-the-art approaches; for example, we see more than 25% improvement in WAPE over other methods on a public dataset that contains more than 100K-dimensional time series.

Motivation & Objective

• Motivate forecasting for extremely high-dimensional time series where global patterns coexist with local dynamics.
• Propose a scalable, normalization-free training approach for deep models handling diverse time-series scales.
• Develop a global temporal convolution matrix factorization (TCN-MF) and a hybrid model that incorporates global outputs as covariates for local prediction.
• Demonstrate superior performance against state-of-the-art baselines on multiple large real-world datasets, including very-high-dimensional series.
• Provide practical training algorithms and initialization techniques to facilitate reliable optimization on diverse datasets.

Proposed method

• LeveledInit: a simple initialization for Temporal Convolution Networks (TCN) that enables training without prior normalization by initializing filters to 1/k and biases to 0.
• TCN-MF: a low-rank matrix factorization of Y^(tr) = F X^(tr) regularized by a Temporal Convolution Network T_X to impose temporal structure on X^(tr) and forecast X^(te).
• Temporal regularization: uses R(X^(tr) | T_X) = (1/|J|) L2( X(:, J), T_X( X(:, J-1) ) ) to encourage basis time-series to follow learned temporal patterns.
• DeepGLO: a two-stage hybrid model where the global model’s predictions are fed as covariates into a local T_Y network, combining global patterns with local series and covariates for forecasting.
• Training: alternating optimization between F, X^(tr) with fixed T_X, and updating T_X via mini-batch SGD; prediction allows rolling and multi-step ahead forecasts without retraining.
• Algorithmic details: Algorithm 1 (LeveledInit TCN training), Algorithm 2 (TCN-MF training with alternating optimization), Algorithm 3 (training F and X^(tr) given T_X), Algorithm 4 (DeepGLO end-to-end training via covariate augmentation).

Experimental results

Research questions

• RQ1 Can a global low-rank representation capture shared temporal patterns across millions of time-series without explicit normalization?
• RQ2 Does incorporating a global temporal model as covariates improve per-series forecasting accuracy compared to purely local models?
• RQ3 Can a simple LeveledInit initialization enable reliable training of temporal convolution networks on highly diverse data scales without normalization?
• RQ4 How does DeepGLO perform against state-of-the-art baselines on large-scale, high-dimensional datasets (e.g., wiki with >100K time-series) for rolling/prediction tasks?

Key findings

• DeepGLO outperforms state-of-the-art baselines on four real-world datasets in rolling prediction tasks.
• The approach works on datasets with very large dimensionality, including a public wiki dataset exceeding 100K time-series, with notable improvements in forecast accuracy.
• LeveledInit enables reliable training of TCNs without apriori normalization across diverse time-series scales.
• The global TCN-MF component captures global temporal patterns via a low-rank factorization, while the local TCN captures per-series dynamics, and the hybrid DeepGLO effectively combines both.
• The model supports rolling predictions and multi-step look-ahead without retraining, unlike some competing methods.