[Paper Review] Deep Factors for Forecasting

Yuyang Wang, Alex Smola+4 more|arXiv (Cornell University)|May 28, 2019

Time Series Analysis and ForecastingComputer Science49 references58 citations

TL;DR

A hybrid global-local forecasting framework that combines a global deep factor (deterministic) with local probabilistic time-series components to efficiently forecast large collections of exchangeable time series and quantify uncertainty.

ABSTRACT

Producing probabilistic forecasts for large collections of similar and/or dependent time series is a practically relevant and challenging task. Classical time series models fail to capture complex patterns in the data, and multivariate techniques struggle to scale to large problem sizes. Their reliance on strong structural assumptions makes them data-efficient, and allows them to provide uncertainty estimates. The converse is true for models based on deep neural networks, which can learn complex patterns and dependencies given enough data. In this paper, we propose a hybrid model that incorporates the benefits of both approaches. Our new method is data-driven and scalable via a latent, global, deep component. It also handles uncertainty through a local classical model. We provide both theoretical and empirical evidence for the soundness of our approach through a necessary and sufficient decomposition of exchangeable time series into a global and a local part. Our experiments demonstrate the advantages of our model both in term of data efficiency, accuracy and computational complexity.

Motivation & Objective

• Motivate forecasting for large collections of related time series with dependencies and uncertainty quantification.
• Propose a global-local model that combines deep neural network factors with local probabilistic components.
• Develop scalable inference that handles non-Gaussian likelihoods and enables parallel per-series processing.
• Show data efficiency and accuracy gains over existing methods on synthetic and real datasets.

Proposed method

• Formulate exchangeable time series with a global latent series g and local time series z_i conditioned on g (de Finetti-based decomposition).
• Introduce Deep Factor Models with Random Effects (DFM+RE) where global factors are deterministic RNN-based g_k and local effects r_i employ Gaussian, ISSM, or GP models.
• Define the latent function u_i = f_i + r_i with f_i a fixed effect a weighted sum of global factors (f_i = sum_k w_{i,k} g_k).
• Model emissions z_{i,t} ~ p(z_{i,t}|u_i(x_{i,t})) with options for Gaussian or non-Gaussian likelihoods.
• Provide training via maximum likelihood and a scalable variational inference scheme that separates global and local computations for efficiency.
• Discuss three instantiations: DF-RNN, DF-LDS, and DF-GP, with corresponding inference (Kalman filter for DF-LDS, GP marginal likelihood for DF-GP).

Experimental results

Research questions

• RQ1 Can a global latent factor structure capture shared dynamics across many time series while local components model idiosyncratic behavior?
• RQ2 How can we perform scalable inference for a global-deterministic plus local-stochastic model, especially with non-Gaussian likelihoods?
• RQ3 Do global-deep factors improve data efficiency and forecast accuracy compared to purely local or purely global models in real-world datasets?
• RQ4 Can exchangeable-time-series theory (de Finetti) justify and guide the global-local decomposition for forecasting?

Key findings

• The proposed global-local framework provides a principled decomposition of exchangeable time series into a global latent process and local series, enabling scalable modeling.
• Global factors, modeled via deterministic RNNs and loadings, improve data efficiency and computation compared to standard RNN forecasters.
• Local effects can be modeled with Gaussian noise, ISSMs, or GPs, allowing exact or variational marginal likelihoods and enabling uncertainty propagation.
• Empirical results on synthetic and real datasets show the DF model can outperform benchmarks such as DeepAR, MQ-RNN, and Prophet in accuracy and variability under limited training data.