[Paper Review] N-BEATS: Neural basis expansion analysis for interpretable time series forecasting

Boris N. Oreshkin, Dmitri Carpov+2 more|arXiv (Cornell University)|May 24, 2019

Forecasting Techniques and Applications33 references517 citations

TL;DR

Introduces N-BEATS, a pure deep learning univariate time series forecasting architecture with backward/forward residuals and a doubly residual stack, achieving state-of-the-art results on M3, M4, and tourism datasets while offering interpretable outputs.

ABSTRACT

We focus on solving the univariate times series point forecasting problem using deep learning. We propose a deep neural architecture based on backward and forward residual links and a very deep stack of fully-connected layers. The architecture has a number of desirable properties, being interpretable, applicable without modification to a wide array of target domains, and fast to train. We test the proposed architecture on several well-known datasets, including M3, M4 and TOURISM competition datasets containing time series from diverse domains. We demonstrate state-of-the-art performance for two configurations of N-BEATS for all the datasets, improving forecast accuracy by 11% over a statistical benchmark and by 3% over last year's winner of the M4 competition, a domain-adjusted hand-crafted hybrid between neural network and statistical time series models. The first configuration of our model does not employ any time-series-specific components and its performance on heterogeneous datasets strongly suggests that, contrarily to received wisdom, deep learning primitives such as residual blocks are by themselves sufficient to solve a wide range of forecasting problems. Finally, we demonstrate how the proposed architecture can be augmented to provide outputs that are interpretable without considerable loss in accuracy.

Motivation & Objective

• Demonstrate that a pure deep learning architecture can outperform traditional statistical methods on univariate time series forecasting without time-series-specific features.
• Propose a deep architecture with backward and forward residuals organized in stacks that is fast to train and scalable across domains.
• Show that the architecture can produce interpretable outputs (trend and seasonality) without sacrificing accuracy.
• Evaluate performance on diverse datasets (M3, M4, tourism) to establish generalization across domains.

Proposed method

• Propose N-BEATS building blocks: fully-connected networks predicting forward (theta^f) and backward (theta^b) expansion coefficients.
• Use basis layers g^f and g^b to map expansion coefficients to forecasts y_hat and backcasts x_hat via basis vectors v^f and v^b.
• Organize blocks into stacks with doubly residual connections: x_l = x_{l-1} - x_hat_{l-1}, y_hat = sum over l of y_hat_l.
• Provide two configurations: a generic DL version (no TS-specific bias) and an interpretable version with inductive biases (trend as polynomial, seasonality as Fourier basis).
• Aggregate partial forecasts across stacks to form the final forecast y_hat; include an ensemble of models trained on varied inputs, metrics, and random initializations.
• Training: train one model per horizon per dataset; use Adam; cross-validated hyperparameters; early stopping; 1024-batch sampling from history windows of length 2H..7H.

Experimental results

Research questions

• RQ1 Can pure deep learning, without TS-specific feature engineering, achieve state-of-the-art forecast accuracy on diverse univariate time series datasets?
• RQ2 Can an architecture be designed to produce interpretable outputs (trend and seasonality) without sacrificing predictive performance?
• RQ3 What is the impact of stacking and residual connections on training stability and generalization in deep TS models?
• RQ4 How does an ensemble of N-BEATS models compare to classical statistical methods and hybrid approaches on benchmark datasets?

Key findings

• N-BEATS configurations achieve state-of-the-art performance on M3, M4, and tourism datasets, with significant improvements over statistical benchmarks and the M4 winner.
• A generic N-BEATS model (no TS-specific components) performs strongly across heterogeneous datasets, challenging the view that deep learning requires hand-crafted features for forecasting.
• An interpretable configuration yields distinct, human-like components (trend and seasonality) with the trend component monotonic and seasonality periodic, while maintaining competitive accuracy.
• Ensembling 180 models across horizons, metrics, and input window lengths yields robust performance and serves as a strong regularization technique.
• The architecture can be trained with shared weights across horizons and datasets, demonstrating generalization and scalability across diverse TS settings.