[Paper Review] Forecasting of Non-Stationary Sales Time Series Using Deep Learning

Bohdan M. Pavlyshenko|arXiv (Cornell University)|May 23, 2022

Forecasting Techniques and Applications4 citations

TL;DR

This paper proposes a deep learning model with a dedicated trend correction block to improve forecasting accuracy for non-stationary sales time series. The model jointly predicts sales and the weight of a time trend term (normalized time × predicted weight), which is added to the forecasted sales; results show a significant RMSE reduction from 1076 to 943 on the full dataset, with even greater improvements on individual stores.

ABSTRACT

The paper describes the deep learning approach for forecasting non-stationary time series with using time trend correction in a neural network model. Along with the layers for predicting sales values, the neural network model includes a subnetwork block for the prediction weight for a time trend term which is added to a predicted sales value. The time trend term is considered as a product of the predicted weight value and normalized time value. The results show that the forecasting accuracy can be essentially improved for non-stationary sales with time trends using the trend correction block in the deep learning model.

Motivation & Objective

• Address forecasting bias in non-stationary sales time series caused by unmodeled time trends.
• Improve prediction accuracy for short or highly non-stationary sales data, especially for new products or stores.
• Develop a deep learning architecture that dynamically learns and corrects for intrinsic time trends per data group (e.g., per store).
• Evaluate the effectiveness of trend correction in a deep learning framework using real-world sales data.

Proposed method

• Integrate a subnetwork block into a deep neural network to predict the weight of a time trend term.
• Model the time trend as the product of the predicted weight and normalized time (e.g., scaled to [0,1] over the sequence).
• Concatenate embeddings of categorical features (Store, Customers) and numerical inputs before feeding into the main network.
• Use ReLU-activated fully connected layers with dropout for regularization in both the main sales prediction and trend weight prediction branches.
• Combine predicted sales and time trend term in the loss function to jointly optimize both components.
• Train the model using PyTorch with standard normalization of numerical features and one-hot or embedding encoding for categorical variables.

Experimental results

Research questions

• RQ1 Can a deep learning model that jointly predicts sales and time trend weights improve forecasting accuracy for non-stationary sales time series?
• RQ2 How does the inclusion of a dedicated trend correction block affect RMSE performance compared to a standard deep learning model?
• RQ3 To what extent does the trend correction block reduce forecasting bias in non-stationary data with varying trend directions (e.g., ascending vs. descending) across stores?
• RQ4 Does the model generalize well across different stores with distinct intrinsic time trends?

Key findings

• The model with the trend correction block achieved an overall RMSE of 943, compared to 1076 for the model without it, representing a 12.3% relative improvement.
• On individual stores with strong time trends, the improvement was substantial: RMSE dropped from 3699 to 1753 (52.5% reduction) and from 2515 to 572 (77.3% reduction).
• The trend correction block significantly reduced forecasting bias in non-stationary data, particularly in cases with divergent trends across stores.
• Feature importance analysis via permutation showed that the model effectively leveraged temporal and categorical features for accurate trend and sales prediction.
• The training and validation loss curves (Figures 11 and 12) indicated better generalization and reduced overfitting with the trend correction block.
• The model successfully learned distinct trend weights per store, enabling optimized trend correction for heterogeneous sales dynamics.