[Paper Review] Profit-oriented sales forecasting: a comparison of forecasting techniques from a business perspective

Tine Van Calster, Filip Van den Bossche+2 more|arXiv (Cornell University)|Feb 3, 2020

Forecasting Techniques and Applications4 citations

TL;DR

This paper proposes a profit-driven, automated framework for tactical sales forecasting that evaluates forecasting techniques not only by accuracy but by their expected business profit. It benchmarks 35 monthly time series using statistical, machine learning, and univariate models, finding that simple seasonal time series models consistently outperform complex methods—especially in profit and accuracy—while external variables add little value despite higher maintenance costs.

ABSTRACT

Choosing the technique that is the best at forecasting your data, is a problem that arises in any forecasting application. Decades of research have resulted into an enormous amount of forecasting methods that stem from statistics, econometrics and machine learning (ML), which leads to a very difficult and elaborate choice to make in any forecasting exercise. This paper aims to facilitate this process for high-level tactical sales forecasts by comparing a large array of techniques for 35 times series that consist of both industry data from the Coca-Cola Company and publicly available datasets. However, instead of solely focusing on the accuracy of the resulting forecasts, this paper introduces a novel and completely automated profit-driven approach that takes into account the expected profit that a technique can create during both the model building and evaluation process. The expected profit function that is used for this purpose, is easy to understand and adaptable to any situation by combining forecasting accuracy with business expertise. Furthermore, we examine the added value of ML techniques, the inclusion of external factors and the use of seasonal models in order to ascertain which type of model works best in tactical sales forecasting. Our findings show that simple seasonal time series models consistently outperform other methodologies and that the profit-driven approach can lead to selecting a different forecasting model.

Motivation & Objective

• To address the challenge of selecting optimal forecasting techniques for high-level tactical sales forecasts, where business impact extends beyond statistical accuracy.
• To integrate a practical, automated profit function into model selection and evaluation, aligning forecasting choices with real business outcomes.
• To evaluate the relative performance of statistical models, machine learning techniques, and models with external factors in a real-world sales context.
• To determine whether complex models (e.g., ML) or simpler seasonal models yield higher expected profit in sales forecasting.
• To assess the value of including external variables in univariate forecasting models, considering both accuracy and long-term maintenance costs.

Proposed method

• A novel expected profit function is introduced, which quantifies the financial impact of forecast errors by combining forecast accuracy with predefined profit margins per product.
• The profit function is embedded into model training and evaluation, enabling automated selection of models that maximize expected profit.
• A comprehensive benchmark is conducted on 35 monthly time series, including real sales data from The Coca-Cola Company and publicly available datasets.
• Models are grouped into three categories: univariate seasonal models (e.g., seasonal ARIMA, ETS), non-seasonal models, and machine learning models (e.g., XGBoost, neural networks).
• External factors are incorporated in selected models (e.g., ARIMAX, XGBoost with exogenous variables), while others remain univariate.
• Performance is evaluated using both traditional accuracy metrics (e.g., MAE, RMSE) and the profit-driven metric, with model selection guided by expected profit.

Experimental results

Research questions

• RQ1 Which forecasting technique yields the highest expected profit in tactical sales forecasting, when profit is explicitly modeled in the evaluation process?
• RQ2 Do machine learning techniques outperform traditional statistical models in terms of both accuracy and profit for high-level sales forecasts?
• RQ3 Is the inclusion of external factors justified in univariate sales forecasting, given the increased model complexity and maintenance costs?
• RQ4 How do seasonal models compare to non-seasonal models in forecasting accuracy and profit generation for monthly sales data?
• RQ5 Can a purely automated, profit-driven model selection process outperform standard accuracy-based benchmarks in real-world sales forecasting applications?

Key findings

• Simple seasonal time series models, such as seasonal ARIMA and ETS, consistently outperform all other techniques in both forecast accuracy and expected profit.
• Machine learning models, despite their popularity, deliver significantly worse results than traditional models in terms of both accuracy and profit, contradicting some prior studies.
• The inclusion of external variables does not improve forecast performance or profit generation and increases model maintenance costs, suggesting limited added value for tactical sales forecasting.
• The Seasonal ARIMAX model performs comparably to simpler univariate seasonal models but does not justify the added complexity and data requirements.
• The profit-driven evaluation framework successfully identifies different optimal models than traditional accuracy-based benchmarks, demonstrating its practical relevance.
• Univariate seasonal models are recommended for tactical sales forecasting due to their superior performance, interpretability, and computational efficiency.