[Paper Review] The Effectiveness of Discretization in Forecasting: An Empirical Study on Neural Time Series Models

Stephan Rabanser, Tim Januschowski+3 more|arXiv (Cornell University)|May 20, 2020

Time Series Analysis and Forecasting17 citations

TL;DR

This paper investigates the impact of data discretization—converting real-valued time series into categorical bins—on the forecasting performance of neural models. It finds that binning consistently improves predictive accuracy across feed-forward, recurrent, and convolutional architectures, with the choice of binning method mattering less than the transformation itself.

ABSTRACT

Time series modeling techniques based on deep learning have seen many advancements in recent years, especially in data-abundant settings and with the central aim of learning global models that can extract patterns across multiple time series. While the crucial importance of appropriate data pre-processing and scaling has often been noted in prior work, most studies focus on improving model architectures. In this paper we empirically investigate the effect of data input and output transformations on the predictive performance of several neural forecasting architectures. In particular, we investigate the effectiveness of several forms of data binning, i.e. converting real-valued time series into categorical ones, when combined with feed-forward, recurrent neural networks, and convolution-based sequence models. In many non-forecasting applications where these models have been very successful, the model inputs and outputs are categorical (e.g. words from a fixed vocabulary in natural language processing applications or quantized pixel color intensities in computer vision). For forecasting applications, where the time series are typically real-valued, various ad-hoc data transformations have been proposed, but have not been systematically compared. To remedy this, we evaluate the forecasting accuracy of instances of the aforementioned model classes when combined with different types of data scaling and binning. We find that binning almost always improves performance (compared to using normalized real-valued inputs), but that the particular type of binning chosen is of lesser importance.

Motivation & Objective

• To systematically evaluate the effect of input and output data transformations—especially discretization—on neural forecasting model performance.
• To address the lack of empirical comparison between various data pre-processing techniques in time series forecasting, particularly in data-abundant, global modeling settings.
• To determine whether discretization, commonly used in NLP and computer vision, improves forecasting accuracy in real-valued time series.
• To assess whether the choice of binning strategy (e.g., equal-width, equal-frequency, quantile-based) significantly affects model performance.
• To explore whether input and output transformations can have a larger impact on performance than architectural differences in neural forecasting models.

Proposed method

• Empirically compare multiple input and output transformation techniques, including normalization, standardization, log/Box-Cox transforms, and various binning strategies (equal-width, equal-frequency, quantile-based).
• Apply these transformations to a diverse set of neural forecasting architectures: feed-forward networks, recurrent neural networks (RNNs), and 1D dilated causal convolutional networks.
• Use probabilistic forecasting setups, including quantile regression, mixture density networks, and discretized logistic mixture likelihoods for output modeling.
• Evaluate models on multiple public time series datasets, measuring performance using standard metrics like MASE, sMAPE, and CRPS.
• Implement hybrid binning strategies that combine global (panel-wide) and local (per-time-series) binning to capture both shared and idiosyncratic patterns.
• Conduct extensive hyperparameter tuning to isolate the effect of transformations from model-specific optimization.

Experimental results

Research questions

• RQ1 Does discretizing real-valued time series inputs improve forecasting performance across diverse neural architectures?
• RQ2 How do different binning strategies (e.g., equal-width, equal-frequency, quantile-based) compare in terms of predictive accuracy?
• RQ3 Is the impact of input and output transformations on model performance comparable to or greater than architectural differences?
• RQ4 Can hybrid binning strategies that combine global and local effects improve forecasting performance over purely global or local approaches?
• RQ5 To what extent does discretization preserve predictive utility despite discarding continuous distance and order information?

Key findings

• Discretization of input time series consistently improves forecasting performance across all tested neural architectures, including feed-forward, RNN, and convolutional models.
• The choice of binning method (e.g., equal-width, equal-frequency, quantile-based) has minimal impact on performance, suggesting robustness across different binning schemes.
• Input and output transformations have a larger effect on predictive performance than differences between model architectures, indicating their critical role in forecasting pipelines.
• Even though the categorical distribution ignores continuous order and distance, it performs surprisingly well, indicating that binning captures sufficient signal for accurate forecasting.
• Hybrid binning strategies—combining global and local binning—show promise by leveraging both shared patterns and individual time series characteristics.
• The study highlights the need for further research into methods that retain the benefits of discretization while incorporating ordinal and distance information, such as discretized logistic mixture likelihoods.