[Paper Review] Multivariate Arrival Times with Recurrent Neural Networks for Personalized Demand Forecasting

Tianle Chen, Brian Keng+1 more|arXiv (Cornell University)|Dec 29, 2018

Forecasting Techniques and ApplicationsDecision Sciences21 references3 citations

TL;DR

This paper proposes MAT-RNN, a novel recurrent neural network model that jointly forecasts multivariate inter-arrival times for personalized demand by modeling distributional parameters of inter-purchase times via negative log-likelihood loss, enabling effective handling of censored and partially observed data. The method achieves state-of-the-art performance in ROC-AUC across multiple product categories, with improvements up to +0.1098 on individual SKUs.

ABSTRACT

Access to a large variety of data across a massive population has made it possible to predict customer purchase patterns and responses to marketing campaigns. In particular, accurate demand forecasts for popular products with frequent repeat purchases are essential since these products are one of the main drivers of profits. However, buyer purchase patterns are extremely diverse and sparse on a per-product level due to population heterogeneity as well as dependence in purchase patterns across product categories. Traditional methods in survival analysis have proven effective in dealing with censored data by assuming parametric distributions on inter-arrival times. Distributional parameters are then fitted, typically in a regression framework. On the other hand, neural-network based models take a non-parametric approach to learn relations from a larger functional class. However, the lack of distributional assumptions make it difficult to model partially observed data. In this paper, we model directly the inter-arrival times as well as the partially observed information at each time step in a survival-based approach using Recurrent Neural Networks (RNN) to model purchase times jointly over several products. Instead of predicting a point estimate for inter-arrival times, the RNN outputs parameters that define a distributional estimate. The loss function is the negative log-likelihood of these parameters given partially observed data. This approach allows one to leverage both fully observed data as well as partial information. By externalizing the censoring problem through a log-likelihood loss function, we show that substantial improvements over state-of-the-art machine learning methods can be achieved. We present experimental results based on two open datasets as well as a study on a real dataset from a large retailer.

Motivation & Objective

• To address the challenge of highly heterogeneous and sparse customer purchase patterns across products in personalized demand forecasting.
• To overcome limitations of traditional survival models that rely on parametric assumptions and linear covariates.
• To integrate the flexibility of deep learning with the statistical rigor of survival analysis for partially observed, sequential purchase data.
• To enable joint modeling of multiple products’ inter-arrival times while leveraging both fully observed and censored data.
• To improve forecasting accuracy for future purchases by modeling distributional estimates of inter-arrival times using RNNs.

Proposed method

• The model uses a sequence-to-sequence RNN architecture to predict parameters of a Weibull distribution for inter-arrival times at each time step.
• At each time step, the model outputs distribution parameters (scale and shape) for the time until next purchase, conditioned on historical purchase sequences.
• The loss function is the negative log-likelihood of the observed or censored inter-arrival times, enabling end-to-end training on partially observed data.
• The model explicitly handles censored data by computing conditional survival probabilities and densities for incomplete inter-arrival times.
• The approach externalizes censoring through likelihood maximization, allowing the RNN to learn complex temporal dependencies without assuming parametric forms for the underlying hazard.
• The model is trained using a likelihood-based objective that combines both complete and incomplete observations, improving generalization on sparse data.

Experimental results

Research questions

• RQ1 Can a deep learning model jointly forecast inter-arrival times across multiple products while effectively modeling censored and partially observed data?
• RQ2 Does modeling distributional parameters of inter-arrival times via RNNs lead to better performance than point-estimate regression or standard RNNs in survival forecasting?
• RQ3 How does joint modeling of multiple products compare to individual product modeling in terms of predictive performance and parameter efficiency?
• RQ4 To what extent does the likelihood-based training objective improve performance on sparse, real-world retail data compared to standard squared-error loss?
• RQ5 Can the integration of survival analysis principles with RNNs yield significant gains in ROC-AUC for next-purchase prediction across diverse product categories?

Key findings

• MAT-RNN outperforms state-of-the-art machine learning methods in ROC-AUC, achieving improvements ranging from +0.0029 to +0.1098 on individual product SKUs.
• The model achieves superior performance on 4 out of 5 product categories in the real retailer dataset, with significant gains in AUC for high-impact SKUs.
• Joint modeling of multiple products consistently improves performance over single-product models, even with fewer parameters.
• The model shows comparable or better performance than traditional survival models and standard RNNs trained with squared-error loss on benchmark and synthetic datasets.
• The use of negative log-likelihood loss enables effective learning from both complete and censored observations, leading to robust performance on sparse data.
• The model demonstrates strong scalability potential for targeted advertising, though inventory-wide extension remains a future challenge due to memory and ETL constraints.