[Paper Review] How Big Should Your Data Really Be? Data-Driven Newsvendor: Learning One Sample at a Time

Omar Besbes, Omar Mouchtaki|arXiv (Cornell University)|Jul 6, 2021

Risk and Portfolio OptimizationDecision Sciences20 citations

TL;DR

This paper provides the first finite-sample exact analysis of the Sample Average Approximation (SAA) algorithm in the data-driven newsvendor problem, revealing that tens of samples are sufficient for strong performance, while more data can degrade SAA's worst-case out-of-sample performance. It further derives a minimax-optimal policy that significantly improves performance over SAA, especially with limited data, and characterizes the exact rate of convergence of the minimax regret with a closed-form multiplicative constant.

ABSTRACT

We study the classical newsvendor problem in which the decision-maker must trade-off underage and overage costs. In contrast to the typical setting, we assume that the decision-maker does not know the underlying distribution driving uncertainty but has only access to historical data. In turn, the key questions are how to map existing data to a decision and what type of performance to expect as a function of the data size. We analyze the classical setting with access to past samples drawn from the distribution (e.g., past demand), focusing not only on asymptotic performance but also on what we call the transient regime of learning, i.e., performance for arbitrary data sizes. We evaluate the performance of any algorithm through its worst-case relative expected regret, compared to an oracle with knowledge of the distribution. We provide the first finite sample exact analysis of the classical Sample Average Approximation (SAA) algorithm for this class of problems across all data sizes. This allows to uncover novel fundamental insights on the value of data: it reveals that tens of samples are sufficient to perform very efficiently but also that more data can lead to worse out-of-sample performance for SAA. We then focus on the general class of mappings from data to decisions without any restriction on the set of policies and derive an optimal algorithm (in the minimax sense) as well as characterize its associated performance. This leads to significant improvements for limited data sizes, and allows to exactly quantify the value of historical information.

Motivation & Objective

• To understand the performance of data-driven policies across all data sizes, not just asymptotically.
• To analyze the worst-case relative regret of the SAA algorithm for arbitrary sample sizes in the newsvendor problem.
• To derive an optimal data-driven policy in the minimax sense and quantify its performance and the value of data.
• To characterize the exact rate of convergence of the minimax regret and determine whether SAA is optimal at the multiplicative constant level.

Proposed method

• Proposes a finite-sample analysis of the SAA algorithm by deriving the exact worst-case relative regret as a function of data size n.
• Introduces a minimax-optimization framework over all data-to-decision mappings to derive the optimal policy for any given data size.
• Derives a closed-form expression for the minimax optimal policy π_cvx(k,γ) that minimizes worst-case regret across all distributions.
• Uses numerical simulations with M=10^5 Monte Carlo repetitions to estimate the number of samples required to achieve target relative regret levels.
• Characterizes the exact rate of convergence of the minimax regret, including the multiplicative constant, using exact finite-sample analysis.
• Compares SAA and the minimax-optimal policy across multiple distributions (Bernoulli, Uniform, Exponential, Log-normal, Pareto) under worst-case and mild distributional assumptions.

Experimental results

Research questions

• RQ1 What is the exact worst-case performance of the SAA algorithm for any finite sample size in the newsvendor problem?
• RQ2 Does more data always improve the out-of-sample performance of SAA, or can it degrade performance?
• RQ3 What is the optimal data-driven policy in the minimax sense, and how does it improve upon SAA for limited data?
• RQ4 What is the exact rate of convergence of the minimax regret, and is SAA optimal at the multiplicative constant level?
• RQ5 How does the value of data vary across different data sizes and distributional assumptions?

Key findings

• The SAA algorithm achieves strong performance with as few as 10–20 samples, indicating that limited data can be highly effective in practice.
• For the worst-case Bernoulli distribution, SAA requires 210 samples to achieve a 5% relative regret, but more data can lead to worse worst-case performance.
• The minimax-optimal policy consistently outperforms SAA across all distributions and data sizes, with up to 30% fewer samples needed to achieve the same regret target.
• The minimax-optimal policy achieves a worst-case relative regret of 10% with only 14 samples under the worst-case Bernoulli distribution, compared to 21 for SAA.
• The exact rate of convergence of the minimax regret is characterized with a semi-closed-form expression for the multiplicative constant, revealing that SAA is rate-optimal but not constant-optimal.
• The study demonstrates that robustification of SAA through minimax optimization yields significant performance gains, especially in the transient regime of limited data.