[Paper Review] Meta Dynamic Pricing: Learning Across Experiments

Hamsa Bastani, David Simchi‐Levi+1 more|arXiv (Cornell University)|Feb 28, 2019

Advanced Bandit Algorithms Research8 citations

TL;DR

This paper proposes a meta dynamic pricing algorithm that learns a shared prior distribution across multiple related products' demand parameters in real time, using online Thompson sampling with uncertainty-aware prior widening. It achieves sublinear meta-regret in N and significantly accelerates learning in experiment-rich environments by balancing meta-exploration and meta-exploitation.

ABSTRACT

We study the problem of learning shared structure \emph{across} a sequence of dynamic pricing experiments for related products. We consider a practical formulation where the unknown demand parameters for each product come from an unknown distribution (prior) that is shared across products. We then propose a meta dynamic pricing algorithm that learns this prior online while solving a sequence of Thompson sampling pricing experiments (each with horizon $T$) for $N$ different products. Our algorithm addresses two challenges: (i) balancing the need to learn the prior (\emph{meta-exploration}) with the need to leverage the estimated prior to achieve good performance (\emph{meta-exploitation}), and (ii) accounting for uncertainty in the estimated prior by appropriately widening the prior as a function of its estimation error, thereby ensuring convergence of each price experiment. Unlike prior-independent approaches, our algorithm's meta regret grows sublinearly in $N$; an immediate consequence of our analysis is that the price of an unknown prior in Thompson sampling is negligible in experiment-rich environments with shared structure (large $N$). Numerical experiments on synthetic and real auto loan data demonstrate that our algorithm significantly speeds up learning compared to prior-independent algorithms or a naive approach of greedily using the updated prior across products.

Motivation & Objective

• To address the challenge of learning shared demand parameter structures across a sequence of dynamic pricing experiments for related products.
• To balance meta-exploration (learning the shared prior) with meta-exploitation (using the prior to improve pricing performance) in a principled way.
• To ensure convergence of individual pricing experiments by accounting for estimation error in the shared prior through adaptive prior widening.
• To demonstrate that the price of an unknown prior becomes negligible in large-N, experiment-rich environments when shared structure is leveraged.

Proposed method

• Proposes a meta dynamic pricing algorithm that runs Thompson sampling experiments sequentially across N related products, each with horizon T.
• Estimates a shared prior distribution over demand parameters across products using online learning techniques.
• Introduces a mechanism to widen the prior distribution based on estimation error, ensuring convergence of individual pricing experiments.
• Balances meta-exploration and meta-exploitation by dynamically adjusting exploration based on uncertainty in the estimated prior.
• Uses a hierarchical Bayesian framework where the shared prior is updated incrementally as new pricing experiment results become available.
• Employs a sublinear meta-regret analysis to show that performance loss due to prior uncertainty diminishes as N increases.

Experimental results

Research questions

• RQ1 How can shared structure across multiple dynamic pricing experiments be leveraged to improve learning efficiency?
• RQ2 What is the trade-off between exploring to learn the shared prior and exploiting it to improve pricing performance?
• RQ3 How can uncertainty in the estimated shared prior be quantified and accounted for to ensure convergence of individual pricing experiments?
• RQ4 What is the impact of prior uncertainty on overall performance in large-scale, experiment-rich environments?
• RQ5 Can a meta-learning approach achieve sublinear meta-regret in the number of products N, even when the prior is initially unknown?

Key findings

• The proposed meta dynamic pricing algorithm achieves sublinear meta-regret in N, indicating that the cost of prior uncertainty diminishes as the number of products increases.
• The algorithm significantly accelerates learning compared to prior-independent methods and naive greedy use of the updated prior on real auto loan data and synthetic data.
• Adaptive widening of the prior based on estimation error ensures convergence of each individual pricing experiment despite uncertainty in the shared prior.
• The price of an unknown prior in Thompson sampling becomes negligible in experiment-rich environments with shared demand structure.
• Numerical experiments confirm that the algorithm outperforms both prior-independent baselines and greedy approaches in terms of learning speed and performance.