[Paper Review] Experimenting under Stochastic Congestion

Shuangning Li, Ramesh Johari+2 more|arXiv (Cornell University)|Feb 22, 2023

Advanced Queuing Theory Analysis7 citations

TL;DR

The paper analyzes randomized experiments in service systems with dynamic congestion and introduces queueing-model–based estimators and a local perturbation design that improve efficiency over standard switchbacks.

ABSTRACT

We study randomized experiments in a service system when stochastic congestion can arise from temporarily limited supply or excess demand. Such congestion gives rise to cross-unit interference between the waiting customers, and analytic strategies that do not account for this interference may be biased. In current practice, one of the most widely used ways to address stochastic congestion is to use switchback experiments that alternatively turn a target intervention on and off for the whole system. We find, however, that under a queueing model for stochastic congestion, the standard way of analyzing switchbacks is inefficient, and that estimators that leverage the queueing model can be materially more accurate. Additionally, we show how the queueing model enables estimation of total policy gradients from unit-level randomized experiments, thus giving practitioners an alternative experimental approach they can use without needing to pre-commit to a fixed switchback length before data collection.

Motivation & Objective

• Motivate the study of randomized experiments in service systems with stochastic congestion and cross-unit interference.
• Model congestion via a single-server queue to capture dynamic interference effects.
• Develop estimators for the policy gradient V'(p) that exploit queueing structure.
• Compare estimators in terms of asymptotic variance and identify conditions under which gains occur.
• Propose a new class of local perturbation experimental designs that align with the model.

Proposed method

• Model congestion with a continuous-time Markovian queueing system with price-dependent arrivals.
• Derive multiple representations of the policy gradient V'(p) to separate direct and indirect effects.
• Develop three plug-in estimators corresponding to the representations: model-free, idle-time-based, and re-weighted direct effect (WDE).
• Analyze two switchback variants: interval switchback and regenerative switchback.
• Provide asymptotic normality results for the estimators under stationarity and shrinking price perturbations.
• Establish variance relationships showing WDE variance is half of idle-time variance and no larger than model-free variance under general conditions.

Experimental results

Research questions

• RQ1 How does stochastic congestion affect the estimation of the impact of price on long-run admission rate in a queueing system?
• RQ2 Can queueing-structure be exploited to design more efficient estimators for V'(p) than standard switchback analysis?
• RQ3 How do different switchback designs (interval vs regenerative) compare in terms of estimator variance and robustness?
• RQ4 Under what conditions does a new re-weighted direct effect estimator outperform model-free and idle-time estimators?
• RQ5 What is the impact of customer preferences and queueing dynamics on estimator variance and accuracy?

Key findings

• Three representations of V'(p) enable alternative estimators that exploit queueing dynamics.
• The re-weighted direct effect (WDE) estimator achieves lower asymptotic variance than idle-time and model-free estimators, with exact variance reductions quantified.
• For M/M/1 queues, the WDE variance matches the model-free variance, and the idle-time variance equals twice the WDE variance.
• Interval and regenerative switchbacks can be used with these estimators, and asymptotic normality holds under the proposed assumptions.
• The paper provides explicit variance formulas and plug-in estimators for confidence intervals based on steady-state probabilities.
• The analysis reveals how customer joining behavior and queue-length distribution influence estimator accuracy and when variance reductions are most pronounced.