[Paper Review] Clip-OGD: An Experimental Design for Adaptive Neyman Allocation in Sequential Experiments

Jessica Dai, Paula Gradu+1 more|arXiv (Cornell University)|May 26, 2023

Optimal Experimental Design Methods4 citations

TL;DR

This paper proposes Clip-OGD, an adaptive experimental design that achieves near-optimal precision in sequential causal inference by minimizing Neyman regret—proving it attains $˜{\mathcal{O}}(\sqrt{T})$ expected regret. The method leverages online convex optimization to dynamically allocate treatments based on observed outcomes, asymptotically recovering the optimal Neyman variance and enabling valid confidence intervals via a conservative variance estimator.

ABSTRACT

From clinical development of cancer therapies to investigations into partisan bias, adaptive sequential designs have become increasingly popular method for causal inference, as they offer the possibility of improved precision over their non-adaptive counterparts. However, even in simple settings (e.g. two treatments) the extent to which adaptive designs can improve precision is not sufficiently well understood. In this work, we study the problem of Adaptive Neyman Allocation in a design-based potential outcomes framework, where the experimenter seeks to construct an adaptive design which is nearly as efficient as the optimal (but infeasible) non-adaptive Neyman design, which has access to all potential outcomes. Motivated by connections to online optimization, we propose Neyman Ratio and Neyman Regret as two (equivalent) performance measures of adaptive designs for this problem. We present Clip-OGD, an adaptive design which achieves $\widetilde{O}(\sqrt{T})$ expected Neyman regret and thereby recovers the optimal Neyman variance in large samples. Finally, we construct a conservative variance estimator which facilitates the development of asymptotically valid confidence intervals. To complement our theoretical results, we conduct simulations using data from a microeconomic experiment.

Motivation & Objective

• To address the challenge of achieving optimal precision in sequential experiments when the true potential outcomes are unknown.
• To formalize performance guarantees for adaptive designs through Neyman Ratio and Neyman Regret as key metrics.
• To develop an adaptive design that asymptotically approaches the variance of the infeasible optimal non-adaptive Neyman design.
• To construct a conservative variance estimator enabling asymptotically valid confidence intervals under the adaptive design.
• To reconcile statistical efficiency with ethical trade-offs in treatment allocation, particularly between minimizing harm and maximizing estimation precision.

Proposed method

• Introduce Neyman Regret as a performance measure: the difference between the cumulative loss of the adaptive design and the optimal non-adaptive Neyman design.
• Propose Clip-OGD, a variant of online stochastic projected gradient descent, to minimize Neyman Regret in sequential treatment allocation.
• Use a clipping mechanism in the gradient update to ensure boundedness and stability in the adaptive probability updates.
• Define the loss function at each time step as $ f_t(p) = \frac{y_t(1)^2}{p} + \frac{y_t(0)^2}{1-p} $, which corresponds to the inverse variance of the difference-in-means estimator.
• Construct a conservative variance estimator that accounts for estimation error in the adaptive design, ensuring asymptotic coverage of confidence intervals.
• Frame the adaptive allocation problem as online convex optimization, leveraging theoretical guarantees from that domain to derive regret bounds.

(a) Original dataset

Experimental results

Research questions

• RQ1 Can an adaptive sequential design achieve variance close to the optimal non-adaptive Neyman design, despite not observing potential outcomes in advance?
• RQ2 What performance metric best captures the efficiency of adaptive designs in sequential causal inference?
• RQ3 Can online optimization techniques be adapted to yield sublinear regret in the context of adaptive treatment allocation?
• RQ4 How can valid confidence intervals be constructed under adaptive designs that do not assume fixed treatment probabilities?
• RQ5 What are the ethical trade-offs between minimizing cumulative regret (patient welfare) and minimizing estimation variance (scientific precision)?

Key findings

• Clip-OGD achieves $\widetilde{\mathcal{O}}(\sqrt{T})$ expected Neyman regret, ensuring the estimator's variance converges to the optimal Neyman variance in large samples.
• The Neyman Regret and Neyman Ratio are equivalent performance measures that directly control the convergence of the estimator's variance to the optimal level.
• A conservative variance estimator is constructed that guarantees asymptotic coverage of confidence intervals under the adaptive design.
• Theoretical analysis shows that any design with suboptimal allocation will incur at least $\Omega(T^{2-q})$ regret under certain moment conditions, highlighting the optimality of Clip-OGD's rate.
• Simulations using microeconomic data validate the theoretical results, showing that Clip-OGD achieves variance close to the optimal Neyman design.
• The ethical analysis reveals a fundamental trade-off: designs minimizing cumulative regret sacrifice estimation precision, and vice versa, underscoring the need for context-specific design choices.

(b) First 100 units have flipped potential outcomes