[Paper Review] Synthetically Controlled Bandits

Vivek F. Farias, Ciamac C. Moallemi+2 more|arXiv (Cornell University)|Feb 14, 2022

Advanced Bandit Algorithms Research4 citations

TL;DR

This paper introduces Synthetically Controlled Thompson Sampling (SCTS), a dynamic experiment design for coarse experimental units subject to interference, where treatment effects are estimated via synthetic controls. SCTS achieves near-optimal regret scaling as $ r\sqrt{T} $ while maintaining valid inference for positive treatment effects, minimizing costly experimentation without sacrificing inferential power when outcomes are beneficial.

ABSTRACT

This paper presents a new dynamic approach to experiment design in settings where, due to interference or other concerns, experimental units are coarse. `Region-split' experiments on online platforms are one example of such a setting. The cost, or regret, of experimentation is a natural concern here. Our new design, dubbed Synthetically Controlled Thompson Sampling (SCTS), minimizes the regret associated with experimentation at no practically meaningful loss to inferential ability. We provide theoretical guarantees characterizing the near-optimal regret of our approach, and the error rates achieved by the corresponding treatment effect estimator. Experiments on synthetic and real world data highlight the merits of our approach relative to both fixed and `switchback' designs common to such experimental settings.

Motivation & Objective

• To address the high cost of experimentation in settings with interference, where coarse units like regions or cities are used due to user interaction effects.
• To minimize regret in dynamic experimentation while preserving the ability to estimate treatment effects accurately when they are positive.
• To develop a bandit algorithm that leverages synthetic controls despite unobserved, noisy, and rotationally ambiguous context vectors.
• To ensure valid statistical inference—particularly confidence intervals and hypothesis tests—under low signal-to-noise ratios and non-stationary data.
• To enable practical deployment in real-world platforms where sub-optimal treatments must be avoided due to high per-epoch cost.

Proposed method

• SCTS employs a Thompson sampling framework with carefully designed exploration noise to balance exploration and exploitation in the absence of direct context observation.
• Context vectors (latent common factors) are recovered post-hoc using principal components analysis (PCA) on historical outcome data.
• The method accounts for noise and rotational ambiguity in context recovery, ensuring robustness to estimation errors.
• A re-randomization test is used for inference, constructing confidence intervals that maintain near-nominal coverage even at low signal-to-noise ratios.
• The algorithm dynamically selects treatments based on synthetic control predictions, minimizing regret while maintaining inferential validity.
• The regret bound is theoretically shown to scale as $ r\sqrt{T} $, where $ r $ is the dimension of the latent context and $ T $ is the horizon.

Experimental results

Research questions

• RQ1 Can a dynamic bandit algorithm achieve near-optimal regret in settings with coarse experimental units and interference, while preserving valid inference for treatment effects?
• RQ2 How can synthetic control methods be integrated into a contextual bandit framework when the true context is unobserved and only noisy, linearly transformed observations are available?
• RQ3 What is the impact of context recovery error—due to noise and rotational ambiguity—on regret and inference in such a setting?
• RQ4 Can valid statistical inference be maintained under low signal-to-noise ratios, particularly when treatment effects are small or dominated by noise?
• RQ5 How does the performance of SCTS compare to fixed and switchback designs in terms of regret and estimation accuracy?

Key findings

• SCTS achieves a regret bound of $ O(r\sqrt{T}) $, which is near-optimal for the underlying contextual bandit problem with latent, unobserved contexts.
• The re-randomization test maintains nearly ideal coverage (e.g., 0.89 at SNR = 0.1) and high power (0.98 at SNR = 1), even at low signal-to-noise ratios.
• At an SNR of 0.2, the test maintains 81% power, demonstrating strong performance under noisy conditions.
• The method enables valid inference for treatment effects only when the effect is positive, which aligns with practical needs for cost-benefit analysis.
• The approach significantly reduces regret compared to fixed and switchback designs, especially in non-stationary or interference-prone environments.
• Confidence intervals constructed via re-randomization maintain conservative coverage across a broad range of SNRs, from 0.01 to 1.