[Paper Review] Balancing Covariates in Randomized Experiments with the Gram-Schmidt Walk Design

Venkat, Prayaag|arXiv (Cornell University)|Nov 8, 2019

Advanced Causal Inference Techniques42 references13 citations

TL;DR

This paper introduces the Gram-Schmidt Walk (GSW) design, a novel experimental design that balances covariates and controls robustness via a tunable parameter. It achieves near-perfect covariate balance asymptotically while maintaining finite-sample robustness, enabling asymptotically valid inference with conservative confidence intervals.

ABSTRACT

In this paper, we initiate the study of the algorithmic problem of certifying lower bounds on the discrepancy of random matrices: given an input matrix A ∈ ℝ^{m × n}, output a value that is a lower bound on disc(A) = min_{x ∈ {± 1}ⁿ} ‖Ax‖_∞ for every A, but is close to the typical value of disc(A) with high probability over the choice of a random A. This problem is important because of its connections to conjecturally-hard average-case problems such as negatively-spiked PCA [Afonso S. Bandeira et al., 2020], the number-balancing problem [Gamarnik and Kızıldağ, 2021] and refuting random constraint satisfaction problems [Prasad Raghavendra et al., 2017]. We give the first polynomial-time algorithms with non-trivial guarantees for two main settings. First, when the entries of A are i.i.d. standard Gaussians, it is known that disc(A) = Θ (√n2^{-n/m}) with high probability [Karthekeyan Chandrasekaran and Santosh S. Vempala, 2014; Aubin et al., 2019; Paxton Turner et al., 2020] and that super-constant levels of the Sum-of-Squares SDP hierarchy fail to certify anything better than disc(A) ≥ 0 when m < n - o(n) [Mrinalkanti Ghosh et al., 2020]. In contrast, our algorithm certifies that disc(A) ≥ exp(-O(n²/m)) with high probability. As an application, this formally refutes a conjecture of Bandeira, Kunisky, and Wein [Afonso S. Bandeira et al., 2020] on the computational hardness of the detection problem in the negatively-spiked Wishart model. Second, we consider the integer partitioning problem: given n uniformly random b-bit integers a₁, …, a_n, certify the non-existence of a perfect partition, i.e. certify that disc(A) ≥ 1 for A = (a₁, …, a_n). Under the scaling b = α n, it is known that the probability of the existence of a perfect partition undergoes a phase transition from 1 to 0 at α = 1 [Christian Borgs et al., 2001]; our algorithm certifies the non-existence of perfect partitions for some α = O(n). We also give efficient non-deterministic algorithms with significantly improved guarantees, raising the possibility that the landscape of these certification problems closely resembles that of e.g. the problem of refuting random 3SAT formulas in the unsatisfiable regime. Our algorithms involve a reduction to the Shortest Vector Problem and employ the Lenstra-Lenstra-Lovász algorithm.

Motivation & Objective

• Address the fundamental trade-off between covariate balance and robustness in experimental design.
• Formalize robustness using worst-case mean squared error and balance using linear functions of covariates.
• Develop a design that allows experimenters to tune the balance-robustness trade-off via a single parameter.
• Ensure asymptotic normality and provide a conservative variance estimator for valid inference.
• Demonstrate that the design escapes the balance-robustness trade-off in large samples, achieving perfect balance with diminishing robustness loss.

Proposed method

• Formulate the design as a distributional discrepancy problem, translating experimental design to algorithmic discrepancy.
• Adapt the Gram–Schmidt Walk algorithm (Bansal et al., 2019) to generate a randomized treatment assignment distribution.
• Use a robustness parameter ϕ to bound the worst-case mean squared error of the Horvitz–Thompson estimator.
• Ensure all linear functions of covariates are simultaneously balanced under the design, with bounds derived from ridge regression loss.
• Derive finite-sample bounds on mean squared error and sampling distribution tails using discrepancy theory.
• Provide a conservative, consistent variance estimator for asymptotically valid confidence intervals.

Experimental results

Research questions

• RQ1 How can the balance-robustness trade-off in experimental design be formally quantified and navigated?
• RQ2 Can a single design achieve both high covariate balance and sufficient robustness across diverse covariate configurations?
• RQ3 What is the asymptotic behavior of the Horvitz–Thompson estimator under the GSW design?
• RQ4 Does the GSW design enable asymptotically valid inference with conservative confidence intervals?
• RQ5 How does the GSW design compare to rerandomization and matched-pair designs in terms of balance, robustness, and mean squared error?

Key findings

• The Gram–Schmidt Walk design bounds the mean squared error of the Horvitz–Thompson estimator by the loss of an implicit ridge regression of potential outcomes on covariates.
• In large samples, the GSW design asymptotically achieves perfect balance for all linear functions of covariates, effectively escaping the balance-robustness trade-off.
• For n = 2960, the GSW design with ϕ = 0.01 achieved a balance operator norm approximately 57 times smaller than rerandomization and 41 times smaller than the matched pair design.
• Under the GSW design with ϕ = 0.01, mean squared error was more than 31 times smaller than under the matched pair design for outcome D.
• Main confidence intervals under GSW with ϕ = 0.01 had widths approximately 10–14% narrower than Bernoulli and complete randomization, with coverage exceeding nominal levels for n ≥ 296.
• The conservative variance estimator ensures asymptotically valid confidence intervals, with alternative intervals based on finite-sample bounds showing higher coverage but increased width.