[Paper Review] An Interval Estimation Approach to Sample Selection Bias

Matthew Tudball, Qingyuan Zhao+3 more|arXiv (Cornell University)|Jun 24, 2019

Advanced Causal Inference Techniques2 references4 citations

TL;DR

This paper proposes a computationally efficient interval estimator for population parameters under sample selection bias, using stochastic programming to derive valid confidence intervals with minimal assumptions about the selection mechanism. It improves interval precision by incorporating common auxiliary data like response rates and covariate means, demonstrating robust performance in simulations and a real-world study on education’s impact on income in the UK Biobank.

ABSTRACT

A widespread and largely unaddressed challenge in statistics is that non-random participation in study samples can bias the estimation of parameters of interest. To address this problem, we propose a computationally efficient interval estimator for a class of population parameters encompassing population means, ordinary least squares and instrumental variables estimands which makes minimal assumptions about the selection mechanism. Using results from stochastic programming, we derive valid confidence intervals and hypothesis tests based on this estimator. In addition, we demonstrate how to tighten the intervals by incorporating additional constraints based on population-level information commonly available to researchers, such as survey response rates and covariate means. We conduct a comprehensive simulation study to evaluate the finite sample performance of our estimator and conclude with a real data study on the causal effect of education on income in the highly-selected UK Biobank cohort. We are able to demonstrate that our method can produce informative bounds under relatively few population-level auxiliary constraints.

Motivation & Objective

• Address the widespread problem of sample selection bias in statistical estimation, where non-random participation distorts parameter estimates.
• Develop a computationally efficient interval estimator applicable to a broad class of parameters, including population means, OLS, and IV estimands.
• Derive valid confidence intervals and hypothesis tests using stochastic programming without strong assumptions about the selection mechanism.
• Improve interval precision by integrating common population-level auxiliary information such as survey response rates and covariate means.
• Demonstrate the method’s finite-sample performance and practical utility through simulations and a real-world causal inference study on education and income.

Proposed method

• Formulate the interval estimation problem using stochastic programming to model uncertainty in the selection mechanism.
• Construct a confidence interval for the parameter of interest by solving a convex optimization problem that bounds the bias due to non-random sampling.
• Use minimal structural assumptions about the selection process, relying only on the existence of a selection indicator and observed outcomes.
• Incorporate auxiliary constraints—such as known population-level response rates and covariate means—into the optimization framework to tighten interval bounds.
• Derive valid statistical inference (confidence intervals and hypothesis tests) based on the resulting interval estimator.
• Apply the method to both simulated data and real-world data from the UK Biobank to evaluate performance under realistic conditions.

Experimental results

Research questions

• RQ1 Can a computationally efficient interval estimator be developed for population parameters under sample selection bias with minimal assumptions about the selection mechanism?
• RQ2 How do auxiliary constraints—such as known survey response rates and covariate means—affect the precision of the resulting confidence intervals?
• RQ3 What is the finite-sample performance of the proposed interval estimator in realistic simulation settings?
• RQ4 Can the method produce informative bounds in highly selected real-world cohorts, such as the UK Biobank, for causal inference on education and income?

Key findings

• The proposed interval estimator produces valid confidence intervals under minimal assumptions about the selection mechanism, ensuring statistical validity.
• Incorporating auxiliary constraints such as response rates and covariate means significantly tightens the interval bounds, improving precision with minimal data requirements.
• The method demonstrates strong finite-sample performance in simulations, maintaining coverage rates close to nominal levels across diverse selection mechanisms.
• In the UK Biobank application, the method produced informative bounds on the causal effect of education on income despite the cohort’s high selection bias.
• The approach remains computationally efficient and scalable, making it practical for real-world use in observational studies with selection bias.
• The method enables robust causal inference even when unmeasured confounding or selection is suspected, by providing bounded estimates with valid statistical inference.