[Paper Review] Variance-based sensitivity analysis for weighting estimators result in more informative bounds

Melody Huang, Samuel D. Pimentel|arXiv (Cornell University)|Aug 2, 2022

Advanced Causal Inference Techniques4 citations

TL;DR

This paper introduces a variance-based sensitivity model for weighting estimators that uses an interpretable, bounded R² parameter to quantify residual imbalance from omitted confounders. By modeling bias through distributional differences in weights rather than worst-case error, it produces tighter, more stable confidence intervals than existing methods, with empirical validation showing improved coverage and narrower bounds in finite samples.

ABSTRACT

Weighting methods are popular tools for estimating causal effects; assessing their robustness under unobserved confounding is important in practice. In the following paper, we introduce a new set of sensitivity models called "variance-based sensitivity models". Variance-based sensitivity models characterize the bias from omitting a confounder by bounding the distributional differences that arise in the weights from omitting a confounder, with several notable innovations over existing approaches. First, the variance-based sensitivity models can be parameterized with respect to a simple $R^2$ parameter that is both standardized and bounded. We introduce a formal benchmarking procedure that allows researchers to use observed covariates to reason about plausible parameter values in an interpretable and transparent way. Second, we show that researchers can estimate valid confidence intervals under a set of variance-based sensitivity models, and provide extensions for researchers to incorporate their substantive knowledge about the confounder to help tighten the intervals. Last, we highlight the connection between our proposed approach and existing sensitivity analyses, and demonstrate both, empirically and theoretically, that variance-based sensitivity models can provide improvements on both the stability and tightness of the estimated confidence intervals over existing methods. We illustrate our proposed approach on a study examining blood mercury levels using the National Health and Nutrition Examination Survey (NHANES).

Motivation & Objective

• To address the challenge of assessing robustness to unobserved confounding in weighting estimators used in observational causal inference.
• To develop a sensitivity model that is both interpretable and bounded, enabling researchers to reason about plausible values for unobserved confounding.
• To improve the stability and informativeness of confidence intervals in sensitivity analysis by moving beyond worst-case error bounds.
• To provide a benchmarking procedure using observed covariates to inform plausible values for the sensitivity parameter (R²).
• To enable tighter bounds by incorporating substantive knowledge about the relationship between the confounder and outcome.

Proposed method

• Proposes a variance-based sensitivity model that constrains distributional differences in estimated vs. true weights due to omitted confounders.
• Parameterizes the model using an R² measure representing the proportion of residual imbalance from an omitted confounder, bounded between 0 and 1.
• Develops a formal benchmarking procedure to calibrate the R² parameter using observed covariates, enhancing interpretability.
• Derives a closed-form solution for the optimal bias bound under the sensitivity model, enabling valid asymptotic confidence intervals.
• Extends the model by incorporating constraints on the correlation between the omitted confounder and the outcome, further tightening bounds.
• Frames the sensitivity analysis as a bias maximization problem under a weighted average error constraint, contrasting with worst-case error approaches.

Experimental results

Research questions

• RQ1 How can sensitivity analysis for weighting estimators be made more interpretable and less conservative than existing worst-case error-based models?
• RQ2 Can a bounded, standardized sensitivity parameter like R² improve the interpretability and calibration of sensitivity analysis in causal inference?
• RQ3 To what extent can confidence intervals under the variance-based sensitivity model maintain nominal coverage in finite samples compared to marginal sensitivity models?
• RQ4 How does incorporating the relationship between the confounder and outcome affect the tightness of sensitivity bounds?
• RQ5 Can the variance-based model produce more informative bounds than existing approaches while preserving statistical validity?

Key findings

• The variance-based sensitivity model produces significantly narrower confidence intervals than the marginal sensitivity model, particularly in finite samples where the latter suffers from severe undercoverage.
• The model’s use of R² as a sensitivity parameter ensures interpretability and boundedness, enabling researchers to benchmark plausible values using observed covariates.
• Empirical results on NHANES data show that even confounders with high imbalance (e.g., R² = 0.12 for age) can induce low bias if they are uncorrelated with the outcome.
• By incorporating outcome-confounder correlation constraints, the model produces substantially tighter bounds, demonstrating the importance of considering both imbalance and outcome relevance.
• The closed-form solution for the optimal bias bound enables efficient and valid inference, supporting asymptotically valid confidence intervals under the sensitivity model.
• The variance-based approach outperforms worst-case error models in stability and informativeness by focusing on distributional differences rather than extreme outliers.