[Paper Review] Contextualizing E-values for Interpretable Sensitivity to Unmeasured Confounding Analyses

Lucy D’Agostino McGowan, Robert A. Greevy|arXiv (Cornell University)|Nov 13, 2020

Statistical Methods and Inference26 references4 citations

TL;DR

This paper introduces the Observed Covariate E-value and an observed bias plot to contextualize VanderWeele and Ding’s E-value within the actual impact of observed covariates in sensitivity analyses for unmeasured confounding. By translating hypothetical unmeasured confounder effects into the scale of observed covariate effects, the method enables researchers to assess whether unmeasured confounding of similar magnitude to observed confounders could plausibly alter study conclusions, thereby improving interpretability and clinical relevance of sensitivity analyses.

ABSTRACT

The strength of evidence provided by epidemiological and observational studies is inherently limited by the potential for unmeasured confounding. Researchers should present a quantified sensitivity to unmeasured confounding analysis that is contextualized by the study's observed covariates. VanderWeele and Ding's E-value provides an easily calculated metric for the magnitude of the hypothetical unmeasured confounding required to render the study's result inconclusive. We propose the Observed Covariate E-value to contextualize the sensitivity analysis' hypothetical E-value within the actual impact of observed covariates, individually or within groups. We introduce a sensitivity analysis figure that presents the Observed Covariate E-values, on the E-value scale, next to their corresponding observed bias effects, on the original scale of the study results. This observed bias plot allows easy comparison of the hypothetical E-values, Observed Covariate E-values, and observed bias effects. We illustrate the methods with a specific example and provide a supplemental appendix with modifiable code that teaches how to implement the method and create a publication quality figure.

Motivation & Objective

• To address the lack of contextual interpretation in E-values for sensitivity to unmeasured confounding.
• To improve the interpretability of E-values by grounding them in the observed effects of measured covariates.
• To provide a practical, visual method for researchers to evaluate the plausibility of unmeasured confounding relative to observed confounding.
• To promote widespread adoption of quantified sensitivity analyses in observational research by simplifying and contextualizing E-values.
• To demonstrate that covariates with high imbalance may not have the largest impact on effect estimates, challenging assumptions about confounder importance.

Proposed method

• Proposes the Observed Covariate E-value as a transformation of the observed bias effect of each covariate onto the E-value scale.
• Uses regression models to estimate the bias introduced by omitting individual or grouped covariates from the analysis.
• Constructs an observed bias plot that displays Observed Covariate E-values alongside their corresponding observed bias effects on the original outcome scale.
• Applies the E-value framework to quantify the strength of hypothetical unmeasured confounding needed to tip the result to non-significance.
• Integrates the method into a full sensitivity analysis workflow, including model comparison and visualization.
• Provides R code via the tipr package to implement the method and generate publication-quality figures.

Experimental results

Research questions

• RQ1 Can the E-value be meaningfully contextualized by the observed impact of measured covariates to improve interpretability?
• RQ2 What is the magnitude of confounding from observed covariates, and how does it compare to the threshold of unmeasured confounding needed to change study conclusions?
• RQ3 Do covariates with high imbalance in exposure groups necessarily have the largest influence on effect estimates?
• RQ4 How can researchers assess the plausibility of unmeasured confounding by comparing it to the effects of observed confounders?
• RQ5 Can a visual plot combining E-values and observed bias effects enhance understanding of sensitivity analysis results?

Key findings

• The Observed Covariate E-value provides a direct comparison between the strength of hypothetical unmeasured confounding and the actual confounding from observed covariates.
• In the illustrative example, dropping all 12 covariates (including APACHE score, physiological measurements, and survival probability) had a much larger impact on the hazard ratio than dropping subsets, indicating synergistic confounding effects.
• Covariates with high imbalance, such as APACHE score and PaO2/FIO2 ratio, did not always have the largest observed bias effects, challenging assumptions about confounder priority.
• The observed bias plot effectively visualizes the relative impact of observed covariates and hypothetical unmeasured confounders on the same scale, improving interpretability.
• The method reveals that removing a combination of covariates—even if individually weak—can produce substantial bias, highlighting the importance of group-level confounding.
• The approach enables researchers to judge whether unmeasured confounders of similar magnitude to observed ones would be sufficient to tip the result, thus assessing plausibility of residual confounding.