[Paper Review] Estimation of Causal Effects of Multiple Treatments in Observational Studies with a Binary Outcome

Liangyuan Hu, Chenyang Gu+3 more|arXiv (Cornell University)|Jan 17, 2020

Advanced Causal Inference Techniques52 references4 citations

TL;DR

This paper proposes Bayesian Additive Regression Trees (BART) for estimating causal effects of multiple treatments with binary outcomes, demonstrating superior performance over IPTW, TMLE, and regression adjustment in bias reduction, root mean squared error, and confidence interval coverage. BART's novel common support rule retains more units and reduces bias compared to generalized propensity score methods, especially under low covariate overlap.

ABSTRACT

There is a dearth of robust methods to estimate the causal effects of multiple treatments when the outcome is binary. This paper uses two unique sets of simulations to propose and evaluate the use of Bayesian Additive Regression Trees (BART) in such settings. First, we compare BART to several approaches that have been proposed for continuous outcomes, including inverse probability of treatment weighting (IPTW), targeted maximum likelihood estimator (TMLE), vector matching and regression adjustment. Results suggest that under conditions of non-linearity and non-additivity of both the treatment assignment and outcome generating mechanisms, BART, TMLE and IPTW using generalized boosted models (GBM) provide better bias reduction and smaller root mean squared error. BART and TMLE provide more consistent 95 per cent CI coverage and better large-sample convergence property. Second, we supply BART with a strategy to identify a common support region for retaining inferential units and for avoiding extrapolating over areas of the covariate space where common support does not exist. BART retains more inferential units than the generalized propensity score based strategy, and shows lower bias, compared to TMLE or GBM, in a variety of scenarios differing by the degree of covariate overlap. A case study examining the effects of three surgical approaches for non-small cell lung cancer demonstrates the methods.

Motivation & Objective

• To address the lack of robust methods for estimating causal effects of multiple treatments when the outcome is binary.
• To evaluate BART's performance in settings with non-linear and non-additive treatment and outcome mechanisms.
• To develop and validate a common support strategy for BART that improves inference by avoiding extrapolation in low-overlap covariate regions.
• To compare BART with established methods—such as IPTW, TMLE, vector matching, and regression adjustment—under varying degrees of covariate overlap.
• To demonstrate the method’s utility through a real-world case study on surgical approaches for non-small cell lung cancer using SEER-Medicare data.

Proposed method

• BART is extended to the multiple treatment and binary outcome setting, modeling potential outcomes using Bayesian nonparametric regression trees.
• A novel common support rule is proposed, based on posterior predictive distributions, to identify and retain units where counterfactual predictions are reliable.
• The method uses posterior predictive distributions to assess whether observed covariate values are within the support of the counterfactual treatment group, discarding units only when uncertainty is high.
• BART is implemented with 5-fold cross-validation to select the optimal prior hyperparameter $k$, with $k=2$ found to minimize misclassification error.
• The approach is compared to IPTW with generalized boosted models (GBM), TMLE, vector matching, and regression adjustment using simulated data under varying degrees of covariate overlap.
• A discarding rule based on 1 standard deviation of posterior predictive uncertainty is adapted from single-treatment settings to the multiple-treatment context.

Experimental results

Research questions

• RQ1 How does BART compare to IPTW, TMLE, and regression adjustment in estimating causal effects of multiple treatments with binary outcomes under non-linear and non-additive mechanisms?
• RQ2 Can a common support rule based on posterior predictive uncertainty improve inference quality and unit retention in BART for multiple treatments?
• RQ3 How does BART perform relative to alternative methods when covariate overlap between treatment groups is low or moderate?
• RQ4 What is the impact of method choice—especially pairwise vs. multi-treatment approaches—on treatment effect estimates in observational studies with unequal treatment group sizes?
• RQ5 How does BART’s computational efficiency and interval estimation performance compare to GBM and matching-based methods in binary outcome settings?

Key findings

• BART, TMLE, and IPTW with GBM showed the lowest bias and root mean squared error under non-linear and non-additive treatment and outcome mechanisms.
• BART and TMLE demonstrated more consistent 95% confidence interval coverage and better large-sample convergence properties than other methods.
• The proposed BART common support rule retained more inferential units than generalized propensity score-based strategies, especially in low-overlap scenarios.
• BART showed lower bias than TMLE and GBM across all scenarios, with the greatest improvement observed under low covariate overlap.
• BART was computationally efficient, completing analyses on a dataset of 11,600 units in under 150 seconds, compared to ~10 minutes for IPTW-GBM.
• In the NSCLC case study, BART suggested robotic-assisted surgery may be associated with lower prolonged LOS and ICU stay compared to open thoracotomy or VATS, depending on the method used.