[Paper Review] Semiparametric Bayesian causal inference using Gaussian process priors

Kolyan Ray, Aad van der Vaart|arXiv (Cornell University)|Aug 13, 2018

Advanced Causal Inference Techniques4 citations

TL;DR

This paper proposes a semiparametric Bayesian method for causal inference with binary outcomes, using Gaussian process priors to nonparametrically model the propensity score and achieve efficient estimation under smoothness conditions. It introduces a novel propensity score-dependent prior that ensures efficiency under weaker regularity assumptions and demonstrates the theoretical superiority of Dirichlet process or Bayesian bootstrap priors for modeling covariate distributions over Gaussian processes.

ABSTRACT

We develop a semiparametric Bayesian approach for estimating the mean response in a missing data model with binary outcomes and a nonparametrically modelled propensity score. Equivalently we estimate the causal effect of a treatment, correcting nonparametrically for confounding. We show that standard Gaussian process priors satisfy a semiparametric Bernstein-von Mises theorem under smoothness conditions. We further propose a novel propensity score-dependent prior that provides efficient inference under strictly weaker conditions. We also show that it is theoretically preferable to model the covariate distribution with a Dirichlet process or Bayesian bootstrap, rather than modelling the covariate density using a Gaussian process prior.

Motivation & Objective

• To develop a semiparametric Bayesian framework for estimating causal effects in binary outcome models with nonparametrically modeled propensity scores.
• To establish conditions under which standard Gaussian process priors satisfy the semiparametric Bernstein-von Mises theorem.
• To propose a new propensity score-dependent prior that enables efficient inference under strictly weaker smoothness conditions than existing methods.
• To compare the theoretical performance of modeling covariate distributions via Dirichlet process or Bayesian bootstrap versus Gaussian process priors.

Proposed method

• The method employs Gaussian process priors to nonparametrically estimate the propensity score in a missing data model with binary outcomes.
• It establishes asymptotic frequentist validity through a semiparametric Bernstein-von Mises theorem under smoothness conditions on the true propensity score.
• A novel propensity score-dependent prior is constructed to adapt to the underlying smoothness of the propensity score, improving efficiency under weaker regularity assumptions.
• The approach models the covariate distribution using either a Dirichlet process or Bayesian bootstrap, rather than a Gaussian process, to avoid bias in posterior concentration.

Experimental results

Research questions

• RQ1 Under what conditions do standard Gaussian process priors lead to valid semiparametric posterior inference in binary outcome models with missing data?
• RQ2 Can a propensity score-dependent prior be constructed to achieve efficient estimation under weaker smoothness conditions than those required by standard priors?
• RQ3 How does the choice of prior for the covariate distribution affect posterior concentration and estimation efficiency in causal inference?
• RQ4 Is the posterior distribution of the mean response asymptotically normal and efficient when using a Gaussian process prior for the propensity score?

Key findings

• Standard Gaussian process priors satisfy the semiparametric Bernstein-von Mises theorem under smoothness conditions, ensuring asymptotic normality and efficiency of the posterior mean.
• The proposed propensity score-dependent prior achieves efficient inference under strictly weaker smoothness conditions than standard priors, improving robustness.
• Modeling the covariate distribution with a Dirichlet process or Bayesian bootstrap is theoretically preferable to using a Gaussian process prior, as it avoids bias in posterior concentration.
• The theoretical results support the use of nonparametric priors for the propensity score and covariate distribution, with the latter being better suited to non-Gaussian or heavy-tailed covariate structures.