[Paper Review] VCBART: Bayesian trees for varying coefficients

Sameer K. Deshpande, Ray Bai+3 more|arXiv (Cornell University)|Mar 13, 2020

Health disparities and outcomes5 citations

TL;DR

VCBART introduces a Bayesian additive regression trees framework for varying coefficient models, enabling flexible, nonparametric estimation of covariate effects that vary with multivariate effect modifiers. It outperforms existing methods in estimation accuracy, uncertainty quantification, and prediction without restrictive functional form assumptions or intensive tuning.

ABSTRACT

The linear varying coefficient models posits a linear relationship between an outcome and covariates in which the covariate effects are modeled as functions of additional effect modifiers. Despite a long history of study and use in statistics and econometrics, state-of-the-art varying coefficient modeling methods cannot accommodate multivariate effect modifiers without imposing restrictive functional form assumptions or involving computationally intensive hyperparameter tuning. In response, we introduce VCBART, which flexibly estimates the covariate effect in a varying coefficient model using Bayesian Additive Regression Trees. With simple default settings, VCBART outperforms existing varying coefficient methods in terms of covariate effect estimation, uncertainty quantification, and outcome prediction. We illustrate the utility of VCBART with two case studies: one examining how the association between later-life cognition and measures of socioeconomic position vary with respect to age and socio-demographics and another estimating how temporal trends in urban crime vary at the neighborhood level. An R package implementing VCBART is available at https://github.com/skdeshpande91/VCBART

Motivation & Objective

• Address the limitations of existing varying coefficient models that require restrictive functional form assumptions or intensive hyperparameter tuning when handling multivariate effect modifiers.
• Develop a method that flexibly estimates covariate effects as nonparametric functions of multiple effect modifiers without imposing rigid parametric structures.
• Enable accurate uncertainty quantification and predictive performance in high-dimensional or complex data settings common in social science and public health research.
• Provide a scalable, default-friendly Bayesian approach that maintains interpretability while achieving performance comparable to black-box models.

Proposed method

• Model the varying coefficient functions βj(Z) using Bayesian Additive Regression Trees (BART), where each βj(Z) is represented as an ensemble of binary regression trees.
• Use a hierarchical prior structure on tree parameters to allow for flexible, nonparametric function learning while maintaining shrinkage and regularization.
• Leverage MCMC sampling to jointly estimate the coefficient functions βj(Z) and the outcome regression, enabling posterior inference and uncertainty quantification.
• Apply a minimax optimal posterior concentration rate theorem (Theorem 1) to establish theoretical consistency of the method up to a logarithmic factor.
• Implement the method via a Gibbs sampler that iteratively updates tree structures and node parameters, with default priors minimizing user tuning.
• Extend the framework to sparse varying coefficient models by replacing normal priors on tree jumps with spike-and-slab priors to induce sparsity in non-zero coefficient functions.

Experimental results

Research questions

• RQ1 How well can VCBART estimate varying coefficient functions when the true functional forms are unknown and potentially complex?
• RQ2 Can VCBART provide well-calibrated uncertainty intervals for coefficient function evaluations and outcome predictions without problem-specific tuning?
• RQ3 How does VCBART perform in comparison to existing state-of-the-art varying coefficient methods in terms of estimation accuracy and predictive performance?
• RQ4 To what extent can VCBART detect non-linear, non-monotonic trends in crime over time across different neighborhoods in Philadelphia?
• RQ5 How do associations between socioeconomic position and later-life cognition vary across age, race, and gender, and can VCBART detect such heterogeneity without pre-specifying interaction terms?

Key findings

• VCBART achieved superior covariate effect function recovery and predictive performance compared to existing state-of-the-art varying coefficient methods on simulated data, without requiring intensive tuning.
• In the Health and Retirement Study (HRS) application, VCBART found no significant association between childhood SEP and later-life cognition after adjusting for early and late adulthood SEP, and no strong time-varying effects by age, race, or gender.
• In the Philadelphia crime analysis, VCBART identified that in 88% of census tracts (338 out of 384), the linear trend coefficient β1 had a 95% credible interval excluding zero, indicating a significant linear trend in crime over time.
• For 32% of tracts (125 out of 384), both the linear and quadratic coefficients (β1 and β2) had credible intervals excluding zero, suggesting a non-linear trend in crime in over a third of neighborhoods.
• The model revealed that crime in Philadelphia declined citywide in the early 2010s before rising again toward the end of the decade, with VCBART capturing this non-monotonic trend through tract-specific polynomial fits.
• Theoretical analysis shows that the VCBART posterior concentrates at the minimax optimal rate for function estimation, up to a logarithmic factor, establishing strong theoretical grounding for the method.