[Paper Review] Estimating Average Treatment Effects with Support Vector Machines

Alexander Tarr, Kosuke Imai|arXiv (Cornell University)|Feb 23, 2021

Advanced Causal Inference Techniques43 references4 citations

TL;DR

This paper proposes using support vector machines (SVM) as a novel method for estimating average treatment effects by leveraging SVM's dual coefficients as kernel-based balancing weights that minimize maximum mean discrepancy (MMD) while maximizing effective sample size. The approach provides a continuous relaxation of cardinality matching and enables efficient exploration of the balance-sample size frontier via SVM path algorithms, yielding stable, bias-reduced causal estimates competitive with state-of-the-art methods.

ABSTRACT

Support vector machine (SVM) is one of the most popular classification algorithms in the machine learning literature. We demonstrate that SVM can be used to balance covariates and estimate average causal effects under the unconfoundedness assumption. Specifically, we adapt the SVM classifier as a kernel-based weighting procedure that minimizes the maximum mean discrepancy between the treatment and control groups while simultaneously maximizing effective sample size. We also show that SVM is a continuous relaxation of the quadratic integer program for computing the largest balanced subset, establishing its direct relation to the cardinality matching method. Another important feature of SVM is that the regularization parameter controls the trade-off between covariate balance and effective sample size. As a result, the existing SVM path algorithm can be used to compute the balance-sample size frontier. We characterize the bias of causal effect estimation arising from this trade-off, connecting the proposed SVM procedure to the existing kernel balancing methods. Finally, we conduct simulation and empirical studies to evaluate the performance of the proposed methodology and find that SVM is competitive with the state-of-the-art covariate balancing methods.

Motivation & Objective

• To develop a new method for estimating average treatment effects (ATE) in observational studies using support vector machines (SVM).
• To establish SVM as a tool for covariate balancing by framing its dual coefficients as weighting schemes that minimize maximum mean discrepancy (MMD).
• To connect SVM to existing causal inference methods, including cardinality matching and kernel optimal matching (KOM), through theoretical and algorithmic insights.
• To demonstrate that the SVM regularization path enables efficient exploration of the balance-sample size frontier, aiding in optimal parameter selection.

Proposed method

• Adapt the dual optimization problem of soft-margin SVM to produce balancing weights that minimize MMD between treatment and control groups.
• Use the dual coefficients of SVM as kernel-based weights that simultaneously maximize effective sample size.
• Interpret the SVM dual problem as a continuous relaxation of the quadratic integer program for finding the largest balanced subset, linking it to cardinality matching.
• Leverage existing SVM path algorithms to compute solutions across the entire regularization path, generating the balance-sample size frontier.
• Relate the method to kernel optimal matching (KOM) by showing that SVM weights minimize worst-case bias due to prognostic score imbalance.
• Use the path algorithm to analyze the trade-off between covariate balance and effective sample size, informing robust parameter choice.

Experimental results

Research questions

• RQ1 Can SVM be used to estimate average treatment effects by achieving covariate balance in observational studies?
• RQ2 How does the SVM dual formulation relate to the problem of finding the largest balanced subset in matching methods?
• RQ3 What is the role of the SVM regularization parameter in balancing the trade-off between covariate balance and effective sample size?
• RQ4 How does the proposed method compare to state-of-the-art weighting and matching methods in terms of bias, variance, and robustness?
• RQ5 Can the SVM path algorithm be used to systematically explore and visualize the balance-sample size frontier for causal inference?

Key findings

• The SVM dual coefficients yield balancing weights that minimize the maximum mean discrepancy (MMD) between treatment and control groups while maximizing effective sample size.
• The SVM dual problem is a continuous relaxation of the integer program for finding the largest balanced subset, establishing a direct link to cardinality matching.
• The regularization parameter in SVM controls the trade-off between covariate balance and effective sample size, enabling systematic exploration of the balance-sample size frontier via path algorithms.
• In simulations and empirical analysis, SVM produces ATE estimates that are competitive with state-of-the-art methods like KOM and CARD, with the most balanced solution (SVMbalance) showing higher standard errors and wider confidence intervals.
• The SVM path reveals that poor regularization parameter choice—such as selecting the most balanced solution—can lead to overly pruned samples and unstable estimates, as seen in the RBF setting where the 95% CI for SVMbalance overlapped with zero.
• The method outperforms KCB and CARD in terms of precision and effect estimate stability under polynomial and RBF kernels, with KOM showing higher estimates but larger standard errors in the polynomial case.