[Paper Review] Reliable Decision Support using Counterfactual Models

Peter Schulam, Suchi Saria|arXiv (Cornell University)|Mar 30, 2017

Complex Systems and Decision MakingDecision Sciences94 citations

TL;DR

The paper introduces Counterfactual Gaussian Processes (CGP) to predict counterfactual outcomes under sequences of actions from observational time-series data, addressing biases from action-influenced training policies and enabling reliable risk assessment and “what-if” reasoning for personalized treatment planning.

ABSTRACT

Decision-makers are faced with the challenge of estimating what is likely to happen when they take an action. For instance, if I choose not to treat this patient, are they likely to die? Practitioners commonly use supervised learning algorithms to fit predictive models that help decision-makers reason about likely future outcomes, but we show that this approach is unreliable, and sometimes even dangerous. The key issue is that supervised learning algorithms are highly sensitive to the policy used to choose actions in the training data, which causes the model to capture relationships that do not generalize. We propose using a different learning objective that predicts counterfactuals instead of predicting outcomes under an existing action policy as in supervised learning. To support decision-making in temporal settings, we introduce the Counterfactual Gaussian Process (CGP) to predict the counterfactual future progression of continuous-time trajectories under sequences of future actions. We demonstrate the benefits of the CGP on two important decision-support tasks: risk prediction and "what if?" reasoning for individualized treatment planning.

Motivation & Objective

• Motivate the unreliability of standard supervised learning for decision support due to action-policy bias in training data.
• Propose counterfactual prediction as an objective to improve generalization across policies.
• Introduce and formulate the Counterfactual Gaussian Process (CGP) for continuous-time trajectories under action sequences.
• Develop an adjusted maximum likelihood objective to learn CGP from observational traces via a marked point process.
• Demonstrate CGP’s utility in reliable risk prediction and in facilitating “what-if” reasoning for individualized treatment planning.

Proposed method

• Model counterfactuals Y[a] for actions a in a set C within a continuous-time framework.
• Represent data as irregular time-series with actions and outcomes using a marked point process (MPP).
• Parameterize the CGP with an outcome model as a Gaussian process (GP) conditioned on history and actions, and an event/action model to capture timing and choices.
• Derive an adjusted maximum likelihood objective that accounts for the action policy via the MPP intensity and history (Equation 3).
• Impose continuous-time no-unmeasured-confounders (NUC) and non-informative measurement time assumptions to connect the CGP to the target counterfactual (Assumptions 3 and 4).
• Estimate CGP parameters by maximizing the observational-trace likelihood, then use CGP to predict counterfactual trajectories Y[s][a] for decision-support tasks.

Experimental results

Research questions

• RQ1 Can counterfactual-based learning yield predictions that are robust to the action policy used to collect training data?
• RQ2 How can we reliably predict counterfactual trajectories under sequences of future actions in continuous time?
• RQ3 Do counterfactual models provide more reliable risk assessment than standard supervised models in time-series data with policy-driven observations?
• RQ4 Can CGP support “what-if” reasoning for individualized treatment planning in medical data?
• RQ5 What assumptions are necessary to connect CGP learned from observational traces to the true counterfactual model?

Key findings

• CGP yields risk predictions that are stable across training-data policies, unlike baseline GP models which vary with policy.
• In simulated data, CGP risk scores show near-identical ranking and AUC across regimes that satisfy core assumptions, while baseline GP does not.
• Violation of core assumptions (continuous-time NUC, non-informative measurement times) leads CGP to lose stability similarly to baseline models.
• In ICU data, CGP enables qualitative counterfactual reasoning for dialysis treatment impacts on creatinine, and improves predictive MAE over baselines (24h: 0.39 vs higher for baselines; 24-48h: 0.62).
• CGP benefits from modeling treatment effects and heterogeneity via a mixture of GPs for the outcome model.