[Paper Review] Bayesian Causal Induction

Pedro A. Ortega|arXiv (Cornell University)|Nov 3, 2011

Bayesian Modeling and Causal Inference8 references3 citations

TL;DR

This paper proposes a Bayesian causal induction framework using probability trees to model competing causal hypotheses, where interventions and structural constraints enable causal inference and extrapolation. It demonstrates that interventions break symmetry between hypotheses, allowing posterior belief updates (e.g., P(h|ŷ,x) = 3/5), and that constraints enable generalization from observed to unobserved variables.

ABSTRACT

Discovering causal relationships is a hard task, often hindered by the need for intervention, and often requiring large amounts of data to resolve statistical uncertainty. However, humans quickly arrive at useful causal relationships. One possible reason is that humans extrapolate from past experience to new, unseen situations: that is, they encode beliefs over causal invariances, allowing for sound generalization from the observations they obtain from directly acting in the world. Here we outline a Bayesian model of causal induction where beliefs over competing causal hypotheses are modeled using probability trees. Based on this model, we illustrate why, in the general case, we need interventions plus constraints on our causal hypotheses in order to extract causal information from our experience.

Motivation & Objective

• To address the challenge of causal induction—inferring causal structure from observational data—when multiple hypotheses yield identical joint distributions.
• To model causal hypotheses as random variables operating at a meta-level, overcoming limitations of standard graphical models.
• To enable causal generalization from observed variables (X, Y) to unobserved ones (U, V) using constraints on possible causal structures.
• To formalize interventions within the probability tree framework to predict manipulated system behavior and break statistical symmetry between hypotheses.
• To investigate how constraints on causal hypotheses and interventions jointly enable extraction and extrapolation of causal knowledge.

Proposed method

• Represent causal hypotheses (e.g., X→Y vs. Y→X) as branches in a probability tree, where each path encodes a sequence of causal resolutions.
• Model the hypothesis H as a root-level random variable that determines the resolution order of downstream variables (e.g., X before Y under H=h).
• Use conditional probabilities to encode stochastic dependencies within each causal hypothesis, such as P(Y=y|h, ¬x) = 1/4.
• Apply interventions (e.g., ŷ for X=x) by fixing the probability mass on a specific outcome, thereby altering the posterior distribution over hypotheses.
• Compute posterior probabilities using Bayes' theorem, e.g., P(h|ŷ,x) = [P(y|h,ŷ)P(ŷ|h)P(h)] / [P(y|h,ŷ)P(ŷ|h)P(h) + P(ŷ|¬h,y)P(y|¬h)P(¬h)].
• Enforce constraints on causal hypotheses (e.g., only X→Y and U→V, not mixed) to enable extrapolation from known to unobserved systems.

Experimental results

Research questions

• RQ1 How can we distinguish between two causal hypotheses (e.g., X→Y vs. Y→X) when they produce identical joint distributions?
• RQ2 What role do interventions play in breaking statistical symmetry between competing causal hypotheses?
• RQ3 Under what conditions can causal knowledge from one system (X,Y) be extrapolated to another (U,V)?
• RQ4 How do constraints on the set of possible causal hypotheses affect the ability to infer and generalize causal structure?
• RQ5 Can a Bayesian framework based on probability trees model meta-level causal reasoning without requiring explicit graphical model extensions?

Key findings

• Interventions introduce statistical asymmetry between hypotheses, enabling causal inference: after intervening on X (X=x), P(h|ŷ,x) = 3/5, favoring X→Y.
• The posterior probability P(h|ŷ,x) = 3/5 is identical when extrapolating from X,Y to U,V, demonstrating successful generalization under structural constraints.
• Without constraints on causal hypotheses, extrapolation from X,Y to U,V would not be possible, as multiple causal structures would remain indistinguishable.
• The probability tree framework naturally encodes heterogeneous resolution orders (e.g., X before Y under H=h, Y before X under H=¬h), modeling different causal hypotheses.
• Constraints on causal hypotheses are essential for generalization, as they reduce ambiguity and allow transfer of causal knowledge to unobserved systems.
• The framework formalizes the idea that 'no causes in, no causes out'—causal knowledge requires both interventions and structural assumptions to be extracted.