[Paper Review] Distinguishing cause from effect using observational data: methods and benchmarks

Joris M. Mooij, Jonas Peters+3 more|arXiv (Cornell University)|Dec 11, 2014

Bayesian Modeling and Causal InferenceComputer Science100 references324 citations

TL;DR

This paper proposes and evaluates methods for distinguishing cause from effect using only observational data, focusing on Additive Noise Models (ANM) and Information Geometric Causal Inference (IGCI). It introduces the CauseEffectPairs benchmark with 100 real-world cause-effect pairs across diverse domains, demonstrating that ANM achieves 63% accuracy and 0.74 AUC on real data, with theoretical consistency proven for the method.

ABSTRACT

The discovery of causal relationships from purely observational data is a fundamental problem in science. The most elementary form of such a causal discovery problem is to decide whether X causes Y or, alternatively, Y causes X, given joint observations of two variables X, Y. An example is to decide whether altitude causes temperature, or vice versa, given only joint measurements of both variables. Even under the simplifying assumptions of no confounding, no feedback loops, and no selection bias, such bivariate causal discovery problems are challenging. Nevertheless, several approaches for addressing those problems have been proposed in recent years. We review two families of such methods: Additive Noise Methods (ANM) and Information Geometric Causal Inference (IGCI). We present the benchmark CauseEffectPairs that consists of data for 100 different cause-effect pairs selected from 37 datasets from various domains (e.g., meteorology, biology, medicine, engineering, economy, etc.) and motivate our decisions regarding the "ground truth" causal directions of all pairs. We evaluate the performance of several bivariate causal discovery methods on these real-world benchmark data and in addition on artificially simulated data. Our empirical results on real-world data indicate that certain methods are indeed able to distinguish cause from effect using only purely observational data, although more benchmark data would be needed to obtain statistically significant conclusions. One of the best performing methods overall is the additive-noise method originally proposed by Hoyer et al. (2009), which obtains an accuracy of 63+-10 % and an AUC of 0.74+-0.05 on the real-world benchmark. As the main theoretical contribution of this work we prove the consistency of that method.

Motivation & Objective

• To address the fundamental challenge of inferring causal direction from purely observational data in the bivariate case.
• To develop and evaluate methods that distinguish cause from effect without requiring experimental interventions or additional observed variables.
• To create a standardized benchmark for evaluating bivariate causal discovery methods on real-world data.
• To assess the empirical performance of ANM and IGCI methods on real and simulated data under realistic assumptions.
• To provide theoretical consistency guarantees for the additive noise model approach to causal discovery.

Proposed method

• Proposes the Additive Noise Model (ANM) as a method to infer causal direction by testing whether the data can be modeled as Y = f(X) + N, where N is independent of X.
• Applies Information Geometric Causal Inference (IGCI) by leveraging the asymmetry in information geometry, particularly the independence of the error term from the cause.
• Employs Gaussian Process regression for non-parametric estimation of the function f in the ANM framework.
• Uses entropy estimators, including the spacing estimator and ent-PSD, to compute information-theoretic measures in IGCI.
• Implements a benchmark dataset called CauseEffectPairs, comprising 100 cause-effect pairs from 37 real-world domains with manually curated ground truth.
• Evaluates methods on both real-world data and artificially simulated data, using metrics like accuracy and AUC.

Experimental results

Research questions

• RQ1 Can causal direction be reliably inferred from observational data when only two variables are observed?
• RQ2 How well do ANM and IGCI methods perform on real-world data compared to simulated data?
• RQ3 What is the empirical performance of ANM and IGCI on the newly proposed CauseEffectPairs benchmark?
• RQ4 Is the additive noise model method consistent under the assumptions of the framework?
• RQ5 How do different entropy estimators and implementation choices affect the performance and computation time of IGCI and ANM?

Key findings

• The additive noise model (ANM) method achieves 63% accuracy and an AUC of 0.74 on the real-world CauseEffectPairs benchmark, indicating it can successfully infer causal direction from observational data.
• IGCI methods are significantly faster than ANM methods, being approximately two orders of magnitude faster, due to the absence of non-parametric regression in IGCI.
• The ANM-MML variant is notably slower due to the computationally intensive MML estimation of marginal distributions.
• The ent-PSD entropy estimator is the slowest among IGCI implementations but is the only one that shows robustness to data perturbations.
• Computation times vary across methods, with Gaussian Process regression being the primary bottleneck in ANM implementations.
• Theoretical consistency of the ANM method is proven under the model assumptions, providing a strong foundation for its use in causal discovery.