[Paper Review] Learning Neural Causal Models from Unknown Interventions

Nan Rosemary Ke, Olexa Bilaniuk+7 more|arXiv (Cornell University)|Oct 2, 2019

Bayesian Modeling and Causal Inference59 references46 citations

TL;DR

This paper introduces SDI, a continuous-optimization method that learns structural causal models from both observational data and unknown interventions, using neural networks and REINFORCE-style gradients to recover DAG structures and generalize to unseen interventions.

ABSTRACT

Promising results have driven a recent surge of interest in continuous optimization methods for Bayesian network structure learning from observational data. However, there are theoretical limitations on the identifiability of underlying structures obtained from observational data alone. Interventional data provides much richer information about the underlying data-generating process. However, the extension and application of methods designed for observational data to include interventions is not straightforward and remains an open problem. In this paper we provide a general framework based on continuous optimization and neural networks to create models for the combination of observational and interventional data. The proposed method is even applicable in the challenging and realistic case that the identity of the intervened upon variable is unknown. We examine the proposed method in the setting of graph recovery both de novo and from a partially-known edge set. We establish strong benchmark results on several structure learning tasks, including structure recovery of both synthetic graphs as well as standard graphs from the Bayesian Network Repository.

Motivation & Objective

• Address identifiability limitations when learning causal graphs from observational data alone.
• Develop a framework that integrates observational and interventional data, even when intervention targets are unknown.
• Enable scalable structure learning for discrete-variable SCMs and partial knowledge scenarios.
• Provide strong empirical benchmarks on synthetic graphs and real-world datasets from the Bayesian Network Repository.

Proposed method

• Parametrize graph structure with a softened adjacency matrix gamma and use Bernoulli(C) edge configurations to sample DAGs.
• Train conditional probability models theta for each variable conditioned on its (sampled) parent set.
• Phase 1: fit theta given observational data under sampled graph configurations via likelihood maximization.
• Phase 2: score graph configurations against interventional data by observing prediction degradation and predict the intervention target when unknown.
• Phase 3: update gamma via a REINFORCE-like gradient using interventional data rewards to refine the edge beliefs.
• Impose a DAG-regularization term to discourage cycles and a sparsity penalty to promote parsimonious graphs.

Experimental results

Research questions

• RQ1 Can the proposed model recover the true causal graph from a mix of observational and unknown interventions?
• RQ2 How does the SDI method compare to state-of-the-art causal discovery methods on real-world datasets?
• RQ3 Does the model generalize to unseen interventions and to partial graph recovery settings?
• RQ4 What is the scalability of SDI with graph size and density?
• RQ5 How well does the method handle unknown intervention targets and partial prior edge information?

Key findings

• SDI achieves strong graph recovery across synthetic graphs and real-world benchmarks, outperforming several baselines on SHD metrics.
• The method can perfectly recover DAGs for multiple synthetic graphs up to a certain size and density, with some difficulty arising for very large or dense graphs.
• SDI generalizes to unseen interventions, maintaining higher log-likelihoods on interventional data than non-causal models.
• In partial graph recovery settings, SDI achieves high accuracy when only a subset of edges is learned, scaling to graphs with dozens of variables.
• An intervention-target prediction heuristic within SDI helps bridge performance between known-target and unknown-target intervention settings.