[Paper Review] Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data

Sindy Löwe, David Madras+2 more|arXiv (Cornell University)|Jun 18, 2020

Bayesian Modeling and Causal Inference58 references70 citations

TL;DR

Proposes Amortized Causal Discovery (ACD), a variational two-part model that learns to infer causal graphs across time-series with shared dynamics, improving causal discovery, even with noise and hidden confounding.

ABSTRACT

On time-series data, most causal discovery methods fit a new model whenever they encounter samples from a new underlying causal graph. However, these samples often share relevant information which is lost when following this approach. Specifically, different samples may share the dynamics which describe the effects of their causal relations. We propose Amortized Causal Discovery, a novel framework that leverages such shared dynamics to learn to infer causal relations from time-series data. This enables us to train a single, amortized model that infers causal relations across samples with different underlying causal graphs, and thus leverages the shared dynamics information. We demonstrate experimentally that this approach, implemented as a variational model, leads to significant improvements in causal discovery performance, and show how it can be extended to perform well under added noise and hidden confounding.

Motivation & Objective

• Motivate causal discovery across time-series with different underlying graphs but shared dynamics.
• Introduce an amortized encoder-decoder framework separating graph prediction from dynamics modeling.
• Develop a probabilistic (variational) implementation enabling learning across samples.
• Enhance robustness to noise and hidden confounding via latent representations.
• Demonstrate scalability and performance improvements over prior methods on diverse datasets.

Proposed method

• Two-component model: an amortized encoder f_phi predicts a causal graph G for a time-series x, and a decoder f_theta models the system dynamics given x and G.
• Formalization where x^{t+1} ≈ g(x^{≤t}, G) + ε, with G invariant across time but specific per sample.
• Variational objective L = E_{q_phi(z|x)}[log p_theta(x|z)] - KL[q_phi(z|x) || p(z)], where z encodes edges of G.
• Encoder uses a graph neural network to produce a distribution over edge types and applies Gumbel-softmax for differentiability.
• Decoder propagates messages along predicted edges and predicts next-step values via a neural function, incorporating self-loops.
• No-edge type is included to mirror Granger causality (no edge means no influence).
• Test-time adaptation (TTA) allows refining the predicted graph at test time for better low-data performance.
• Extension to hidden confounding by introducing latent variables in the encoder to model unobserved influences, improving robustness.

Experimental results

Research questions

• RQ1 Can we learn a shared dynamics model across samples with different underlying causal graphs?
• RQ2 Does decoupling graph prediction from dynamics modeling improve Granger-causal discovery across datasets?
• RQ3 How does ACD perform under observation noise and hidden confounding?
• RQ4 Can test-time adaptation and latent confounders enhance robustness and accuracy?
• RQ5 Is ACD applicable to standard single-graph causal discovery and to multi-graph scenarios?

Key findings

• ACD substantially outperforms baseline causal discovery methods on multiple datasets (e.g., Kuramoto, Netsim) in AUROC.
• On Kuramoto, Amortized Causal Discovery achieves AUROC 0.952, outperforming all listed baselines.
• On Netsim, Amortized Causal Discovery achieves AUROC 0.688, competitive with or surpassing several baselines.
• ACD’s encoder-based predictions improve with more training data, and test-time adaptation yields strong performance in low-data regimes.
• ACD remains robust to added observational noise, with graceful degradation in AUROC as noise increases.
• Latent-variable extensions for hidden confounding improve edge recovery and predictive performance in experiments with unobserved temperature or time-series.