[Paper Review] Direct Learning with Guarantees of the Difference DAG Between Structural Equation Models

Asish Ghoshal, Kevin Bello+1 more|arXiv (Cornell University)|Jun 28, 2019

Bayesian Modeling and Causal Inference14 references4 citations

TL;DR

This paper proposes a direct learning algorithm for estimating the structural difference between two linear structural equation models (SEMs) with the same topological ordering, leveraging precision matrix differences to achieve sample efficiency. The method guarantees exact recovery of the difference DAG using only $\mathcal{O}(d^2 \log p)$ samples, where $d$ is the sparsity of the difference DAG, and establishes matching fundamental sample complexity lower bounds.

ABSTRACT

Discovering cause-effect relationships between variables from observational data is a fundamental challenge in many scientific disciplines. However, in many situations it is desirable to directly estimate the change in causal relationships across two different conditions, e.g., estimating the change in genetic expression across healthy and diseased subjects can help isolate genetic factors behind the disease. This paper focuses on the problem of directly estimating the structural difference between two structural equation models (SEMs), having the same topological ordering, given two sets of samples drawn from the individual SEMs. We present an principled algorithm that can recover the difference SEM in $\mathcal{O}(d^2 \log p)$ samples, where $d$ is related to the number of edges in the difference SEM of $p$ nodes. We also study the fundamental limits and show that any method requires at least $Ω(d' \log \frac{p}{d'})$ samples to learn difference SEMs with at most $d'$ parents per node. Finally, we validate our theoretical results with synthetic experiments and show that our method outperforms the state-of-the-art. Moreover, we show the usefulness of our method by using data from the medical domain.

Motivation & Objective

• To directly estimate the difference in causal structures between two SEMs under the same topological ordering, avoiding indirect estimation via individual model recovery.
• To achieve finite-sample guarantees for difference DAG recovery, especially when the individual SEMs are dense but their difference is sparse.
• To establish fundamental sample complexity limits for learning difference SEMs, showing a lower bound of $\Omega(d'\log(p/d'))$ samples.
• To validate the method's superiority over state-of-the-art approaches through synthetic and real-world experiments, including a medical fMRI dataset.

Proposed method

• The algorithm recursively eliminates vertices and re-estimates the difference in precision matrices over the remaining variables, exploiting existing methods for precision matrix difference estimation.
• It avoids estimating individual regression coefficients or noise variances in the original SEMs, focusing solely on the difference structure.
• The method relies on a mutual incoherence condition to ensure stable recovery of the difference DAG from the estimated precision matrix difference.
• Finite-sample guarantees are derived using concentration inequalities, with sample complexity depending on the $\ell_1$-norm of the difference precision matrix and the minimum diagonal variance across conditions.
• The algorithm is implemented in a finite-sample version with a threshold $\epsilon$ to control estimation error, ensuring exact recovery with high probability.
• Theoretical analysis leverages results from high-dimensional statistics, particularly on difference of precision matrices, to derive tight bounds.

Experimental results

Research questions

• RQ1 Can we directly estimate the difference DAG between two SEMs with fewer samples than indirect methods that estimate individual models first?
• RQ2 What is the fundamental sample complexity lower bound for learning the difference DAG between two SEMs with at most $d'$ parents per node?
• RQ3 How does the sample complexity of direct difference estimation compare to indirect estimation via individual model recovery?
• RQ4 Under what conditions can we achieve finite-sample guarantees for difference DAG recovery?
• RQ5 Does the proposed method outperform existing approaches in both synthetic and real-world settings?

Key findings

• The proposed algorithm achieves exact recovery of the difference DAG with high probability using $\mathcal{O}(d^2 \log p)$ samples, where $d$ is the number of edges in the difference DAG.
• The method's sample complexity depends only on the sparsity of the difference DAG and the $\ell_1$-norm of the difference precision matrix, not on the full model complexity.
• A matching lower bound of $\Omega(d'\log(p/d'))$ samples is established, showing the method is sample-optimal up to logarithmic factors.
• The method outperforms indirect approaches that estimate individual SEMs first, especially when the individual models are dense but the difference is sparse.
• In real-world fMRI experiments, the method successfully identified biologically relevant changes in brain connectivity, particularly in the visual cortex, with only 12 out of 157 regions participating in the difference DAG.
• Synthetic experiments confirm the logarithmic dependence on the number of variables $p$, validating the theoretical $\mathcal{O}(\log p)$ sample complexity scaling.