[Paper Review] A polynomial-time algorithm for learning nonparametric causal graphs

Ming Gao, Yi Ding+1 more|arXiv (Cornell University)|Jun 22, 2020

Bayesian Modeling and Causal Inference58 references4 citations

TL;DR

This paper presents a polynomial-time algorithm for learning nonparametric causal DAGs without requiring linearity, additivity, independent noise, or faithfulness. It uses a model-agnostic approach with nonparametric estimators and provides explicit finite-sample guarantees, achieving $O(nd^5)$ time complexity and $\Omega((d^2/\varepsilon)^{1+0pt/2})$ sample complexity, with improved scalability under sparsity or smoothness assumptions.

ABSTRACT

We establish finite-sample guarantees for a polynomial-time algorithm for learning a nonlinear, nonparametric directed acyclic graphical (DAG) model from data. The analysis is model-free and does not assume linearity, additivity, independent noise, or faithfulness. Instead, we impose a condition on the residual variances that is closely related to previous work on linear models with equal variances. Compared to an optimal algorithm with oracle knowledge of the variable ordering, the additional cost of the algorithm is linear in the dimension $d$ and the number of samples $n$. Finally, we compare the proposed algorithm to existing approaches in a simulation study.

Motivation & Objective

• To develop a provably efficient and statistically consistent algorithm for learning nonlinear, nonparametric DAGs from data.
• To remove restrictive assumptions such as linearity, additivity, independent noise, and faithfulness commonly used in prior causal structure learning methods.
• To provide explicit, nonasymptotic finite-sample complexity bounds for both statistical accuracy and computational efficiency.
• To unify and generalize identifiability conditions based on residual variances to model-free settings.
• To demonstrate that greedy algorithms like CAM can provably fail to recover the true DAG, highlighting the need for more robust methods.

Proposed method

• Extends existing variable-ordering algorithms (e.g., Ghoshal & Honorio, Chen et al.) to a nonparametric, model-free setting using plug-in nonparametric estimators (e.g., kernel smoothers, neural networks).
• Imposes a residual variance condition related to equal variances in linear models, which enables identifiability without parametric assumptions.
• Analyzes the algorithm’s time and sample complexity in terms of the chosen nonparametric estimator, making the dependence explicit.
• Employs a layer decomposition of the DAG to guide the topological ordering process, improving computational tractability.
• Uses generalized additive models (GAMs) as a practical instantiation for empirical evaluation.
• Introduces a pre-processing step (neighbourhood selection) to improve runtime, especially for high-dimensional settings.

Experimental results

Research questions

• RQ1 Can a nonparametric causal DAG learning algorithm be constructed with both finite-sample statistical guarantees and polynomial-time computational complexity?
• RQ2 Does the proposed method remain consistent and efficient under model-free assumptions, without requiring linearity, additivity, or independent noise?
• RQ3 How does the algorithm’s performance compare to state-of-the-art methods in terms of structure learning accuracy and runtime, especially on dense graphs with hub nodes?
• RQ4 Can identifiability be established in nonparametric settings using residual variance conditions, and how does this generalize prior results?
• RQ5 Why do greedy algorithms like CAM fail to recover the correct DAG in identifiable settings, and can this be formally characterized?

Key findings

• The proposed algorithm runs in $O(nd^5)$ time and requires $\Omega((d^2/\varepsilon)^{1+0pt/2})$ samples, with the exponential dependence on $d$ reducible under sparsity or smoothness assumptions.
• The method outperforms most state-of-the-art algorithms in simulations, particularly on denser graphs with hub nodes, when implemented with generalized additive models.
• Greedy algorithms such as CAM can provably fail to recover the true DAG even when identifiability conditions are satisfied, as demonstrated in Example 5 and Figure 1(a).
• The inclusion of a pre-processing step (neighbourhood selection) reduces CAM’s runtime from over 8.5 hours to under 90 seconds for $d=40$, highlighting its computational impact.
• For $d=40$, NPVAR runs in ~2 minutes, while CAM without pre-processing takes over 8.5 hours, underscoring the importance of efficient pre-processing.
• The algorithm achieves strong performance across multiple settings, with SHD (structural Hamming distance) improving with sample size and maintaining low error even at $d=70$.