[Paper Review] A Recursive Markov Boundary-Based Approach to Causal Structure Learning

Ehsan Mokhtarian, Sina Akbari+2 more|arXiv (Cornell University)|Oct 10, 2020

Bayesian Modeling and Causal Inference26 references4 citations

TL;DR

This paper proposes MARVEL, a recursive constraint-based causal structure learning method that leverages Markov boundary information to identify and remove 'removable' variables—those whose removal preserves d-separation relations—thereby reducing the number of required conditional independence (CI) tests. The method achieves an upper bound of 𝒪(pΔ_in²2^Δ_in) CI tests, significantly improving upon prior state-of-the-art bounds while establishing a matching lower bound of Ω(p² + pΔ_in2^Δ_in), demonstrating its theoretical efficiency.

ABSTRACT

Constraint-based methods are one of the main approaches for causal structure learning that are particularly valued as they are asymptotically guaranteed to find a structure that is Markov equivalent to the causal graph of the system. On the other hand, they may require an exponentially large number of conditional independence (CI) tests in the number of variables of the system. In this paper, we propose a novel recursive constraint-based method for causal structure learning that significantly reduces the required number of CI tests compared to the existing literature. The idea of the proposed approach is to use Markov boundary information to identify a specific variable that can be removed from the set of variables without affecting the statistical dependencies among the other variables. Having identified such a variable, we discover its neighborhood, remove that variable from the set of variables, and recursively learn the causal structure over the remaining variables. We further provide a lower bound on the number of CI tests required by any constraint-based method. Comparing this lower bound to our achievable bound demonstrates the efficiency of the proposed approach. Our experimental results show that the proposed algorithm outperforms state-of-the-art both on synthetic and real-world structures.

Motivation & Objective

• To reduce the number of conditional independence (CI) tests required in constraint-based causal structure learning, which can otherwise scale exponentially with the number of variables.
• To develop a recursive framework that identifies and removes variables that do not affect statistical dependencies, based on Markov boundary information.
• To establish a theoretical lower bound on the number of CI tests required by any constraint-based method, providing a benchmark for efficiency.
• To demonstrate that the proposed method, MARVEL, achieves a provably tighter upper bound on CI tests than existing approaches.

Proposed method

• Introduces the concept of a 'removable variable'—a variable whose removal preserves all d-separation relations in a DAG, characterized graphically via its neighborhood and Markov boundary structure.
• Employs Markov boundary discovery algorithms (e.g., PC or similar) to identify the Markov boundary of each variable, enabling identification of removable variables.
• Applies a recursive procedure: identify a removable variable, orient its edges, remove it from the graph, and update Markov boundary information for remaining variables.
• Uses the in-degree Δ_in of the causal DAG as a key parameter to bound the complexity, focusing on the worst-case number of CI tests.
• Derives an upper bound of 𝒪(pΔ_in²2^Δ_in) CI tests for MARVEL, leveraging the recursive removal of variables and bounded Markov boundary sizes.
• Establishes a theoretical lower bound of Ω(p² + pΔ_in2^Δ_in) on the number of CI tests required by any constraint-based method, using adversarial graph constructions to prove necessity.

Experimental results

Research questions

• RQ1 Can a recursive approach that leverages Markov boundary information reduce the number of conditional independence tests required in constraint-based causal structure learning?
• RQ2 What is the theoretical minimum number of CI tests required by any constraint-based method, and how does it compare to achievable bounds?
• RQ3 Can the concept of a 'removable variable' be formally defined and characterized graphically to enable efficient recursive structure learning?
• RQ4 Does the proposed MARVEL algorithm achieve a provably better upper bound on CI tests than existing state-of-the-art methods?

Key findings

• MARVEL reduces the worst-case number of required conditional independence tests to 𝒪(pΔ_in²2^Δ_in), which is a significant improvement over the prior state-of-the-art bounds of 𝒪(p²2^α) and 𝒪(p² + pα²2^α), where α is the maximum Markov boundary size.
• The method establishes a matching lower bound of Ω(p² + pΔ_in2^Δ_in), proving that the proposed upper bound is asymptotically optimal up to constant factors.
• The recursive removal of removable variables—defined as variables whose removal preserves d-separation—enables efficient pruning of the variable set while maintaining statistical equivalence.
• Theoretical analysis shows that even in the worst case, the number of CI tests scales sub-exponentially with the number of variables when the in-degree Δ_in is bounded.
• Empirical results on synthetic and real-world datasets demonstrate that MARVEL outperforms state-of-the-art methods in terms of both efficiency and accuracy of causal structure recovery.
• The construction of adversarial graphs (e.g., clustered complete graphs and empty graphs) proves that no constraint-based method can perform fewer than Ω(p² + pΔ_in2^Δ_in) CI tests in the worst case.