[Paper Review] Kernel-based Conditional Independence Test and Application in Causal Discovery

Kun Zhang, Jonas Peters+2 more|arXiv (Cornell University)|Feb 14, 2012

Bayesian Modeling and Causal Inference25 references346 citations

TL;DR

This paper proposes a kernel-based conditional independence test (KCI-test) that uses kernel methods to assess conditional independence in high-dimensional continuous data, offering improved performance over existing methods—especially with large conditioning sets or small sample sizes—by leveraging a test statistic with a derived asymptotic distribution under the null hypothesis of conditional independence.

ABSTRACT

Conditional independence testing is an important problem, especially in Bayesian network learning and causal discovery. Due to the curse of dimensionality, testing for conditional independence of continuous variables is particularly challenging. We propose a Kernel-based Conditional Independence test (KCI-test), by constructing an appropriate test statistic and deriving its asymptotic distribution under the null hypothesis of conditional independence. The proposed method is computationally efficient and easy to implement. Experimental results show that it outperforms other methods, especially when the conditioning set is large or the sample size is not very large, in which case other methods encounter difficulties.

Motivation & Objective

• Address the challenge of conditional independence testing in high-dimensional continuous data, a critical task in causal discovery and Bayesian network learning.
• Overcome the curse of dimensionality that plagues traditional conditional independence tests when dealing with continuous variables.
• Develop a computationally efficient and statistically sound method for testing conditional independence that remains effective even when the conditioning set is large.
• Provide a theoretically grounded test statistic with a known asymptotic distribution under the null hypothesis of conditional independence.
• Enable robust causal discovery by accurately identifying conditional independence relationships in complex, high-dimensional datasets.

Proposed method

• Propose a kernel-based test statistic that measures conditional dependence using reproducing kernel Hilbert space (RKHS) norms.
• Construct a test statistic based on the Hilbert-Schmidt independence criterion (HSIC) applied to conditional distributions.
• Derive the asymptotic distribution of the test statistic under the null hypothesis of conditional independence, enabling p-value computation.
• Use a two-sample U-statistic approach to estimate the test statistic efficiently from observed data.
• Apply a centered kernel matrix to remove bias and improve the accuracy of the conditional independence assessment.
• Implement the method using a three-way decomposition of the kernel matrix to handle conditioning on a set of variables.

Experimental results

Research questions

• RQ1 How can conditional independence be reliably tested in high-dimensional continuous data where traditional methods fail due to the curse of dimensionality?
• RQ2 What kernel-based statistic can be constructed to provide a valid and powerful test for conditional independence with known asymptotic properties?
• RQ3 How does the proposed KCI-test perform compared to existing methods when the conditioning set is large or sample size is small?
• RQ4 Can the kernel-based approach maintain statistical power and computational efficiency in real-world causal discovery tasks?
• RQ5 What is the theoretical justification for the test statistic's asymptotic distribution under the null hypothesis of conditional independence?

Key findings

• The KCI-test significantly outperforms existing methods in conditional independence testing, particularly when the conditioning set is large.
• The method maintains high statistical power even with small sample sizes, where other approaches often fail due to overfitting or instability.
• The proposed test statistic has a well-defined asymptotic distribution under the null hypothesis, enabling accurate p-value estimation.
• The kernel-based approach is computationally efficient and scalable, making it suitable for high-dimensional data common in causal discovery.
• Empirical results demonstrate that the KCI-test improves the accuracy of causal structure learning in Bayesian networks compared to baseline methods.
• The method is robust to the choice of kernel bandwidth when properly tuned, showing consistent performance across different data configurations.