[Paper Review] Conditional Independence Testing using Generative Adversarial Networks

Alexis Bellot, Mihaela van der Schaar|arXiv (Cornell University)|Jul 9, 2019

Machine Learning and Algorithms29 references22 citations

TL;DR

This paper proposes a novel conditional independence test using generative adversarial networks (GANs) to model the null hypothesis of conditional independence, enabling high-dimensional testing with improved statistical power. By training a GAN to generate data under the null, the method achieves valid type I error control and superior power over existing methods, especially in high-dimensional settings, as validated on synthetic and genetic data.

ABSTRACT

We consider the hypothesis testing problem of detecting conditional dependence, with a focus on high-dimensional feature spaces. Our contribution is a new test statistic based on samples from a generative adversarial network designed to approximate directly a conditional distribution that encodes the null hypothesis, in a manner that maximizes power (the rate of true negatives). We show that such an approach requires only that density approximation be viable in order to ensure that we control type I error (the rate of false positives); in particular, no assumptions need to be made on the form of the distributions or feature dependencies. Using synthetic simulations with high-dimensional data we demonstrate significant gains in power over competing methods. In addition, we illustrate the use of our test to discover causal markers of disease in genetic data.

Motivation & Objective

• Address the challenge of conditional independence testing in high-dimensional data where traditional methods suffer from low power due to spurious correlations.
• Develop a test that maintains valid type I error control without requiring parametric assumptions on the underlying distributions or dependence structures.
• Improve statistical power in high-dimensional settings by leveraging generative modeling to simulate data under the null hypothesis of conditional independence.
• Enable robust causal discovery in complex, high-dimensional biological data such as genetic markers and drug response relationships.

Proposed method

• Train a conditional GAN to model the joint distribution under the null hypothesis $\mathcal{H}_0: X \perp\! olimits\perp Y|Z $, generating samples where $X$ and $Y$ are conditionally independent given $Z$.
• Use the generated samples to empirically estimate the null distribution of a test statistic measuring $X$-$Y$ dependence under $\mathcal{H}_0$.
• Define a test statistic based on the dependence between $X$ and $Y$ in both real and generated data, using a discriminator trained to distinguish real from generated samples.
• Optimize the GAN with a trade-off hyperparameter $\lambda$ to balance density approximation accuracy and power, ensuring type I error control.
• Compute a lower bound on the test statistic's distribution under $\mathcal{H}_0$ via numerical integration to validate the test's validity.
• Compare the observed test statistic on real data to the empirical null distribution from generated samples to compute a $p$-value and make a decision on $\mathcal{H}_0$.

Experimental results

Research questions

• RQ1 Can a GAN-based approach achieve valid type I error control in conditional independence testing without assuming specific parametric forms for the joint distribution?
• RQ2 Does the proposed method maintain high statistical power in high-dimensional settings where traditional kernel-based and permutation-based tests fail?
• RQ3 How does the trade-off between density approximation quality and test power affect the performance of the GAN-based test?
• RQ4 Can the method effectively identify non-linear, high-dimensional dependencies in real-world genetic data, such as those between mutations and drug response?
• RQ5 How does the GAN-based test compare to established methods like kernel-based tests, permutation tests, and elastic net regression in identifying causal genetic markers?

Key findings

• The proposed GAN-based conditional independence test (GCIT) achieves approximately valid type I error control in finite samples, even in high-dimensional settings, without requiring parametric assumptions.
• In synthetic simulations, GCIT demonstrated significant power gains over competing methods, particularly as dimensionality increased, with improved detection of conditional dependence.
• On real genetic data from the CCLE dataset, GCIT identified biologically relevant markers such as PIP5K1A and MAP3K5 as conditionally dependent on drug response, consistent with prior biological evidence.
• For the FLT3 gene, GCIT correctly identified no conditional dependence, which was later confirmed by a follow-up genetic study, demonstrating its reliability in avoiding false positives.
• The method outperformed elastic net regression and random forest importance scores in detecting non-linear dependencies, highlighting its advantage in capturing complex relationships.
• The trade-off between GAN training quality and test power was empirically validated: higher $\lambda$ values improved power but slightly degraded type I error control, with a clear trade-off observed in Figure 3.