[Paper Review] Latent Variable Discovery Using Dependency Patterns

Xuhui Zhang, Kevin B. Korb+2 more|arXiv (Cornell University)|Jul 22, 2016

Bayesian Modeling and Causal Inference3 references4 citations

TL;DR

This paper proposes a systematic method to identify dependency patterns—called 'triggers'—that signal the presence of latent variables in Bayesian networks. By exhaustively analyzing d-separation structures in fully observed DAGs and comparing them to those generated by single-cause latent variable models, the authors identify dependency patterns that cannot be explained by observed-only models, enabling more rigorous latent variable discovery in causal inference.

ABSTRACT

The causal discovery of Bayesian networks is an active and important research area, and it is based upon searching the space of causal models for those which can best explain a pattern of probabilistic dependencies shown in the data. However, some of those dependencies are generated by causal structures involving variables which have not been measured, i.e., latent variables. Some such patterns of dependency "reveal" themselves, in that no model based solely upon the observed variables can explain them as well as a model using a latent variable. That is what latent variable discovery is based upon. Here we did a search for finding them systematically, so that they may be applied in latent variable discovery in a more rigorous fashion.

Motivation & Objective

• To identify dependency patterns in observed variables that can only be explained by latent variable models, not by fully observed models.
• To address the challenge of detecting unmeasured confounders in causal discovery by formalizing 'triggers' that signal latent variable presence.
• To provide a systematic, data preprocessing method to guide causal discovery algorithms toward models with latent variables.
• To improve the accuracy and efficiency of latent variable discovery by focusing search on structures that inherently favor latent variable explanations.

Proposed method

• Enumerated all possible fully observed DAGs for 3, 4, and 5 observed variables, excluding isolated nodes.
• For each DAG, computed all d-separating evidence sets and derived the corresponding dependency matrices using d-separation criteria.
• Generated all possible single-cause latent variable models where the latent variable is a common cause of two or more observed variables.
• Computed the dependency structures in observed variables under each latent variable model, conditioned on all possible evidence sets.
• Identified triggers as dependency sets from latent models that cannot be matched by any fully observed DAG under d-separation.
• Used the resulting triggers to guide a modified PC algorithm (trigger-PC), enhancing detection of latent structures in simulated data.

Experimental results

Research questions

• RQ1 Which dependency patterns in observed variables cannot be explained by any fully observed Bayesian network, indicating the presence of latent variables?
• RQ2 How can we systematically identify such dependency patterns as 'triggers' for latent variable discovery?
• RQ3 Can these triggers be used to improve the performance of causal discovery algorithms in detecting latent confounders?
• RQ4 How do the number and structure of triggers scale with the number of observed variables?
• RQ5 What is the empirical effectiveness of using triggers in a modified PC algorithm for latent variable detection?

Key findings

• For three observed variables, no triggers were found—any dependency pattern explainable by a latent model could also be explained by a fully observed DAG.
• For four observed variables, two distinct triggers were identified, each corresponding to a unique dependency structure that cannot be replicated by any fully observed model.
• For five observed variables, 57 unique triggers were discovered, demonstrating a significant increase in complex dependency patterns that signal latent confounding.
• In simulated data with 5 observed variables, the trigger-PC algorithm achieved up to 48% true positive detection of latent structures (e.g., 48 out of 57 cases with 10,000 samples and medium arc strength when variables had 3 states).
• The trigger-PC algorithm significantly outperformed standard PC in detecting latent structures, especially under medium and high arc strength conditions, with zero false positives in some configurations.
• The method remains robust across varying data sample sizes and variable cardinalities, with consistent detection performance even at low arc strengths when sufficient data was available.