[Paper Review] Combining Linear Non-Gaussian Acyclic Model with Logistic Regression Model for Estimating Causal Structure from Mixed Continuous and Discrete Data

Chao Li, Shohei Shimizu|arXiv (Cornell University)|Feb 16, 2018

Bayesian Modeling and Causal Inference10 references3 citations

TL;DR

This paper proposes a hybrid causal model combining Linear Non-Gaussian Acyclic Models (LiNGAM) for continuous variables and logistic regression for discrete variables, enabling identifiability of full causal structures from mixed data without discretization. The method uses a BIC scoring function for model selection and demonstrates asymptotic consistency in simulations, outperforming discretization-based approaches like PC algorithm.

ABSTRACT

Estimating causal models from observational data is a crucial task in data analysis. For continuous-valued data, Shimizu et al. have proposed a linear acyclic non-Gaussian model to understand the data generating process, and have shown that their model is identifiable when the number of data is sufficiently large. However, situations in which continuous and discrete variables coexist in the same problem are common in practice. Most existing causal discovery methods either ignore the discrete data and apply a continuous-valued algorithm or discretize all the continuous data and then apply a discrete Bayesian network approach. These methods possibly loss important information when we ignore discrete data or introduce the approximation error due to discretization. In this paper, we define a novel hybrid causal model which consists of both continuous and discrete variables. The model assumes: (1) the value of a continuous variable is a linear function of its parent variables plus a non-Gaussian noise, and (2) each discrete variable is a logistic variable whose distribution parameters depend on the values of its parent variables. In addition, we derive the BIC scoring function for model selection. The new discovery algorithm can learn causal structures from mixed continuous and discrete data without discretization. We empirically demonstrate the power of our method through thorough simulations.

Motivation & Objective

• To address the limitation of existing causal discovery methods that either ignore discrete variables or discretize continuous variables, leading to information loss or approximation errors.
• To develop a unified causal model that simultaneously handles continuous and discrete variables using identifiable structural assumptions.
• To derive a BIC scoring function for model selection in mixed-variable settings, enabling consistent causal structure learning.
• To empirically validate the method's performance against state-of-the-art approaches like the PC algorithm with discretization.

Proposed method

• The model assumes continuous variables follow a linear non-Gaussian acyclic model (LiNGAM) with additive non-Gaussian noise.
• Discrete variables are modeled using logistic regression, where the log-odds of the outcome depend linearly on parent variables.
• The joint likelihood is factorized based on the DAG structure, combining Gaussian and logistic conditional densities.
• A BIC scoring function is derived to evaluate candidate DAG structures, incorporating model complexity and log-likelihood.
• A search algorithm is implemented in Python to maximize the BIC score over all possible DAGs.
• The method avoids data discretization, preserving information and enabling full causal structure identification.

Experimental results

Research questions

• RQ1 Can a hybrid causal model that combines LiNGAM and logistic regression identify the true causal structure in mixed continuous and discrete data?
• RQ2 Does the proposed BIC scoring function enable consistent recovery of the true DAG as sample size increases?
• RQ3 How does the performance of the proposed method compare to discretization-based approaches like the PC algorithm on mixed data?
• RQ4 To what extent does the number of continuous variables affect the performance of discretization-based methods?
• RQ5 Can the method achieve high accuracy when the skeleton is known, reducing the required sample size?

Key findings

• The proposed method achieves asymptotic consistency, correctly recovering the true causal structure as the number of samples increases.
• The method significantly outperforms the discretized PC algorithm, especially as the number of continuous variables increases, due to reduced information loss from discretization.
• When the skeleton is known, the hybrid-oracle method achieves 80% accuracy in recovering the full DAG structure with only 100 samples.
• The performance of the discretized PC algorithm degrades with more continuous variables, confirming the negative impact of discretization on causal discovery.
• The BIC scoring function enables unique model selection under the proposed assumptions, avoiding Markov equivalence classes.
• The method successfully identifies the full causal structure without discretization, demonstrating the advantage of modeling mixed data types directly.