[Paper Review] Estimating a Causal Order among Groups of Variables in Linear Models

Doris Entner, Patrik O. Hoyer|arXiv (Cornell University)|Jul 9, 2012

Bayesian Modeling and Causal Inference8 references4 citations

TL;DR

This paper proposes a method to estimate causal order among groups of multi-dimensional variables using linear non-Gaussian models, extending LiNGAM to vector-valued groups. It introduces algorithms based on independence testing, pairwise measures, and regularization that outperform ad hoc approaches like mean-based LiNGAM, especially with small samples and high-dimensional groups.

ABSTRACT

The machine learning community has recently devoted much attention to the problem of inferring causal relationships from statistical data. Most of this work has focused on uncovering connections among scalar random variables. We generalize existing methods to apply to collections of multi-dimensional random vectors, focusing on techniques applicable to linear models. The performance of the resulting algorithms is evaluated and compared in simulations, which show that our methods can, in many cases, provide useful information on causal relationships even for relatively small sample sizes.

Motivation & Objective

• To address the limitation of existing causal discovery methods that rely on aggregated scalar variables, which can lose information and yield incorrect results in the large-sample limit.
• To develop computationally efficient algorithms for causal order estimation among groups of variables, leveraging known group structures to avoid exponential complexity.
• To extend scalar-variable causal discovery methods (e.g., LiNGAM, DirectLiNGAM) to multi-dimensional random vectors while preserving statistical consistency.
• To evaluate performance under finite sample sizes and high-dimensional settings, particularly when group-level dependencies and non-Gaussianity are present.

Proposed method

• Adapts the LiNGAM framework to groups of variables by modeling the data-generating process as a set of linear equations with lower block-triangular coefficient matrices.
• Employs a two-stage algorithm: first identifying an exogenous group via independence testing, then iteratively regressing out its effects to recover the causal order.
• Introduces three variants: the Pairwise Measure (based on independence between residuals), the Trace Method (using covariance matrix trace), and GroupDirectLiNGAM (a block-structured extension of DirectLiNGAM).
• Applies L²-regularization to stabilize covariance matrix estimates in high-dimensional, low-sample-size scenarios.
• Uses subgroup sampling and cross-validation to improve robustness when sample size is limited relative to dimensionality.
• Employs non-Gaussianity and conditional independence testing to identify causal order, avoiding Markov equivalence issues common in constraint-based methods.

Experimental results

Research questions

• RQ1 Can causal order among groups of multi-dimensional variables be reliably estimated from finite samples when group-level dependencies and non-Gaussianity are present?
• RQ2 How does the performance of group-level causal discovery compare to ad hoc methods that first aggregate variables to scalars?
• RQ3 To what extent do regularization and subgroup sampling improve causal order estimation in high-dimensional, low-sample-size settings?
• RQ4 Does the proposed method maintain consistency and accuracy when error terms within groups are dependent, violating standard ICA assumptions?

Key findings

• The Pairwise Measure and GroupDirectLiNGAM methods achieve the lowest error rates in simulations, with performance improving significantly with increasing sample size.
• The mean-based DirectLiNGAM approach performs no better than random guessing, indicating that aggregation leads to loss of causal information.
• The modified ICA-LiNGAM method performs better than random but fails to converge with increasing sample size, likely due to violation of i.i.d. error assumptions.
• Subgroup sampling reduces error rates, especially for small sample sizes, with the Naïve Pairwise Measure showing initial advantage but lacking consistency.
• L²-regularization improves stability for the Trace Method and Pairwise Measure in low-sample regimes, particularly when covariance matrices are poorly estimated.
• The proposed methods consistently outperform baseline approaches across all tested configurations, including 5-group models with 6 or 12 variables per group and sample sizes of 200–1000.