[Paper Review] Linear Causal Disentanglement via Interventions

Chandler Squires, Anna Seigal+2 more|arXiv (Cornell University)|Nov 29, 2022

Bayesian Modeling and Causal Inference4 citations

TL;DR

This paper proposes a method for linear causal disentanglement using interventions to uniquely identify latent causal structures from observed data. It proves that interventions on each latent variable are both necessary and sufficient for identifiability, leveraging a novel partial-order-based matrix decomposition to recover the causal model and demonstrating accurate recovery on synthetic, semi-synthetic, and single-cell RNA-seq data.

ABSTRACT

Causal disentanglement seeks a representation of data involving latent variables that relate to one another via a causal model. A representation is identifiable if both the latent model and the transformation from latent to observed variables are unique. In this paper, we study observed variables that are a linear transformation of a linear latent causal model. Data from interventions are necessary for identifiability: if one latent variable is missing an intervention, we show that there exist distinct models that cannot be distinguished. Conversely, we show that a single intervention on each latent variable is sufficient for identifiability. Our proof uses a generalization of the RQ decomposition of a matrix that replaces the usual orthogonal and upper triangular conditions with analogues depending on a partial order on the rows of the matrix, with partial order determined by a latent causal model. We corroborate our theoretical results with a method for causal disentanglement that accurately recovers a latent causal model.

Motivation & Objective

• To establish conditions under which a linear latent causal model is uniquely identifiable from observational and interventional data.
• To address the fundamental problem of non-identifiability in causal disentanglement when interventions are missing.
• To develop a practical method for recovering the latent causal structure from finite-sample data.
• To validate the method on synthetic data, semi-synthetic data, and real single-cell RNA sequencing data.
• To explore the biological interpretability of learned latent variables by linking them to survival outcomes in cancer patients.

Proposed method

• Proposes a generalized RQ decomposition of a matrix, replacing orthogonal and upper triangular constraints with conditions based on a partial order derived from the latent causal graph.
• Uses data from observational contexts and perfect interventions on each latent variable to estimate the mixing matrix $G$, the structural coefficients $B_k$, and the latent structure $H$.
• Applies the generalized RQ decomposition to recover the latent causal model from covariance matrices across multiple contexts.
• Adapts the theoretical algorithm to the finite-sample setting using empirical covariance estimation and matrix decomposition techniques.
• Employs a score-based approach for parameter estimation, with potential for penalized maximum likelihood or gradient-based optimization.
• Validates the method through reconstruction accuracy on synthetic and semi-synthetic datasets and applies it to a single-cell RNA-seq dataset of KRAS-mutated lung cancer patients.

Experimental results

Research questions

• RQ1 What conditions on interventions are necessary and sufficient for unique identification of a linear latent causal model?
• RQ2 Can a generalized matrix decomposition technique be developed to recover the latent causal structure under a partial order derived from the causal graph?
• RQ3 How well can the proposed method recover the true latent causal model in finite-sample settings with synthetic and semi-synthetic data?
• RQ4 Can the learned latent variables in a real biological dataset be meaningfully linked to known biological pathways and clinical outcomes?
• RQ5 To what extent are the parameters of the latent causal model partially identifiable when fewer than $d+1$ contexts are available?

Key findings

• Interventions on each latent variable are both necessary and sufficient for identifiability of the linear causal disentanglement model in the worst case.
• The proposed method accurately recovers the generative model on synthetic and semi-synthetic data, even with high-dimensional latent spaces.
• On a single-cell RNA-seq dataset of 83 genes and 83 KRAS mutations, the method successfully identified five latent variables significantly associated with patient survival (p < 0.1 after correction).
• The learned latent structure highlights key functional residues in KRAS (e.g., G12, G13), consistent with known biological drivers of cancer.
• The generalized RQ decomposition enables unique recovery of the latent causal model by encoding the causal order into matrix factorization constraints.
• The method demonstrates robustness to noise and performs well even with large sample sizes up to $10^8$, as shown in ablation studies.