[Paper Review] Conditional independences and causal relations implied by sets of equations

Tineke Blom, Mirthe M. van Diepen+1 more|arXiv (Cornell University)|Jul 14, 2020

Bayesian Modeling and Causal Inference31 references4 citations

TL;DR

This paper introduces a novel framework for analyzing causal and probabilistic relationships in systems of equations by leveraging Simon's causal ordering algorithm to construct a causal ordering graph (COG) and a Markov ordering graph (MOG). Under unique solvability assumptions, the COG captures intervention effects—especially soft and perfect interventions—while the MOG encodes conditional independences in observational data, resolving ambiguities in traditional causal models like SCM and Bayesian networks.

ABSTRACT

Real-world complex systems are often modelled by sets of equations with endogenous and exogenous variables. What can we say about the causal and probabilistic aspects of variables that appear in these equations without explicitly solving the equations? We make use of Simon's causal ordering algorithm (Simon, 1953) to construct a causal ordering graph and prove that it expresses the effects of soft and perfect interventions on the equations under certain unique solvability assumptions. We further construct a Markov ordering graph and prove that it encodes conditional independences in the distribution implied by the equations with independent random exogenous variables, under a similar unique solvability assumption. We discuss how this approach reveals and addresses some of the limitations of existing causal modelling frameworks, such as causal Bayesian networks and structural causal models.

Motivation & Objective

• To address the limitations of existing causal modeling frameworks—particularly causal Bayesian networks and structural causal models (SCMs)—in representing both causal relations and conditional independences unambiguously.
• To formalize soft and perfect interventions on systems of equations and show how they affect variable distributions without solving the equations explicitly.
• To demonstrate that standard causal discovery algorithms (e.g., PC) may fail to provide a clear causal interpretation when applied to data from dynamical systems with feedback.
• To establish a formal link between equation-based models and graphical representations that encode both intervention responses and conditional independence constraints.
• To show that the causal ordering graph (COG) and Markov ordering graph (MOG) provide a more accurate and scalable representation of causal and probabilistic structure than traditional SCM graphs in certain settings.

Proposed method

• Adapt and extend Simon’s causal ordering algorithm (1953) to construct a directed cluster graph, the causal ordering graph (COG), which encodes the causal structure implied by a set of equations.
• Define a bipartite graph representation B = ⟨V, F, E⟩, where V is the set of endogenous variables and F the set of equations, with edges indicating variable-equation dependencies.
• Construct the Markov ordering graph (MOG) from the COG by projecting the structure onto the variable vertices, which encodes conditional independences under the assumption of independent exogenous variables.
• Introduce the concept of maximal unique solvability w.r.t. the COG, ensuring that solutions to the equations are well-defined and path-dependent on the topological order of clusters.
• Formalize soft and perfect interventions on the bipartite graph and prove that the COG correctly represents their effects, with interventions altering cluster structure and parent relationships.
• Prove that under maximal unique solvability, interventions on variables in SV and equations in SF preserve the topological ordering of clusters in the COG, and that variable solutions remain unchanged if no directed path exists from intervened variables to the target.

Experimental results

Research questions

• RQ1 Can a single graphical model simultaneously represent both conditional independences and intervention responses in equation-based systems?
• RQ2 How do soft and perfect interventions affect the distribution of endogenous variables in systems of equations, and how can these effects be captured without solving the equations?
• RQ3 Why do standard causal discovery algorithms like PC fail to provide a clear causal interpretation when applied to data from dynamical systems with feedback?
• RQ4 What is the relationship between the causal ordering graph (COG) and the structural causal model (SCM) graph, and in what ways can COG provide a more accurate or stronger Markov property?
• RQ5 Under what conditions does the Markov ordering graph (MOG) correctly encode conditional independences in the joint distribution implied by a system of equations with independent exogenous variables?

Key findings

• The causal ordering graph (COG) correctly represents the effects of soft and perfect interventions on equation-based systems under the assumption of maximal unique solvability.
• The Markov ordering graph (MOG), derived from the COG, encodes conditional independences in the observational distribution of variables when exogenous variables are independent.
• For systems with feedback at equilibrium, the output of the PC algorithm may not have a straightforward causal interpretation, as it fails to capture intervention effects accurately.
• The COG and MOG are shown to be more expressive than standard SCM graphs in modeling interventions and conditional independences, especially in systems with cycles or feedback.
• When there is no directed path from an intervened variable to a target variable in the COG, the solution of the target variable remains unchanged almost surely after intervention.
• If a directed path exists from an intervened variable to a target variable in the COG, the distribution of the target variable may change after intervention, which is captured by the altered cluster dependencies in the COG.