[Paper Review] Emergence and Causality in Complex Systems: A Survey on Causal Emergence and Related Quantitative Studies

Bing Yuan, Jiang Zhang+7 more|arXiv (Cornell University)|Dec 28, 2023

Complex Systems and Decision Making4 citations

TL;DR

This paper presents a comprehensive review of causal emergence (CE) theory, proposing that macro-level causality can emerge from micro-level interactions in complex systems. It introduces effective information (EI) as a quantitative measure to detect CE, integrating machine learning and neural networks to identify causal structures from data, with applications in neuroscience, AI, and systems biology.

ABSTRACT

Emergence and causality are two fundamental concepts for understanding complex systems. They are interconnected. On one hand, emergence refers to the phenomenon where macroscopic properties cannot be solely attributed to the cause of individual properties. On the other hand, causality can exhibit emergence, meaning that new causal laws may arise as we increase the level of abstraction. Causal emergence theory aims to bridge these two concepts and even employs measures of causality to quantify emergence. This paper provides a comprehensive review of recent advancements in quantitative theories and applications of causal emergence. Two key problems are addressed: quantifying causal emergence and identifying it in data. Addressing the latter requires the use of machine learning techniques, thus establishing a connection between causal emergence and artificial intelligence. We highlighted that the architectures used for identifying causal emergence are shared by causal representation learning, causal model abstraction, and world model-based reinforcement learning. Consequently, progress in any of these areas can benefit the others. Potential applications and future perspectives are also discussed in the final section of the review.

Motivation & Objective

• To bridge the gap between emergence and causality in complex systems by developing a quantitative framework for causal emergence.
• To address the challenge of identifying causal emergence from empirical data using machine learning and neural network techniques.
• To establish effective information (EI) as a core measure for quantifying causal emergence across different levels of system abstraction.
• To explore the dual role of machine learning in both detecting causal emergence (CE with ML) and enhancing causal understanding in ML systems (CE for ML).
• To provide a roadmap for future research in causal emergence, including applications in artificial intelligence, neuroscience, and systems biology.

Proposed method

• Uses effective information (EI) as a measure of causal emergence, defined as the mutual information between input and output under intervention (do-calculus).
• Employs state transition probability matrices (TPMs) to model system dynamics and compute joint and marginal probabilities under maximum entropy distributions.
• Derives EI from TPMs using the formula: $ EI = \frac{1}{N}\sum_{i,j}TPM(i,j)\log_2\left(\frac{N \cdot TPM(i,j)}{\sum_k TPM(k,j)}\right) $, enabling data-driven computation.
• Applies machine learning and deep learning models to detect causal structures at macroscopic levels from high-dimensional data.
• Integrates causal discovery algorithms with representation learning to identify effective macro-variables that maximize EI.
• Uses the do-calculus framework to distinguish between passive observation and active intervention, ensuring causal validity in EI estimation.

Experimental results

Research questions

• RQ1 How can causal emergence be quantitatively measured in complex systems using information-theoretic tools?
• RQ2 What role does effective information (EI) play in identifying macro-level causal laws that are not present at the micro-level?
• RQ3 How can machine learning techniques be leveraged to detect causal emergence from observational and experimental data?
• RQ4 In what ways does causal emergence manifest in artificial intelligence systems, such as large language models?
• RQ5 How does adaptation and evolution contribute to the emergence of causal structures in complex systems?

Key findings

• Effective information (EI) provides a robust, information-theoretic measure for quantifying causal emergence, capturing the degree to which macro-level interventions produce predictable effects.
• The method enables detection of causal emergence from data using only the system's transition probability matrix (TPM), without requiring full joint distributions.
• Causal emergence is observed in systems where macro-variables—aggregations of micro-states—exhibit higher EI than their micro-counterparts, indicating stronger causal power.
• Machine learning models, especially deep neural networks, can identify macro-variables that maximize EI, demonstrating the feasibility of data-driven causal discovery.
• Causal emergence is not merely an artifact of coarse-graining but reflects genuine causal structure that emerges at higher levels of abstraction.
• The framework reveals that downward causation and mind-body interactions may be understood as emergent causal phenomena, where macro-level states influence micro-level dynamics.