[Paper Review] From Verification to Causality-based Explications

Christel Baier, Clemens Dubslaff+5 more|arXiv (Cornell University)|May 20, 2021

Formal Methods in Verification88 references4 citations

TL;DR

This paper surveys causality-based reasoning in formal verification, proposing a framework that extends traditional model checking by integrating Halpern-Pearl causality, Shapley value-based responsibility, and probabilistic causation to explain *why* systems behave as they do. It introduces formal definitions of probabilistic causes (p-causes) and establishes relationships between different causality notions, enabling deeper understanding of system behavior in both non-probabilistic and stochastic models.

ABSTRACT

In view of the growing complexity of modern software architectures, formal models are increasingly used to understand why a system works the way it does, opposed to simply verifying that it behaves as intended. This paper surveys approaches to formally explicate the observable behavior of reactive systems. We describe how Halpern and Pearl's notion of actual causation inspired verification-oriented studies of cause-effect relationships in the evolution of a system. A second focus lies on applications of the Shapley value to responsibility ascriptions, aimed to measure the influence of an event on an observable effect. Finally, formal approaches to probabilistic causation are collected and connected, and their relevance to the understanding of probabilistic systems is discussed.

Motivation & Objective

• To address the growing need for causal explanations in complex software systems beyond mere correctness verification.
• To formalize and unify qualitative and quantitative causality concepts—such as actual causation and responsibility—for use in formal verification.
• To extend causality reasoning from deterministic models to probabilistic systems, particularly Markov chains, using probability-raising and counterfactual principles.
• To identify open challenges in causality-based verification, including nondeterministic models (Markov decision processes) and partially observable systems (hidden Markov models).

Proposed method

• Adopts Halpern and Pearl’s structural equation models and counterfactual logic to define actual causation in reactive systems.
• Introduces p-causes as prefixes in Markov chains that raise the probability of a property to at least p, with conditions ensuring minimality and counterfactual dependence.
• Applies the Shapley value to quantify responsibility, measuring an agent’s contribution to an outcome in a coalition-theoretic framework.
• Defines global, local, and p-causes with axiomatic conditions (PAC1, PAC2, PAC3) to formalize probabilistic causality with counterfactual grounding.
• Establishes logical implications between causality notions: p-cause ⇒ local PR-cause ⇒ global PR-cause, especially for singleton causes.
• Uses measurable path sets in ω-regular languages to ensure mathematical rigor in probabilistic model checking contexts.

Experimental results

Research questions

• RQ1 How can formal causality concepts be used to explain *why* a system behaves in a certain way, beyond verifying correctness?
• RQ2 What are the formal criteria for identifying a probabilistic cause in a Markov chain, and how do they relate to counterfactual reasoning?
• RQ3 How can the Shapley value be applied to measure responsibility of individual agents in system behavior?
• RQ4 What are the key differences and relationships between global, local, and p-causes in probabilistic models?
• RQ5 What are the open challenges in extending causality reasoning to nondeterministic and partially observable stochastic systems?

Key findings

• For any ω-regular property in a DTMC, a p-cause exists for every p ∈ (0,1], with a 1-cause always existing due to the existence of prefixes that make the property almost surely true.
• p-causes with p > Pr(ℒ) imply local PR-causes, and local PR-causes imply global PR-causes, establishing a hierarchy of causality notions.
• When the cause is a singleton, local and global PR-causes are equivalent, simplifying responsibility assessment in symmetric cases.
• The PAC3 condition ensures counterfactual dependence by requiring that the effect occurs with probability 0 if the cause is absent, embedding counterfactual logic into probabilistic causality.
• The framework supports practical applications in runtime monitoring and probabilistic debugging by identifying high-impact system events.
• The paper identifies significant open problems, including extending causality to Markov decision processes and hidden Markov models, where states or events are not fully observable.