[Paper Review] Causality, Responsibility and Blame in Team Plans

Natasha Alechina, Joseph Y. Halpern+1 more|arXiv (Cornell University)|May 20, 2020

AI-based Problem Solving and Planning23 references4 citations

TL;DR

This paper applies formal causality, responsibility, and blame frameworks—based on Halpern and Chockler’s definitions—to team plans, showing that causality and responsibility can be computed in polynomial time for monotone and conjunctive team plans. It provides a tractable method for identifying agents most responsible for plan failures, with applications in trust management, recovery, and resource allocation.

ABSTRACT

Many objectives can be achieved (or may be achieved more effectively) only by a group of agents executing a team plan. If a team plan fails, it is often of interest to determine what caused the failure, the degree of responsibility of each agent for the failure, and the degree of blame attached to each agent. We show how team plans can be represented in terms of structural equations, and then apply the definitions of causality introduced by Halpern [2015] and degree of responsibility and blame introduced by Chockler and Halpern [2004] to determine the agent(s) who caused the failure and what their degree of responsibility/blame is. We also prove new results on the complexity of computing causality and degree of responsibility and blame, showing that they can be determined in polynomial time for many team plans of interest.

Motivation & Objective

• To address the challenge of determining which agents are causally responsible for team plan failures, especially when failures result from cascading dependencies.
• To apply formal definitions of causality, responsibility, and blame (from Halpern and Chockler) to team plans in a way that aligns with intuitive judgments.
• To analyze the computational complexity of causality and responsibility in team plans, identifying tractable subclasses.
• To enable practical deployment of blame and responsibility reasoning in real-time or run-time systems involving multi-agent coordination.
• To extend the framework to include uncertainty and expected blame via probabilistic reasoning.

Proposed method

• Represent team plans as structural causal models using endogenous variables for task execution and preconditions.
• Model each agent’s action as a variable (e.g., in_a(t) = 0 if agent a fails to perform task t), with equations capturing dependencies.
• Use Halpern’s definition of actual causality and Chockler and Halpern’s degree of responsibility to quantify agent contributions to failure.
• Prove that for monotone causal models (common in team plans), causality can be computed in polynomial time.
• Show that for postcondition-minimal plans (where preconditions are uniquely determined), both causality and responsibility are computable in polynomial time.
• Incorporate uncertainty via expected blame, defined as the expected degree of responsibility under uncertainty about agent actions.

Experimental results

Research questions

• RQ1 How can formal causality be applied to team plans to identify which agents caused a failure?
• RQ2 What is the computational complexity of determining causality and responsibility in team plans?
• RQ3 Which structural properties of team plans (e.g., monotonicity, conjunctivity) lead to tractable computation of responsibility?
• RQ4 Can blame be meaningfully quantified in team plans under uncertainty, and how is it related to responsibility?
• RQ5 How do dependencies between tasks affect the assignment of responsibility and blame to individual agents?

Key findings

• Causality in team plans can be computed in polynomial time when the underlying causal model is monotone, which holds for most practical team plans.
• The degree of responsibility and blame for agents in team plans is NP-complete in general, but becomes polynomial-time computable for postcondition-minimal plans.
• For postcondition-minimal plans, where preconditions are uniquely determined by prior steps, both causality and responsibility are computable in polynomial time.
• The framework naturally extends to handle uncertainty, allowing the computation of expected blame as the expected degree of responsibility.
• The causal models derived from team plans are inherently monotone, meaning their structural equations contain no negations, which enables efficient computation.
• The results show that causality and responsibility reasoning in team plans is computationally feasible for many realistic plan structures, supporting real-time deployment.