[Paper Review] On The Reasons Behind Decisions

Adnan Darwiche, Auguste Hirth|arXiv (Cornell University)|Feb 21, 2020

Machine Learning and Data Classification63 citations

TL;DR

The paper develops a theory and tractable algorithms to explain decisions of Boolean classifiers using sufficient, complete, and necessary reasons, and to analyze bias and counterfactuals.

ABSTRACT

Recent work has shown that some common machine learning classifiers can be compiled into Boolean circuits that have the same input-output behavior. We present a theory for unveiling the reasons behind the decisions made by Boolean classifiers and study some of its theoretical and practical implications. We define notions such as sufficient, necessary and complete reasons behind decisions, in addition to classifier and decision bias. We show how these notions can be used to evaluate counterfactual statements such as "a decision will stick even if ... because ... ." We present efficient algorithms for computing these notions, which are based on new advances on tractable Boolean circuits, and illustrate them using a case study.

Motivation & Objective

• Define sufficient, complete, and necessary reasons behind classifier decisions.
• Introduce bias and counterfactual concepts for decisions.
• Provide algorithms to compute these notions using tractable Boolean circuits.
• Show how complete reasons enable robust explanations and bias detection.

Proposed method

• Represent classifiers as propositional formulas with input features as variables.
• Use prime implicants to define sufficient reasons and derive complete reasons as their disjunction.
• Introduce complete reasons and prove their relation to sufficient/necessary reasons.
• Compute reasons via Decision-DNNF (OBDD-compatible) circuits combined with consensus and filtering operations.
• Propose linear-time consensus construction and instance-based filtering to obtain tractable explanations.
• Illustrate methods with a case study on college admission classifiers.

Experimental results

Research questions

• RQ1 How can we formally define and compute informative reasons behind a classifier's decision?
• RQ2 How do sufficient, complete, and necessary reasons relate and support explanations?
• RQ3 Can complete reasons be used to detect decision and classifier bias?
• RQ4 How can tractable circuit representations enable efficient reasoning about decisions and counterfactuals?

Key findings

• Every decision has at least one sufficient reason represented by a prime implicant.
• The complete reason for a decision is the disjunction of all sufficient reasons and uniquely captures all triggering properties.
• Sufficient reasons can be used to derive counterfactuals and necessary reasons when applicable.
• Decision bias can be detected when all sufficient reasons contain protected features.
• Classifier bias can be inferred from the complete reason even for unbiased individual decisions.
• The approach enables linear-time computation of complete reasons via consensus and filtering on Decision-DNNF circuits.