[Paper Review] Local Rule-Based Explanations of Black Box Decision Systems

Riccardo Guidotti, Anna Monreale+4 more|arXiv (Cornell University)|May 28, 2018

Explainable Artificial Intelligence (XAI)Computer Science35 references196 citations

TL;DR

LORE provides local, interpretable explanations for black-box decisions by creating a synthetic neighborhood with a genetic algorithm, training a local decision tree, and extracting a decision rule with counterfactual rules.

ABSTRACT

The recent years have witnessed the rise of accurate but obscure decision systems which hide the logic of their internal decision processes to the users. The lack of explanations for the decisions of black box systems is a key ethical issue, and a limitation to the adoption of machine learning components in socially sensitive and safety-critical contexts. %Therefore, we need explanations that reveals the reasons why a predictor takes a certain decision. In this paper we focus on the problem of black box outcome explanation, i.e., explaining the reasons of the decision taken on a specific instance. We propose LORE, an agnostic method able to provide interpretable and faithful explanations. LORE first leans a local interpretable predictor on a synthetic neighborhood generated by a genetic algorithm. Then it derives from the logic of the local interpretable predictor a meaningful explanation consisting of: a decision rule, which explains the reasons of the decision; and a set of counterfactual rules, suggesting the changes in the instance's features that lead to a different outcome. Wide experiments show that LORE outperforms existing methods and baselines both in the quality of explanations and in the accuracy in mimicking the black box.

Motivation & Objective

• Clarify why a specific decision was made for a given instance in opaque models.
• Provide meaningful, human-understandable explanations using simple logical rules.
• Offer counterfactuals that suggest minimal changes to flip the outcome.
• Maintain model-agnostic explanations that do not rely on internals of the black box.

Proposed method

• Generate a balanced synthetic neighborhood around the target instance x using a genetic algorithm with two fitness functions to produce Z_= and Z_\neq that preserve or flip the black box outcome.
• Train a local interpretable predictor c (a decision tree) on the neighborhood Z to mimic the black box behavior in the vicinity of x.
• Extract a local explanation as a pair e = <r, Φ> where r is a decision rule explaining the outcome and Φ is a set of minimal counterfactual rules for reversing it.
• Derive counterfactuals by identifying paths in the local tree that lead to alternative outcomes and selecting minimal modifications to x.
• Use a distance-based, mixed-type feature metric to guide neighborhood generation and ensure proximity to x.
• Provide explanations in terms of interpretable logic rules that are consistent with c and satisfied by x.

Experimental results

Research questions

• RQ1 How effectively can LORE explain the decision on a specific instance in a black-box predictor?
• RQ2 Does a locally learned decision tree accurately mimic the black box in the neighborhood of the instance?
• RQ3 Are the generated explanations (rules and counterfactuals) informative and actionable for users?
• RQ4 How does LORE compare to existing explanation approaches in both explanation quality and mimicry accuracy?

Key findings

• LORE outperforms existing methods and baselines in the quality of explanations and in accuracy in mimicking the black box.
• The method uses a genetic algorithm to generate a neighborhood that is dense near the decision boundary, improving local explanations.
• The local explanation consists of a rule consistent with the local predictor and a set of minimal counterfactual rules.
• Counterfactuals are derived from paths in the local decision tree and are designed to be minimal with respect to feature changes.
• The approach is agnostic to the underlying black-box model and is applicable to relational, tabular data.