[Paper Review] From Formal Boosted Tree Explanations to Interpretable Rule Sets

Yu, Jinqiang, Alexey Ignatiev+1 more|arXiv (Cornell University)|Jul 4, 2019

Explainable Artificial Intelligence (XAI)47 references33 citations

TL;DR

This paper proposes a logic-based framework, XPlainer, to validate, repair, and refine heuristic explanations for XGBoosted boosted trees by computing globally correct, minimal explanations using constraint-based abduction. It reveals that Anchor and LIME explanations are often globally incorrect—Anchor is optimistic in over 99% of cases on two datasets—demonstrating the critical need for global validation of heuristic explanations.

ABSTRACT

Recent years have witnessed a fast-growing interest in computing explanations for Machine Learning (ML) models predictions. For non-interpretable ML models, the most commonly used approaches for computing explanations are heuristic in nature. In contrast, recent work proposed rigorous approaches for computing explanations, which hold for a given ML model and prediction over the entire instance space. This paper extends earlier work to the case of boosted trees and assesses the quality of explanations obtained with state-of-the-art heuristic approaches. On most of the datasets considered, and for the vast majority of instances, the explanations obtained with heuristic approaches are shown to be inadequate when the entire instance space is (implicitly) considered.

Motivation & Objective

• To address the lack of global guarantees in heuristic ML explanations, which are locally computed and may fail over the full instance space.
• To develop a scalable constraint-based encoding for boosted trees to compute globally correct, minimal explanations using abductive reasoning.
• To design and implement algorithms that validate, repair (when optimistic), and refine (when pessimistic) heuristic explanations.
• To empirically assess the quality of state-of-the-art heuristic explanations (LIME and Anchor) using globally correct explanations as a gold standard.

Proposed method

• Proposes a novel constraint-based encoding of XGBoosted boosted trees to enable efficient computation of global explanations via abductive reasoning.
• Computes subset-minimal and cardinality-minimal explanations using SAT-based reasoning to ensure global correctness and minimality.
• Integrates validation, repair, and refinement algorithms into the XPlainer tool, using logical consistency checks and minimization over the instance space.
• Employs on-demand abduction to compute explanations only when needed, improving scalability over full compilation approaches.
• Uses the XGBoost model and trained instance space to evaluate heuristic explanations against the ground truth of global explanations.
• Employs a logic-based framework to detect redundant literals (pessimism) and missing necessary literals (optimism) in heuristic explanations.

Experimental results

Research questions

• RQ1 How accurate are heuristic explanations (e.g., LIME and Anchor) when evaluated against globally correct, minimal explanations for XGBoosted models?
• RQ2 To what extent are heuristic explanations optimistic (i.e., fail to apply on a large portion of the instance space) or pessimistic (i.e., contain redundant features)?
• RQ3 Can a logic-based framework like XPlainer effectively validate, repair, and refine heuristic explanations for boosted trees?
• RQ4 What is the computational cost of computing globally correct explanations compared to heuristic methods like LIME and Anchor?

Key findings

• On two datasets, Anchor is optimistic in over 99% of instances, meaning its explanations fail to apply globally for more than 99% of the input space.
• On two other datasets, Anchor is optimistic in over 80% of instances, indicating widespread global inaccuracy of its explanations.
• LIME is often more pessimistic than Anchor, but neither method dominates in terms of optimism, showing both have significant global reliability issues.
• Global explanations computed by XPlainer are either subset-minimal or cardinality-minimal, ensuring they are neither optimistic nor pessimistic by construction.
• The average size of XPlainer’s subset-minimal explanations is 15–50% of the total features, while cardinality-minimal explanations reduce this to 15–36%, improving interpretability.
• XPlainer’s runtime is comparable to LIME and Anchor, with validation being negligible and repair/refinement being faster in subset-minimal mode than in cardinality-minimal mode.