[Paper Review] On the Tractability of SHAP Explanations

Guy Van den Broeck, Anton Lykov+2 more|arXiv (Cornell University)|Sep 18, 2020

Explainable Artificial Intelligence (XAI)27 references4 citations

TL;DR

This paper establishes the computational complexity of SHAP explanations across three key settings: fully-factorized distributions, naive Bayes distributions, and empirical distributions. It proves that computing SHAP values is #P-hard for logistic regression and neural networks under fully-factorized distributions, intractable for naive Bayes models, and #P-hard even on empirical data, showing that exact computation is generally infeasible for common models despite widespread use in practice.

ABSTRACT

SHAP explanations are a popular feature-attribution mechanism for explainable AI. They use game-theoretic notions to measure the influence of individual features on the prediction of a machine learning model. Despite a lot of recent interest from both academia and industry, it is not known whether SHAP explanations of common machine learning models can be computed efficiently. In this paper, we establish the complexity of computing the SHAP explanation in three important settings. First, we consider fully-factorized data distributions, and show that the complexity of computing the SHAP explanation is the same as the complexity of computing the expected value of the model. This fully-factorized setting is often used to simplify the SHAP computation, yet our results show that the computation can be intractable for commonly used models such as logistic regression. Going beyond fully-factorized distributions, we show that computing SHAP explanations is already intractable for a very simple setting: computing SHAP explanations of trivial classifiers over naive Bayes distributions. Finally, we show that even computing SHAP over the empirical distribution is #P-hard.

Motivation & Objective

• To determine the computational complexity of SHAP explanations across different data distributions and model classes.
• To resolve the open question of whether SHAP values can be computed efficiently for common machine learning models.
• To demonstrate that even simplified settings like fully-factorized or empirical distributions lead to intractable computation for standard models.
• To clarify the theoretical limits of SHAP explanation in practice, especially given its widespread adoption.

Proposed method

• Analyzes SHAP computation complexity under three data distribution types: fully-factorized, naive Bayes, and empirical distributions.
• Reduces the problem of computing expected model outputs to SHAP value computation, showing equivalence in complexity for fully-factorized settings.
• Uses a reduction from #P-complete problems, such as computing the expected value of PP2CNF formulas, to prove #P-hardness.
• Constructs a polynomial-time oracle-based transformation to compute SHAP values via a modified function and feature augmentation.
• Employs matrix determinant analysis and recurrence relations to prove non-singularity of coefficient matrices in the reduction.
• Applies a transformation to the original formula to derive the expected value using polynomial interpolation and coefficient extraction.

Experimental results

Research questions

• RQ1 Is computing SHAP explanations tractable for commonly used models like logistic regression under fully-factorized data distributions?
• RQ2 Does the assumption of feature independence in data distributions enable efficient SHAP computation?
• RQ3 What is the complexity of SHAP explanations over naive Bayes distributions, which relax the independence assumption?
• RQ4 Is SHAP computation tractable when using the empirical distribution derived from training data?
• RQ5 Can the polynomial-time algorithm for decision trees proposed by Lundberg et al. (2020) be correct for exact SHAP computation?

Key findings

• Computing SHAP explanations for logistic regression and neural networks with sigmoid activations is #P-hard under fully-factorized data distributions.
• Even for trivial classifiers over naive Bayes distributions, SHAP computation is intractable, implying hardness for linear and logistic regression models.
• SHAP computation over the empirical distribution is #P-hard, ruling out efficient exact computation in practice.
• The complexity of SHAP explanations is equivalent to the complexity of computing the expected model output under fully-factorized distributions.
• The polynomial-time algorithm for decision trees by Lundberg et al. (2020) cannot be correct for exact SHAP computation, as shown by the #P-hardness result.
• A reduction from PP2CNF model counting to SHAP computation proves that exact SHAP values are computationally intractable in general.