[Paper Review] The many Shapley values for model explanation

Mukund Sundararajan, Amir Najmi|arXiv (Cornell University)|Aug 22, 2019

Explainable Artificial Intelligence (XAI)Computer Science19 references36 citations

TL;DR

This paper analyzes multiple Shapley-based attribution methods for model explanations, shows problematic behavior of CES, and introduces Baseline Shapley (BShap) with a uniqueness guarantee, comparing it to Integrated Gradients (IG).

ABSTRACT

The Shapley value has become a popular method to attribute the prediction of a machine-learning model on an input to its base features. The use of the Shapley value is justified by citing [16] showing that it is the \emph{unique} method that satisfies certain good properties (\emph{axioms}). There are, however, a multiplicity of ways in which the Shapley value is operationalized in the attribution problem. These differ in how they reference the model, the training data, and the explanation context. These give very different results, rendering the uniqueness result meaningless. Furthermore, we find that previously proposed approaches can produce counterintuitive attributions in theory and in practice---for instance, they can assign non-zero attributions to features that are not even referenced by the model. In this paper, we use the axiomatic approach to study the differences between some of the many operationalizations of the Shapley value for attribution, and propose a technique called Baseline Shapley (BShap) that is backed by a proper uniqueness result. We also contrast BShap with Integrated Gradients, another extension of Shapley value to the continuous setting.

Motivation & Objective

• Motivate attribution of a model's prediction to input features using Shapley values and address non-uniqueness across operationalizations.
• Analyze how different Shapley extensions (CES, BShap, RBShap, IG) depend on model, data, and baseline choices.
• Provide axiomatic foundations and uniqueness results for Baseline Shapley (BShap) and relate them to cost-sharing theory.
• Compare Baseline Shapley with Integrated Gradients and other Shapley-based approaches.
• Demonstrate empirical implications of these methods via a diabetes prediction case study.

Proposed method

• Formalize the attribution problem with a real-valued model f and feature set N.
• Define three Shapley-based extensions: Conditional Expectations Shapley (CES), Baseline Shapley (BShap), and Random Baseline Shapley (RBShap); and contrast with Integrated Gradients (IG).
• Develop axioms (Dummy, Efficiency, Linearity, Symmetry, Affine Scale Invariance, Demand Monotonicity, Proportionality) to compare methods.
• Show how CES depends on the reference distribution D and can yield counterintuitive attributions; introduce a Downward Closure property and an algorithm to compute CES(hat{D}).
• Provide theoretical reductions that link model explanation to cost-sharing, establishing uniqueness results for BShap and IG under corresponding axiom sets.
• Relate BShap to CES via distribution choices and discuss RBShap as an averaging variant.

Experimental results

Research questions

• RQ1 How do different Shapley-based attribution methods (CES, BShap, RBShap, IG) differ in assumptions, computations, and outputs?
• RQ2 Under what axioms are Baseline Shapley (BShap) and Integrated Gradients (IG) unique solutions for model explanation?
• RQ3 What are the pitfalls of CES, especially its dependence on feature distributions and sparsity, and how do these affect attribution quality?
• RQ4 How can model explanation be reduced to cost-sharing, and what does this imply about the relationships between BShap and IG?
• RQ5 What empirical evidence demonstrates the practical implications of these attribution methods on real data (e.g., diabetes progression task)?

Key findings

• CES attributions depend crucially on the chosen feature distribution D and can be highly sensitive to sparsity.
• CES can assign non-zero attributions to dummy features and can violate linearity and other intuitions in simple examples.
• Baseline Shapley (BShap) satisfies key axioms (Linearity, Dummy, ASI, DM, Symmetry) and is unique under these axioms for attribution problems.
• Integrated Gradients (IG) is the unique method satisfying Linearity, Dummy, ASI, Proportionality, and Symmetry, providing a distinct principled alternative to BShap.
• BShap aligns with CES under certain distributions, but unlike CES, it does not rely on a distribution and uses an explicit baseline to reflect explanation context.
• In a diabetes prediction case study, the authors illustrate how attribution methods can yield nuanced and potentially counterintuitive results; the dataset showed the model explaining 35% of variance (R^2) with BMI, BP, and a serum measurement as strong contributors.
• RBShap and the relationship between CES and RBShap are discussed, showing that averaging baselines can recover CES under independent feature distributions.