[Paper Review] A Unified Approach to Interpreting Model Predictions

Scott Lundberg, Su‐In Lee|arXiv (Cornell University)|May 22, 2017

Explainable Artificial Intelligence (XAI)Computer Science8 references7,621 citations

TL;DR

The paper introduces SHAP, a unified additive feature attribution framework that uniquely satisfies local accuracy, missingness, and consistency, unifying six prior methods and enabling model-agnostic and model-specific approximations. It provides theoretical guarantees and practical estimation methods (Kernel SHAP, Deep SHAP) with experiments showing better alignment with human intuition.

ABSTRACT

Understanding why a model makes a certain prediction can be as crucial as the prediction's accuracy in many applications. However, the highest accuracy for large modern datasets is often achieved by complex models that even experts struggle to interpret, such as ensemble or deep learning models, creating a tension between accuracy and interpretability. In response, various methods have recently been proposed to help users interpret the predictions of complex models, but it is often unclear how these methods are related and when one method is preferable over another. To address this problem, we present a unified framework for interpreting predictions, SHAP (SHapley Additive exPlanations). SHAP assigns each feature an importance value for a particular prediction. Its novel components include: (1) the identification of a new class of additive feature importance measures, and (2) theoretical results showing there is a unique solution in this class with a set of desirable properties. The new class unifies six existing methods, notable because several recent methods in the class lack the proposed desirable properties. Based on insights from this unification, we present new methods that show improved computational performance and/or better consistency with human intuition than previous approaches.

Motivation & Objective

• Motivate the need for interpretable explanations alongside high model accuracy in complex models.
• Introduce a unifying framework for additive feature attributions that encompasses existing methods.
• Establish a unique solution within this class that satisfies desirable properties and connect to game-theoretic Shapley values.
• Develop practical SHAP value estimation methods and demonstrate improvements over prior approaches.

Proposed method

• Define additive feature attribution explanations as a linear model of binary simplified inputs (phi_i coefficients).
• Show six existing methods fit the same explanation model form under this definition (LIME, DeepLIFT, Layer-Wise Relevance Propagation, Shapley-based methods).
• Prove, using cooperative game theory, that a unique solution exists satisfying local accuracy, missingness, and consistency (Shapley values).
• Define SHAP values as the Shapley values of the conditional expectation function of the original model.
• Propose model-agnostic (Kernel SHAP, Shapley sampling) and model-specific (Linear SHAP, Low-Order SHAP, Max SHAP, Deep SHAP) approximations.
• Provide algorithms to compute SHAP values and discuss their connections to existing methods.

Experimental results

Research questions

• RQ1 Can additive feature attribution methods be unified under a single theoretical framework?
• RQ2 What properties should explanations satisfy to be reliable and intuitively aligned with human judgments?
• RQ3 How can SHAP values be estimated efficiently for both model-agnostic and model-specific scenarios?
• RQ4 Do SHAP-derived explanations better align with human intuition than prior methods across tasks like image and text/DL models?
• RQ5 How can existing methods be improved or extended within the SHAP framework?

Key findings

• There is a unique additive explanation model satisfying local accuracy, missingness, and consistency for a given input mapping.
• SHAP values unify six prior methods and provide a principled basis for feature attributions via Shapley values.
• Kernel SHAP offers a model-agnostic, regression-based estimation with improved sample efficiency over previous Shapley-valued methods.
• Model-specific variants (Linear SHAP, Deep SHAP, Max SHAP) enable faster or more accurate attributions for particular architectures.
• Human-subject studies show SHAP explanations align more closely with human intuition than LIME or DeepLIFT in tested scenarios.
• Experiments on MNIST demonstrate SHAP and its derivatives provide explanations that better reflect class differences and input importance.