[Paper Review] Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation

Marco Ancona, Cengiz Öztireli+1 more|arXiv (Cornell University)|Mar 26, 2019

Explainable Artificial Intelligence (XAI)Computer Science42 references48 citations

TL;DR

The paper introduces Deep Approximate Shapley Propagation (DASP), a polynomial-time method to approximate Shapley values for DNNs, and shows it yields more accurate attributions than state-of-the-art methods across several datasets.

ABSTRACT

The problem of explaining the behavior of deep neural networks has recently gained a lot of attention. While several attribution methods have been proposed, most come without strong theoretical foundations, which raises questions about their reliability. On the other hand, the literature on cooperative game theory suggests Shapley values as a unique way of assigning relevance scores such that certain desirable properties are satisfied. Unfortunately, the exact evaluation of Shapley values is prohibitively expensive, exponential in the number of input features. In this work, by leveraging recent results on uncertainty propagation, we propose a novel, polynomial-time approximation of Shapley values in deep neural networks. We show that our method produces significantly better approximations of Shapley values than existing state-of-the-art attribution methods.

Motivation & Objective

• Motivate the use of Shapley values for reliable local explanations in DNNs through an axiomatic comparison.
• Develop a polynomial-time approximation algorithm (DASP) for Shapley values in deep networks.
• Empirically evaluate DASP against state-of-the-art attribution methods on diverse datasets and architectures.

Proposed method

• Formulate Shapley values for neural network inputs with a zero baseline.
• Approximate expected coalitions by evaluating contributions from coalitions of varying sizes k using a distribution of coalitions.
• Propagate input uncertainty through the network using Lightweight Probabilistic Networks (LPN) and moment matching to obtain output statistics.
• Represent coalitions as Gaussian-distributed inputs and compute mean/variance propagation through network layers (linear, ReLU, max pooling).
• Compute approximate Shapley values by comparing network outputs with and without each feature across sampled coalition sizes (K) and averaging contributions.
• Achieve polynomial-time complexity with O(N^2) evaluations when testing all coalition sizes, or O(KN) evaluations for a reduced set of sizes.

Experimental results

Research questions

• RQ1 Can Shapley values provide a principled axiomatically grounded attribution for DNNs across architectures?
• RQ2 Is it feasible to approximate Shapley values for DNN inputs in polynomial time without excessive loss of accuracy?
• RQ3 How does DASP compare to existing backpropagation- and perturbation-based attribution methods in terms of accuracy and efficiency across datasets and architectures?

Key findings

• DASP yields more accurate Shapley-valued attributions than competing methods on the tested tasks.
• The method scales to N input features with polynomial-time evaluations (O(N^2) when using all coalition sizes).
• Input coalitions are modeled as Gaussian distributions, enabling propagation of mean and variance through the network via LPN-based uncertainty propagation.
• Empirical results on Parkinsons disability assessment, DNA sequence classification, and MNIST show favorable performance versus bias-prone or sampling-based approaches.