[Paper Review] Faith-Shap: The Faithful Shapley Interaction Index

Che-Ping Tsai, Chih‐Kuan Yeh+1 more|arXiv (Cornell University)|Mar 2, 2022

Bayesian Modeling and Causal Inference20 citations

TL;DR

This paper defines a unique Faithful Shapley Interaction index (Faith-Shap) that extends Shapley values to feature interactions via faithful linear approximation, under natural interaction axioms, and provides efficient computation and properties.

ABSTRACT

Shapley values, which were originally designed to assign attributions to individual players in coalition games, have become a commonly used approach in explainable machine learning to provide attributions to input features for black-box machine learning models. A key attraction of Shapley values is that they uniquely satisfy a very natural set of axiomatic properties. However, extending the Shapley value to assigning attributions to interactions rather than individual players, an interaction index, is non-trivial: as the natural set of axioms for the original Shapley values, extended to the context of interactions, no longer specify a unique interaction index. Many proposals thus introduce additional less ''natural'' axioms, while sacrificing the key axiom of efficiency, in order to obtain unique interaction indices. In this work, rather than introduce additional conflicting axioms, we adopt the viewpoint of Shapley values as coefficients of the most faithful linear approximation to the pseudo-Boolean coalition game value function. By extending linear to $\ell$-order polynomial approximations, we can then define the general family of faithful interaction indices. We show that by additionally requiring the faithful interaction indices to satisfy interaction-extensions of the standard individual Shapley axioms (dummy, symmetry, linearity, and efficiency), we obtain a unique Faithful Shapley Interaction index, which we denote Faith-Shap, as a natural generalization of the Shapley value to interactions. We then provide some illustrative contrasts of Faith-Shap with previously proposed interaction indices, and further investigate some of its interesting algebraic properties. We further show the computational efficiency of computing Faith-Shap, together with some additional qualitative insights, via some illustrative experiments.

Motivation & Objective

• Motivate the need for attributions to feature interactions beyond individual features in explainable AI.
• Introduce the Faith-Interaction framework by extending Shapley-style faithfulness to higher-order interactions.
• Derive unique Faithful Shapley (Faith-Shap) and Faithful Banzhaf (Faith-Banzhaf) indices under interaction axioms.
• Compare Faith-Shap with existing interaction indices and explore algebraic properties and efficiency of computation.

Proposed method

• Model explanations as a weighted regression to approximate v(S) by sums of interaction terms up to order ell.
• Use Möbius transform a(v, S) to represent contributions and derive closed-form expressions for interaction indices.
• Define Faith-Interaction indices as solutions to a weighted least squares objective with a proper weighting function μ(S).
• Prove that Faith-Interaction indices satisfy interaction linearity, symmetry, and dummy axioms under suitable μ(S) (and finite/∞ constraints).
• Specialize to Faith-Shap by enforcing the interaction efficiency axiom, yielding a specific μ(S) and a closed-form formula (Eq. 16).
• Show that Faith-Shap reduces to standard Shapley values when ell = 1, and discuss relationships to Faith-Banzhaf (ell-based) and other indices.

Experimental results

Research questions

• RQ1 How can we uniquely extend Shapley-like attributions from individual features to feature interactions while preserving natural axioms?
• RQ2 Can a faithfulness-based alternative yield a unique interaction index (Faith-Shap) that satisfies linearity, symmetry, dummy, and efficiency?
• RQ3 How do Faithful Shapley and related indices compare to existing interaction indices in theory and computation?
• RQ4 What are the algebraic properties and practical computational benefits of Faith-Shap in estimating interactions up to a chosen order?

Key findings

• A unique Faithful Shapley Interaction index (Faith-Shap) is obtained by combining interaction linearity, symmetry, dummy, and efficiency with a faithful-by-construction framework.
• The Faith-Interaction class yields closed-form solutions when weighting functions are finite, expressed via Möbius transforms and a feature-incidence matrix (p(S)).
• Faith-Shap generalizes Shapley values to interactions and reduces to standard Shapley values when the maximum interaction order is one.
• A complementary Faith-Banzhaf index arises under generalized 2-efficiency, with its own closed-form and relationships to the Möbius transform.
• The framework enables computationally efficient estimation through a weighted linear regression formulation, and provides algebraic characterizations linked to cardinal indices and multilinear extensions.