[Paper Review] Statistical Aspects of SHAP: Functional ANOVA for Model Interpretation

Andrew Herren, P. Richard Hahn|arXiv (Cornell University)|Aug 21, 2022

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper establishes a formal connection between SHAP and functional ANOVA, demonstrating that SHAP values are equivalent to weighted sums of ANOVA components. It reveals that SHAP's accuracy hinges critically on the choice of baseline distribution and the number of estimated ANOVA terms, highlighting that interpretability is not free and depends on statistical assumptions embedded in the method.

ABSTRACT

SHAP is a popular method for measuring variable importance in machine learning models. In this paper, we study the algorithm used to estimate SHAP scores and outline its connection to the functional ANOVA decomposition. We use this connection to show that challenges in SHAP approximations largely relate to the choice of a feature distribution and the number of $2^p$ ANOVA terms estimated. We argue that the connection between machine learning explainability and sensitivity analysis is illuminating in this case, but the immediate practical consequences are not obvious since the two fields face a different set of constraints. Machine learning explainability concerns models which are inexpensive to evaluate but often have hundreds, if not thousands, of features. Sensitivity analysis typically deals with models from physics or engineering which may be very time consuming to run, but operate on a comparatively small space of inputs.

Motivation & Objective

• To clarify the statistical foundations of SHAP by linking it to functional ANOVA decomposition.
• To investigate how the choice of baseline distribution and the number of ANOVA terms impact SHAP approximations.
• To highlight that SHAP's interpretability is not inherent but depends on implicit statistical assumptions.
• To bridge the gap between machine learning interpretability and sensitivity analysis in computer experiments.
• To guide practitioners in making informed decisions about baseline distributions and approximation strategies in SHAP.

Proposed method

• The authors represent SHAP values as linear combinations of functional ANOVA components, which are contrasts of conditional expectations.
• They use the functional ANOVA framework to formalize the computation of SHAP values as weighted averages over all feature subsets.
• The method involves estimating $2^p$ ANOVA terms, where $p$ is the number of features, under different baseline distributions.
• The paper compares four baseline distributions: global independent, global correlated, local independent, and single baseline, using Monte Carlo sampling.
• It evaluates SHAP values across four synthetic functions with varying linearity and interaction structures.
• The analysis uses both theoretical derivations and empirical computation to assess the impact of baseline choice on estimated Shapley values.

Experimental results

Research questions

• RQ1 How does the choice of baseline distribution affect the estimation of SHAP values?
• RQ2 To what extent do the number of estimated ANOVA terms influence the accuracy of SHAP approximations?
• RQ3 In what ways does the functional ANOVA decomposition clarify the statistical underpinnings of SHAP?
• RQ4 How do correlations between features affect SHAP values when using different baseline assumptions?
• RQ5 Can insights from sensitivity analysis improve the convergence and reliability of SHAP estimates?

Key findings

• The choice of baseline distribution profoundly affects SHAP values, with correlated baselines producing significantly different results than independent or local baselines.
• Using a global correlated baseline distribution leads to substantial shifts in estimated Shapley values, especially in the presence of feature interactions.
• Local independent baselines only yield nonzero Shapley values for functions with strong nonlinearity or symmetry, limiting their general applicability.
• The single baseline approach, while intuitive, raises the critical question of where to place the reference point, with no universally optimal choice.
• SHAP values computed under different baselines are not interchangeable, even for the same model and input, indicating that the estimand is sensitive to distributional assumptions.
• The functional ANOVA lens reveals that SHAP is not a single method but a family of estimands, each dependent on the underlying distribution used in conditional expectation estimation.