[Paper Review] Consistent Individualized Feature Attribution for Tree Ensembles

Scott Lundberg, Gabriel Erion+1 more|arXiv (Cornell University)|Feb 12, 2018

Forest ecology and management22 references552 citations

TL;DR

The paper identifies inconsistencies in common tree-model feature attributions and introduces SHAP values with a fast Tree SHAP algorithm and SHAP interaction values, enabling exact, consistent, individualized explanations for tree ensembles.

ABSTRACT

Interpreting predictions from tree ensemble methods such as gradient boosting machines and random forests is important, yet feature attribution for trees is often heuristic and not individualized for each prediction. Here we show that popular feature attribution methods are inconsistent, meaning they can lower a feature's assigned importance when the true impact of that feature actually increases. This is a fundamental problem that casts doubt on any comparison between features. To address it we turn to recent applications of game theory and develop fast exact tree solutions for SHAP (SHapley Additive exPlanation) values, which are the unique consistent and locally accurate attribution values. We then extend SHAP values to interaction effects and define SHAP interaction values. We propose a rich visualization of individualized feature attributions that improves over classic attribution summaries and partial dependence plots, and a unique "supervised" clustering (clustering based on feature attributions). We demonstrate better agreement with human intuition through a user study, exponential improvements in run time, improved clustering performance, and better identification of influential features. An implementation of our algorithm has also been merged into XGBoost and LightGBM, see http://github.com/slundberg/shap for details.

Motivation & Objective

• Motivate and formalize the need for consistent individualized feature attribution for tree ensembles.
• Propose SHAP values as the unique consistent and locally accurate attribution method within additive feature attribution.
• Develop fast, exact Tree SHAP algorithms to compute SHAP values for large tree ensembles.
• Extend SHAP to SHAP interaction values to capture pairwise feature interactions.
• Demonstrate practical benefits via visualizations, clustering, and real-data applications.

Proposed method

• Define f_x(S)=E[f(x) | x_S] and derive SHAP values as the unique consistent, locally accurate attributions.
• Develop Tree SHAP to compute SHAP values in O(TLD^2) time, a polynomial-time alternative to the naive O(TL2^M) approach.
• Extend SHAP to SHAP interaction values using the Shapley interaction index to quantify pairwise feature interactions.
• Introduce visualization tools: SHAP dependence plots and SHAP summary plots, plus supervised clustering based on SHAP attributions.
• Provide an implementation integrated with XGBoost and LightGBM for practical use.

Experimental results

Research questions

• RQ1 Can SHAP values provide the only consistent and locally accurate individualized feature attributions for tree ensembles?
• RQ2 How can SHAP values be computed efficiently for large tree ensembles?
• RQ3 What is the role of SHAP interaction values in revealing feature interactions within tree models?
• RQ4 Do SHAP-based visualizations and supervised clustering improve interpretation and actionable insights compared to existing methods?

Key findings

• SHAP values are the only consistent and locally accurate individualized attributions for tree ensembles under missingness and conditional dependence.
• Tree SHAP computes exact SHAP values in O(TLD^2) time, enabling scalable explanations for large models.
• SHAP interaction values offer a principled, symmetric measure of feature interactions within predictions.
• SHAP-based visualizations (summary and dependence plots) and supervised clustering improve alignment with human intuition and clustering performance.
• Empirical demonstrations show faster runtimes, better identification of influential features, and clearer interaction insights compared to prior methods.