[Paper Review] GRAPHSHAP: Explaining Identity-Aware Graph Classifiers Through the Language of Motifs

Alan Perotti, Paolo Bajardi+2 more|arXiv (Cornell University)|Feb 17, 2022

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

GRAPHSHAP introduces a model-agnostic, Shapley-based method to explain identity-aware graph classifiers using network motifs as an interpretable explanation language. It efficiently approximates motif importance scores via a progressive kernel, enabling scalable, actionable explanations—demonstrated on brain networks where few motifs (e.g., M5) strongly explain autism spectrum disorder classifications.

ABSTRACT

Most methods for explaining black-box classifiers (e.g. on tabular data, images, or time series) rely on measuring the impact that removing/perturbing features has on the model output. This forces the explanation language to match the classifier's feature space. However, when dealing with graph data, in which the basic features correspond to the edges describing the graph structure, this matching between features space and explanation language might not be appropriate. Decoupling the feature space (edges) from a desired high-level explanation language (such as motifs) is thus a major challenge towards developing actionable explanations for graph classification tasks. In this paper we introduce GRAPHSHAP, a Shapley-based approach able to provide motif-based explanations for identity-aware graph classifiers, assuming no knowledge whatsoever about the model or its training data: the only requirement is that the classifier can be queried as a black-box at will. For the sake of computational efficiency we explore a progressive approximation strategy and show how a simple kernel can efficiently approximate explanation scores, thus allowing GRAPHSHAP to scale on scenarios with a large explanation space (i.e. large number of motifs). We showcase GRAPHSHAP on a real-world brain-network dataset consisting of patients affected by Autism Spectrum Disorder and a control group. Our experiments highlight how the classification provided by a black-box model can be effectively explained by few connectomics patterns.

Motivation & Objective

• Address the lack of actionable, high-level explanations for graph classification models, especially when feature space (edges) does not align with human-understandable structures.
• Decouple the model's internal feature space (edges) from a desired explanation language (motifs), enabling interpretable insights without model access.
• Develop a scalable, black-box explanation method for identity-aware graph classifiers—where node identities are consistent across graphs—using motif-based attribution.
• Enable efficient computation of Shapley values for large motif spaces through a progressive approximation kernel, making large-scale motif attribution feasible.
• Validate the method on real-world brain network data to uncover connectomic patterns linked to autism spectrum disorder.

Proposed method

• Adapt Shapley values from cooperative game theory to attribute importance to motifs rather than edges, treating each motif as a 'player' in a game where the 'gain' is the model's prediction change.
• Define a coalitional game where the value function measures the difference in model output when a motif is added or removed from the graph, using toggle masking to simulate presence/absence.
• Introduce a progressive approximation strategy that computes Shapley values up to depth-1 (considering only single-motif coalitions), drastically reducing computational cost.
• Design a kernel-based approximation method that efficiently estimates Shapley scores by sampling and averaging over motif subsets, enabling scalability to large motif spaces.
• Use toggle masking to simulate motif presence/absence in a graph: for each motif, the model is queried with the original graph and a version where the motif is toggled (added/removed).
• Aggregate local explanations across the dataset to derive global motif importance rankings, enabling hypothesis generation and validation.

Experimental results

Research questions

• RQ1 Can motif-based explanations provide more actionable and interpretable insights than edge-level explanations for graph classifiers?
• RQ2 How can Shapley values be adapted to attribute importance to higher-order network structures (motifs) rather than individual edges?
• RQ3 What approximation strategy enables scalable computation of motif Shapley values in large motif spaces without sacrificing accuracy?
• RQ4 To what extent can motif-based explanations uncover biologically meaningful patterns in real-world brain network data?
• RQ5 How do local and global motif importance scores correlate, and can global rankings guide the discovery of key connectomic patterns?

Key findings

• The depth-1 approximation of Shapley values achieved a median Pearson correlation of 0.997 with exact values on the brain network dataset, confirming high fidelity at a fraction of the cost.
• Only a small subset of motifs (e.g., M5) showed consistently high global importance, indicating that a few connectomic patterns dominate model predictions.
• M5 was the most globally important motif, with strong local explanation scores for both ASD patients P1 and P2, suggesting a central role in classification.
• M2 had high local importance for one patient (P2) but negligible global importance, highlighting the value of local explanations in uncovering nuanced decision drivers.
• The method enabled efficient attribution across 100 motifs with linear computational cost, saving approximately 10^26 operations compared to exact computation.
• The approach successfully generated actionable, neuroscience-relevant hypotheses—e.g., the absence of M5 strongly contributed to ASD classification—validating its utility in real-world applications.