[Paper Review] Explainability Techniques for Graph Convolutional Networks

Federico Baldassarre, Hossein Azizpour|arXiv (Cornell University)|May 31, 2019

Machine Learning in Materials Science47 references59 citations

TL;DR

This paper analyzes gradient-based and decomposition-based explainability techniques for Graph Networks, comparing them on a toy infection task and a chemistry solubility task, and provides a PyTorch implementation for GN explanations.

ABSTRACT

Graph Networks are used to make decisions in potentially complex scenarios but it is usually not obvious how or why they made them. In this work, we study the explainability of Graph Network decisions using two main classes of techniques, gradient-based and decomposition-based, on a toy dataset and a chemistry task. Our study sets the ground for future development as well as application to real-world problems.

Motivation & Objective

• Motivate the need for interpretable Graph Networks in chemistry, physics, and related domains.
• Evaluate the applicability of explanation techniques (gradient-based and decomposition-based) to graph-structured models.
• Contrast explanations from different methods and highlight challenges and future directions for GN explainability.

Proposed method

• Adopt the Graph Network (GN) formalism with message-passing updates using phi for edges/nodes and rho for aggregations as in Battaglia et al. 2018.
• Apply three explanation techniques—Sensitivity Analysis (SA), Guided Backpropagation (GBP), and Layer-wise Relevance Propagation (LRP)—to GNs.
• Use an epsilon-stabilized LRP rule to decompose outputs into input relevance, with adaptation for regression tasks.
• Implement explainability methods on top of PyTorch autograd for flexible graph inputs.
• Conduct experiments on two tasks: an Infection toy dataset to compare explanations against infection dynamics and a Solubility dataset to attribute solubility-relevant features.
• Provide discussion on how pooling, edge importance, and heterogeneous features affect explanations.

Experimental results

Research questions

• RQ1 How do gradient-based explanation methods compare to decomposition-based methods when explaining GN predictions?
• RQ2 What are the challenges and considerations when applying GN explainability to irregular graph structures and heterogeneous features?
• RQ3 How do pooling choices and edge connectivity influence the quality and interpretability of explanations in Graph Networks?
• RQ4 Can decomposition-based explanations (like LRP) produce more intuitive attributions for GN predictions compared to variation-based methods?

Key findings

• Variation-based explanations (SA, GBP) may diverge from human intuition about causes in graph-based decisions.
• Decomposition-based explanations (LRP) tend to yield more natural and interpretable attributions for GN predictions.
• LRP can attribute positive and negative relevance to specific atomic features in chemistry graphs, aligning with known solubility-related factors (e.g., R-OH groups increase solubility).
• The role of edges is enhanced by multi-step aggregation in GNs, which helps reveal important connections even when edge features are uninformative.
• Pooling choices (sum/mean vs max) significantly affect explanations, with max pooling potentially obscuring multiple contributing inputs; LRP can mitigate some limitations.
• The authors provide an open-source PyTorch implementation to support GN explanations (github.com/baldassarreFe/graph-network-explainability).