[Paper Review] Explain Graph Neural Networks to Understand Weighted Graph Features in Node Classification

Xiaoxiao Li, João Saúde|arXiv (Cornell University)|Feb 2, 2020

Advanced Graph Neural Networks18 references4 citations

TL;DR

This paper proposes a novel framework for explaining Graph Neural Networks (GNNs) in node classification tasks on weighted, directed graphs by identifying informative components (edge patterns) and node feature importance. It introduces post-hoc explanation methods—MMI mask and GGD—along with a pipeline for disentangling topological vs. feature-based contributions, demonstrating that explanations align with human reasoning on synthetic and real-world datasets like PubMed and Bitcoin OTC.

ABSTRACT

Real data collected from different applications that have additional topological structures and connection information are amenable to be represented as a weighted graph. Considering the node labeling problem, Graph Neural Networks (GNNs) is a powerful tool, which can mimic experts' decision on node labeling. GNNs combine node features, connection patterns, and graph structure by using a neural network to embed node information and pass it through edges in the graph. We want to identify the patterns in the input data used by the GNN model to make a decision and examine if the model works as we desire. However, due to the complex data representation and non-linear transformations, explaining decisions made by GNNs is challenging. In this work, we propose new graph features' explanation methods to identify the informative components and important node features. Besides, we propose a pipeline to identify the key factors used for node classification. We use four datasets (two synthetic and two real) to validate our methods. Our results demonstrate that our explanation approach can mimic data patterns used for node classification by human interpretation and disentangle different features in the graphs. Furthermore, our explanation methods can be used for understanding data, debugging GNN models, and examine model decisions.

Motivation & Objective

• To address the challenge of explaining GNN decisions in weighted, directed graphs where both node and edge features influence classification.
• To develop post-hoc explanation methods that identify which graph components (edges and node features) are most influential in GNN predictions.
• To create a pipeline that disentangles whether topological structure or node features are the primary drivers in GNN classification.
• To evaluate explanation quality using consistency, contrastivity, and sparsity metrics on real and synthetic datasets.
• To validate that explanations align with human interpretation and can support debugging, trust-building, and feature engineering.

Proposed method

• Proposes two explanation perspectives: Informative Components Detection (focusing on edge patterns) and Node Feature Importance (focusing on feature relevance).
• Adapts CNN visualization techniques (e.g., Grad-CAM, LRP) to GNNs for weighted, directed graphs with edge weights and node features.
• Introduces two explanation methods: MMI mask (maximally meaningful subgraph) and GGD (gradient-based graph disentanglement) for identifying key subgraphs and feature contributions.
• Uses distance and similarity maps to disentangle the role of topology (connectivity patterns) versus node features in classification decisions.
• Employs a pipeline with evaluation metrics: consistency (within-class similarity), contrastivity (between-class difference), and sparsity (few high-importance edges) to validate explanation quality.
• Applies the framework to four datasets: two synthetic (SynComp, SynNode) and two real (PubMed, Bitcoin OTC), with GNNs trained on node classification.

Experimental results

Research questions

• RQ1 Which edge patterns and node features are most influential in a GNN’s node classification decision on weighted, directed graphs?
• RQ2 Can explanation methods identify components that align with human reasoning about graph structure and feature importance?
• RQ3 Is topological structure or node features the dominant factor in GNN classification for different graph types?
• RQ4 How can explanation quality be quantitatively evaluated in graph-level interpretability?
• RQ5 Can disentangled explanations reveal group similarities and distinguishable patterns across node classes?

Key findings

• On the PubMed dataset, the GGD method achieved a consistency score of 2.14 and contrastivity of 2.07, with sparsity of 0.049, indicating high-quality, sparse explanations.
• On the Bitcoin OTC dataset, the MMI mask method achieved consistency of 1.81 and contrastivity of 2.45, with sparsity of 0.132, showing strong discriminative performance.
• Distance maps revealed that topology was the key factor in SynComp (where in-class distances were smaller), while node features dominated in SynNode and PubMed.
• Similarity maps showed significant within-class similarity in SynNode and PubMed, confirming that node features were the primary classification signal in these datasets.
• The informative components detected by the MMI mask matched human-defined rules: risky nodes had negative ratings, trustworthy nodes had high positive ratings, and neutral nodes had many rating 1s.
• The explanation pipeline successfully disentangled the contributions of topology and features, enabling model debugging and transparency in GNN decisions.