[Paper Review] Towards Explanation for Unsupervised Graph-Level Representation Learning

Qinghua Zheng, Jihong Wang+5 more|arXiv (Cornell University)|May 20, 2022

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper introduces Unsupervised Subgraph Information Bottleneck (USIB), a novel method for explaining unsupervised graph-level representations using the Information Bottleneck principle. By maximizing mutual information between explanatory subgraphs and representations while minimizing redundancy, USIB achieves state-of-the-art explanation fidelity, especially on synthetic data, and theoretically links representation robustness and expressiveness to explanation quality.

ABSTRACT

Due to the superior performance of Graph Neural Networks (GNNs) in various domains, there is an increasing interest in the GNN explanation problem "\emph{which fraction of the input graph is the most crucial to decide the model's decision?}" Existing explanation methods focus on the supervised settings, \eg, node classification and graph classification, while the explanation for unsupervised graph-level representation learning is still unexplored. The opaqueness of the graph representations may lead to unexpected risks when deployed for high-stake decision-making scenarios. In this paper, we advance the Information Bottleneck principle (IB) to tackle the proposed explanation problem for unsupervised graph representations, which leads to a novel principle, extit{Unsupervised Subgraph Information Bottleneck} (USIB). We also theoretically analyze the connection between graph representations and explanatory subgraphs on the label space, which reveals that the expressiveness and robustness of representations benefit the fidelity of explanatory subgraphs. Experimental results on both synthetic and real-world datasets demonstrate the superiority of our developed explainer and the validity of our theoretical analysis.

Motivation & Objective

• To address the unexplored problem of explaining unsupervised graph-level representations generated by GNNs.
• To develop a method that identifies the most informative subgraphs responsible for representation decisions without relying on labels.
• To establish a theoretical connection between representation quality (expressiveness and robustness) and explanation fidelity.
• To overcome the challenge of comparing subgraphs (non-Euclidean) and representations (Euclidean) in explanation space.
• To demonstrate the effectiveness of the proposed method across synthetic and real-world graph datasets.

Proposed method

• Propose USIB, a novel explanation framework based on the Information Bottleneck principle, to identify subgraphs most predictive of graph representations.
• Formalize the explanation task using mutual information: maximize I(subgraph; representation) and minimize I(subgraph; input graph).
• Use a differentiable relaxation of mutual information to optimize subgraph selection via gradient descent.
• Integrate a subgraph masking mechanism that allows end-to-end training of the explainer with the GNN encoder.
• Theoretical analysis shows that expressive and robust representations yield higher-fidelity explanations.
• Apply the method to both pre-trained and fine-tuned GNNs to evaluate generalization across model capacities.

Experimental results

Research questions

• RQ1 Can we develop a faithful and general-purpose explanation method for unsupervised graph-level representations?
• RQ2 How do the expressiveness and robustness of graph representations affect the fidelity of explanatory subgraphs?
• RQ3 Is there a theoretical link between representation quality and explanation quality in unsupervised settings?
• RQ4 Can USIB outperform existing unsupervised explanation methods on both synthetic and real-world graphs?
• RQ5 Does the performance of the explainer depend on the underlying GNN's capacity and training regime?

Key findings

• USIB achieves state-of-the-art performance on benchmark datasets, with a 14.8% relative improvement in ACC-AUC and 10.44% in Recall@5 on the synthetic BA3 dataset.
• On real-world datasets (NCI1, PROTEINS), USIB outperforms other unsupervised explanation methods, though it lags behind supervised baselines due to lack of label supervision.
• There is a strong positive correlation between representation expressiveness (accuracy on clean graphs) and explanation fidelity, especially on datasets with high model accuracy.
• Robustness (accuracy on noisy graphs) also correlates positively with explanation quality, particularly when expressiveness differences are small.
• Qualitative results show that untrained GNNs focus on base graph structures, while fine-tuned models shift attention to meaningful motifs like 'house' or 'cycle'.
• USIB's limitation lies in treating edges independently, ignoring higher-order substructure importance, suggesting a need for future work on structural-aware explanation.