[Paper Review] Global Explainability of GNNs via Logic Combination of Learned Concepts

Steve Azzolin, Antonio Longa+3 more|arXiv (Cornell University)|Oct 13, 2022

Topic Modeling11 citations

TL;DR

This paper introduces GLGExplainer, the first global explainability method for Graph Neural Networks (GNNs) that generates human-interpretable logic formulas by combining learned graphical concepts from local explanations. Using a fully differentiable architecture, it aggregates local explanations into Boolean combinations of concepts, achieving high fidelity and faithfulness to model behavior on both synthetic and real-world datasets.

ABSTRACT

While instance-level explanation of GNN is a well-studied problem with plenty of approaches being developed, providing a global explanation for the behaviour of a GNN is much less explored, despite its potential in interpretability and debugging. Existing solutions either simply list local explanations for a given class, or generate a synthetic prototypical graph with maximal score for a given class, completely missing any combinatorial aspect that the GNN could have learned. In this work, we propose GLGExplainer (Global Logic-based GNN Explainer), the first Global Explainer capable of generating explanations as arbitrary Boolean combinations of learned graphical concepts. GLGExplainer is a fully differentiable architecture that takes local explanations as inputs and combines them into a logic formula over graphical concepts, represented as clusters of local explanations. Contrary to existing solutions, GLGExplainer provides accurate and human-interpretable global explanations that are perfectly aligned with ground-truth explanations (on synthetic data) or match existing domain knowledge (on real-world data). Extracted formulas are faithful to the model predictions, to the point of providing insights into some occasionally incorrect rules learned by the model, making GLGExplainer a promising diagnostic tool for learned GNNs.

Motivation & Objective

• Address the lack of global explainability methods for GNNs that capture the model's overall decision-making behavior beyond instance-level explanations.
• Overcome limitations of existing global approaches, such as generating synthetic prototypes without compositional structure or requiring domain knowledge.
• Enable faithful, human-interpretable global explanations by deriving logic formulas from learned graphical concepts extracted from local explanations.
• Provide a diagnostic tool to uncover both correct and incorrect rules learned by GNNs, supporting model debugging and trust-building in real-world applications.

Proposed method

• Utilize off-the-shelf local explainers to extract instance-level subgraph explanations for each prediction.
• Cluster local explanations into graphical concepts using a differentiable clustering mechanism, forming interpretable, high-level units.
• Train an end-to-end differentiable model (E-LEN) to map node features and concepts to class predictions, learning logic-based decision rules.
• Apply a discretization trick to enforce hard prototype assignments, stabilizing concept learning and improving formula fidelity.
• Optimize the entire pipeline end-to-end using a loss function combining cross-entropy, concept purity, and fidelity objectives.
• Extract final logic formulas as Boolean combinations of learned concepts, ensuring alignment with model predictions and data domain semantics.

Experimental results

Research questions

• RQ1 Can global explanations for GNNs be effectively expressed as logic formulas over learned graphical concepts derived from local explanations?
• RQ2 How well can a fully differentiable architecture combine local explanations into coherent, high-level logical rules that faithfully represent the GNN’s behavior?
• RQ3 Does the proposed method outperform existing global explainers like XGNN and GLocalX in terms of fidelity and interpretability on both synthetic and real-world data?
• RQ4 To what extent can GLGExplainer uncover incorrect or spurious rules learned by GNNs, serving as a diagnostic tool?
• RQ5 How does the discretization trick impact the quality and stability of concept learning and formula fidelity?

Key findings

• GLGExplainer achieves high fidelity (up to 0.95 on BAMultiShapes, 0.89 on HIN, and 0.84 on Mutagenicity) in summarizing GNN behavior using logic formulas.
• With optimal prototype counts (6 for BAMultiShapes, 2 for Mutagenicity, 4 for HIN), the method balances fidelity and concept purity, with concept purity reaching 0.99 on Mutagenicity.
• The discretization trick reduces concept vector entropy to zero and more than doubles fidelity compared to the non-discretized variant, significantly improving formula faithfulness.
• GLGExplainer successfully identifies both correct and occasionally incorrect rules learned by the model, demonstrating diagnostic capability.
• The method outperforms XGNN, which fails to generate meaningful global explanations across all evaluated tasks.
• Fidelity correlates strongly with formula accuracy, indicating that the extracted logic formulas are reliable proxies for the model’s predictions.