[Paper Review] Towards the Explanation of Graph Neural Networks in Digital Pathology with Information Flows

Junchi Yu, Tingyang Xu+1 more|arXiv (Cornell University)|Dec 18, 2021

Machine Learning in Healthcare5 citations

TL;DR

This paper proposes IFEXPLAINER, a novel GNN explainer for digital pathology that generates explanations by measuring directional information flow using a new metric, f-information, ensuring both necessity and sufficiency. It outperforms existing methods on the BRACS breast cancer subtyping dataset in fidelity, separability, and pathologist-intelligible explanations.

ABSTRACT

As Graph Neural Networks (GNNs) are widely adopted in digital pathology, there is increasing attention to developing explanation models (explainers) of GNNs for improved transparency in clinical decisions. Existing explainers discover an explanatory subgraph relevant to the prediction. However, such a subgraph is insufficient to reveal all the critical biological substructures for the prediction because the prediction will remain unchanged after removing that subgraph. Hence, an explanatory subgraph should be not only necessary for prediction, but also sufficient to uncover the most predictive regions for the explanation. Such explanation requires a measurement of information transferred from different input subgraphs to the predictive output, which we define as information flow. In this work, we address these key challenges and propose IFEXPLAINER, which generates a necessary and sufficient explanation for GNNs. To evaluate the information flow within GNN's prediction, we first propose a novel notion of predictiveness, named $f$-information, which is directional and incorporates the realistic capacity of the GNN model. Based on it, IFEXPLAINER generates the explanatory subgraph with maximal information flow to the prediction. Meanwhile, it minimizes the information flow from the input to the predictive result after removing the explanation. Thus, the produced explanation is necessarily important to the prediction and sufficient to reveal the most crucial substructures. We evaluate IFEXPLAINER to interpret GNN's predictions on breast cancer subtyping. Experimental results on the BRACS dataset show the superior performance of the proposed method.

Motivation & Objective

• Address the lack of faithful GNN explanations in digital pathology by ensuring explanations are both necessary and sufficient for predictions.
• Overcome limitations of existing explainers that produce subgraphs which are necessary but not sufficient, or vice versa.
• Develop a new measure of information flow—f-information—that is directional and accounts for the realistic capacity of GNN models.
• Propose a comprehensive evaluation framework based on subgraph-level separability to assess explanation quality from a pathologist’s perspective.
• Enable efficient and faithful interpretation of GNN predictions in clinical tasks such as breast cancer subtyping.

Proposed method

• Introduce f-information as a directional, GNN-capacity-aware measure of information flow from input subgraphs to the prediction.
• Formulate IFEXPLAINER as an optimization problem that maximizes information flow from the explanatory subgraph to the prediction.
• Minimize residual information flow after subgraph removal to ensure the explanation is necessary and sufficient.
• Use a differentiable node mask with entropy and size regularization to generate binarized, interpretable importance scores.
• Design a tractable optimization scheme to efficiently generate explanations without requiring access to internal GNN embeddings.
• Propose pathologist-intelligible evaluation metrics based on subgraph-level separability to assess explanation quality.

Experimental results

Research questions

• RQ1 Can we define a more faithful measure of information flow in GNNs that accounts for model capacity and directionality?
• RQ2 How can we generate GNN explanations that are both necessary and sufficient for a given prediction?
• RQ3 Can we design an explanation method that produces binarized, pathologist-intelligible importance scores?
• RQ4 How do the proposed evaluation metrics based on subgraph separability reflect the quality of explanations compared to existing methods?
• RQ5 To what extent does IFEXPLAINER outperform state-of-the-art explainers in fidelity and interpretability on real-world digital pathology data?

Key findings

• IFEXPLAINER achieves the highest Fidelity- score across all sparsity levels on the BRACS dataset, indicating the explanation is most necessary for the prediction.
• The Fidelity+ score of IFEXPLAINER is superior to all baselines, confirming that the explanation is most sufficient in changing the prediction when removed.
• Visualization results show IFEXPLAINER produces binarized importance scores, with clear emphasis on nuclei critical for breast cancer subtyping, unlike gradient-based or GRAPHLRP methods that assign low scores to most nuclei.
• Abllation studies confirm that both the size and entropy regularization terms are essential: removing them leads to explanations that are either too large or lack binary structure.
• The proposed separability-based evaluation metrics effectively distinguish high-quality explanations, with IFEXPLAINER outperforming baselines on both pair-wise and aggregated separability scores.
• IFEXPLAINER generates explanations that are both more faithful and more interpretable than existing methods, as validated by qualitative and quantitative analysis.