[Paper Review] MProtoNet: A Case-Based Interpretable Model for Brain Tumor Classification with 3D Multi-parametric Magnetic Resonance Imaging

Yuanyuan Wei, Roger Tam+1 more|arXiv (Cornell University)|Apr 13, 2023

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

MProtoNet is a novel case-based interpretable deep learning model for 3D multi-parametric MRI brain tumor classification, extending ProtoPNet to medical imaging with a new attention module using soft masking and online-CAM loss. It achieves state-of-the-art interpretability, with a best activation precision of 0.713 ± 0.058, without requiring human-annotated labels during training.

ABSTRACT

Recent applications of deep convolutional neural networks in medical imaging raise concerns about their interpretability. While most explainable deep learning applications use post hoc methods (such as GradCAM) to generate feature attribution maps, there is a new type of case-based reasoning models, namely ProtoPNet and its variants, which identify prototypes during training and compare input image patches with those prototypes. We propose the first medical prototype network (MProtoNet) to extend ProtoPNet to brain tumor classification with 3D multi-parametric magnetic resonance imaging (mpMRI) data. To address different requirements between 2D natural images and 3D mpMRIs especially in terms of localizing attention regions, a new attention module with soft masking and online-CAM loss is introduced. Soft masking helps sharpen attention maps, while online-CAM loss directly utilizes image-level labels when training the attention module. MProtoNet achieves statistically significant improvements in interpretability metrics of both correctness and localization coherence (with a best activation precision of $0.713\pm0.058$) without human-annotated labels during training, when compared with GradCAM and several ProtoPNet variants. The source code is available at https://github.com/aywi/mprotonet.

Motivation & Objective

• To address the lack of interpretable deep learning models for 3D multi-parametric MRI in brain tumor classification.
• To improve attention localization in medical imaging, especially for 3D volumes where spatial coherence is critical.
• To develop a case-based reasoning model that provides both attribution maps and prototypical explanations without relying on human-annotated labels.
• To evaluate interpretability using metrics like activation precision and incremental deletion score, independent of classification accuracy.
• To extend ProtoPNet to 3D medical imaging, overcoming limitations in localizing relevant tumor regions in complex 3D MRI data.

Proposed method

• Proposes MProtoNet, the first prototype network for 3D multi-parametric MRI in brain tumor classification, extending ProtoPNet to medical imaging.
• Introduces a new attention module with soft masking to sharpen attention maps and improve localization precision.
• Employs an online-CAM loss that directly uses image-level labels during training to guide attention learning.
• Uses a 3D ResNet backbone shared across all models to ensure fair comparison on classification performance.
• Applies soft masking to refine attention maps by suppressing irrelevant regions, enhancing spatial coherence.
• Trains the model end-to-end using only image-level labels, avoiding the need for bounding box or segmentation annotations.

Figure 1: The overall architecture of MProtoNet. Online-CAM only occurs during training.

Experimental results

Research questions

• RQ1 Can a case-based prototype network be effectively extended to 3D multi-parametric MRI for brain tumor classification?
• RQ2 How do soft masking and online-CAM loss improve attention localization and interpretability in 3D medical images?
• RQ3 Does the proposed MProtoNet achieve better interpretability than GradCAM and ProtoPNet variants without human-annotated labels?
• RQ4 To what extent does the model maintain classification performance while enhancing interpretability?
• RQ5 Can the model generate both attention maps and case-based prototypical explanations that are both correct and spatially coherent?

Key findings

• MProtoNet C achieves the highest activation precision of 0.713 ± 0.058, significantly outperforming XProtoNet (p < 0.001) and other baselines.
• The incremental deletion score (IDS) of MProtoNet C is 0.079 ± 0.034, showing statistically significant improvement over XProtoNet (p = 0.031).
• The model maintains balanced accuracy of 0.858 ± 0.048, comparable to baseline CNNs and XProtoNet, indicating no degradation in classification performance.
• Soft masking and online-CAM loss together contribute to statistically significant gains in both IDS and activation precision, confirming their importance.
• Visualization results show MProtoNet C produces more coherent and localized attention maps than GradCAM and ProtoPNet variants, especially in complex 3D tumor regions.
• The model achieves superior interpretability without requiring any human-annotated labels during training, demonstrating strong potential for real-world 3D medical imaging applications.

Figure 2: Demonstration of the case-based reasoning in MProtoNet. It can provide case-based prototypical explanations with both attention maps and similarity scores.