[Paper Review] PIPNet3D: Interpretable Detection of Alzheimer in MRI Scans

Lisa Anita De Santi, Jörg Schlötterer+5 more|arXiv (Cornell University)|Mar 27, 2024

Brain Tumor Detection and ClassificationNeuroscience3 citations

TL;DR

PIPNet3D is a 3D interpretable deep learning model for Alzheimer’s disease diagnosis from structural MRI scans, using part-prototypes learned in 3D space to enable human-understandable reasoning. It achieves performance on par with blackbox models while producing clinically aligned, compact, and coherent prototypes that improve with expert feedback.

ABSTRACT

Information from neuroimaging examinations is increasingly used to support diagnoses of dementia, e.g., Alzheimer's disease. While current clinical practice is mainly based on visual inspection and feature engineering, Deep Learning approaches can be used to automate the analysis and to discover new image-biomarkers. Part-prototype neural networks (PP-NN) are an alternative to standard blackbox models, and have shown promising results in general computer vision. PP-NN's base their reasoning on prototypical image regions that are learned fully unsupervised, and combined with a simple-to-understand decision layer. We present PIPNet3D, a PP-NN for volumetric images. We apply PIPNet3D to the clinical diagnosis of Alzheimer's Disease from structural Magnetic Resonance Imaging (sMRI). We assess the quality of prototypes under a systematic evaluation framework, propose new functionally grounded metrics to evaluate brain prototypes and develop an evaluation scheme to assess their coherency with domain experts. Our results show that PIPNet3D is an interpretable, compact model for Alzheimer's diagnosis with its reasoning well aligned to medical domain knowledge. Notably, PIPNet3D achieves the same accuracy as its blackbox counterpart; and removing the remaining clinically irrelevant prototypes from its decision process does not decrease predictive performance.

Motivation & Objective

• To develop an interpretable 3D deep learning model for Alzheimer’s disease diagnosis from structural MRI scans.
• To address the limitations of existing part-prototype networks in 3D medical imaging, including lack of spatial coherence and semantic quality.
• To evaluate prototype quality using novel metrics and expert assessments grounded in medical domain knowledge.
• To demonstrate that removing clinically irrelevant prototypes does not degrade performance, enhancing model compactness and interpretability.
• To provide a self-explanatory model that aligns with radiological knowledge and supports clinical decision-making.

Proposed method

• PIPNet3D is a 3D-CNN-based part-prototype neural network (PP-NN) that learns a small set of 3D prototypical regions (VOIs) from MRI volumes in an unsupervised manner.
• The model uses a 3D convolutional backbone to extract features, followed by global max-pooling to generate prototype presence scores.
• A linear, sparse decision layer computes predictions based on prototype activations, ensuring interpretability and compactness.
• Novel evaluation metrics—Prototype Brain Entropy and Prototype Localization Consistency—are introduced to assess functional and spatial coherence of prototypes.
• The model is trained with a contrastive pre-training objective to improve prototype quality and generalization.
• Expert evaluation involves radiologists rating prototype coherence, localization, and pattern consistency, with feedback used to refine model interpretability.

Figure 1: Overview of PiPNet3D. 3D prototypes are learned through a CNN backbone. Representations are optimized through a contrastive pre-training step. A linear sparse decision layer computes the predictions based on prototype (VOI) activations.

Experimental results

Research questions

• RQ1 Can a 3D part-prototype network produce interpretable and clinically meaningful explanations for Alzheimer’s disease diagnosis in sMRI?
• RQ2 How do the proposed metrics—Prototype Brain Entropy and Localization Consistency—correlate with prototype quality and model performance?
• RQ3 To what extent do expert-validated prototypes align with established medical knowledge in neuroimaging?
• RQ4 Does removing clinically irrelevant prototypes affect the model’s predictive accuracy or compactness?
• RQ5 Can a self-explanatory model achieve performance comparable to blackbox deep learning models in AD diagnosis?

Key findings

• PIPNet3D achieves the same classification accuracy as its blackbox counterpart, demonstrating that interpretability does not compromise performance.
• Prototype Brain Entropy showed a strong negative correlation (r = -0.89) with F1 score, indicating that purer, less ambiguous prototypes lead to higher model accuracy.
• The average number of prototypes per local explanation was 3.8, with high sparsity (mean 0.993), indicating compact and focused explanations.
• Expert evaluation revealed high inter-rater reliability (ICC: 0.80–0.85) and strong coherence in prototype localization (0.70) and pattern (0.90), especially for AD cases.
• Removing prototypes deemed clinically irrelevant by experts did not result in statistically significant performance drop, confirming that model robustness is preserved while improving compactness.
• Incoherent prototypes were more frequent for the CN (cognitively normal) class (localization coherence: 0.54), possibly due to the model’s need to find evidence for all classes, including absence of pathology.

Figure 2: Example of local and global explanation inherently provided by PIPNet3D (fold 5). The global explanation shows all prototypes of the models decision process. The local explanation shows the model’s reasoning for one particular patient.