[Paper Review] ADAPT: Alzheimer Diagnosis through Adaptive Profiling Transformers

Yifeng Wang, Ke Chen+1 more|arXiv (Cornell University)|Jan 12, 2024

Brain Tumor Detection and ClassificationNeuroscience3 citations

TL;DR

MedTransformer proposes a 2D Vision Transformer-based model that decomposes 3D MRI scans into axial, coronal, and sagittal slices, using shared and dimension-specific self-attention, intra- and inter-dimension cross-attention, and a novel fusion attention mechanism to improve Alzheimer’s disease classification. It achieves state-of-the-art performance across multiple datasets, with a 9.03% accuracy gain over ablated variants on ADNI.

ABSTRACT

Automated diagnosis of Alzheimer Disease(AD) from brain imaging, such as magnetic resonance imaging (MRI), has become increasingly important and has attracted the community to contribute many deep learning methods. However, many of these methods are facing a trade-off that 3D models tend to be complicated while 2D models cannot capture the full 3D intricacies from the data. In this paper, we introduce a new model structure for diagnosing AD, and it can complete with performances of 3D models while essentially is a 2D method (thus computationally efficient). While the core idea lies in new perspective of cutting the 3D images into multiple 2D slices from three dimensions, we introduce multiple components that can further benefit the model in this new perspective, including adaptively selecting the number of sclices in each dimension, and the new attention mechanism. In addition, we also introduce a morphology augmentation, which also barely introduces new computational loads, but can help improve the diagnosis performances due to its alignment to the pathology of AD. We name our method ADAPT, which stands for Alzheimer Diagnosis through Adaptive Profiling Transformers. We test our model from a practical perspective (the testing domains do not appear in the training one): the diagnosis accuracy favors our ADAPT, while ADAPT uses less parameters than most 3D models use.

Motivation & Objective

• Address the limitations of 3D CNNs and pure 3D transformers in Alzheimer’s disease diagnosis due to high computational cost and data scarcity.
• Overcome the inefficiency and poor generalization of 3D models by leveraging 2D Vision Transformers on 2D slices of 3D MRI.
• Improve feature extraction and cross-slice dependency modeling across axial, coronal, and sagittal planes using specialized attention mechanisms.
• Enhance model robustness and generalization through a novel morphology augmentation technique based on erosion and dilation.
• Demonstrate superior performance on multi-institutional datasets (ADNI, AIBL, MIRIAD, OASIS) compared to ablated variants and existing methods.

Proposed method

• Decomposes 3D MRI volumes into three orthogonal 2D slice sequences: axial, coronal, and sagittal views.
• Employs shared self-attention encoders across all three dimensions to learn common spatial patterns.
• Uses dimension-specific self-attention encoders to capture unique structural features within each view direction.
• Applies intra-dimension cross-attention to aggregate features across slices within the same anatomical plane.
• Introduces inter-dimension cross-attention to fuse representations across axial, coronal, and sagittal views.
• Proposes a fusion attention mechanism that combines multi-view features while preserving unique information, enhancing global context modeling.

Experimental results

Research questions

• RQ1 Can a 2D Vision Transformer-based model effectively classify Alzheimer’s disease from 3D MRI scans by processing 2D slices across multiple anatomical planes?
• RQ2 How do different attention mechanisms—shared, dimension-specific, intra-dimension, and inter-dimension—impact diagnostic performance?
• RQ3 To what extent does morphology augmentation improve model generalization and performance on AD vs. normal classification?
• RQ4 How does the number of slices and transformer depth (number of layers) per encoder block affect model accuracy and robustness?
• RQ5 What is the contribution of each architectural component (e.g., class token attention, fusion attention) to the final diagnostic performance?

Key findings

• MedTransformer achieves a test accuracy of 0.811 on the OASIS dataset, outperforming the next best variant by 2.8%.
• On the ADNI dataset, MedTransformer achieves 78.1% test accuracy, a 9.03% improvement over the variant without morphology augmentation.
• The ablation study shows that removing the fusion attention mechanism leads to the largest performance drop, indicating its critical role in feature integration.
• Using 30 slices per view dimension yields optimal performance, as both fewer and more slices lead to suboptimal results due to insufficient or noisy information.
• The model with 3 layers per encoder block (1+1+2+2 configuration) achieves the highest accuracy across all datasets, demonstrating optimal depth balance.
• Visualization of attention maps confirms that MedTransformer focuses on atrophic brain regions—key AD indicators—validating its clinical interpretability and pathology relevance.