[论文解读] Cross-modality Guidance-aided Multi-modal Learning with Dual Attention for MRI Brain Tumor Grading

Dunyuan Xu, Xi Wang+2 more|arXiv (Cornell University)|Jan 17, 2024

Domain Adaptation and Few-Shot Learning被引用 4

一句话总结

该论文提出了一种基于双注意力机制的跨模态引导多模态学习框架，用于基于MRI的脑肿瘤分级，利用主要模态引导次要模态的特征学习，并通过空间注意力和切片注意力机制增强信息特征的提取。该方法在BraTS2018数据集上达到0.985 ± 0.019的AUC，在BraTS2019数据集上达到0.966 ± 0.021的AUC，显著优于当前最先进方法。

ABSTRACT

Brain tumor represents one of the most fatal cancers around the world, and is very common in children and the elderly. Accurate identification of the type and grade of tumor in the early stages plays an important role in choosing a precise treatment plan. The Magnetic Resonance Imaging (MRI) protocols of different sequences provide clinicians with important contradictory information to identify tumor regions. However, manual assessment is time-consuming and error-prone due to big amount of data and the diversity of brain tumor types. Hence, there is an unmet need for MRI automated brain tumor diagnosis. We observe that the predictive capability of uni-modality models is limited and their performance varies widely across modalities, and the commonly used modality fusion methods would introduce potential noise, which results in significant performance degradation. To overcome these challenges, we propose a novel cross-modality guidance-aided multi-modal learning with dual attention for addressing the task of MRI brain tumor grading. To balance the tradeoff between model efficiency and efficacy, we employ ResNet Mix Convolution as the backbone network for feature extraction. Besides, dual attention is applied to capture the semantic interdependencies in spatial and slice dimensions respectively. To facilitate information interaction among modalities, we design a cross-modality guidance-aided module where the primary modality guides the other secondary modalities during the process of training, which can effectively leverage the complementary information of different MRI modalities and meanwhile alleviate the impact of the possible noise.

研究动机与目标

• 解决单模态模型和传统多模态融合中噪声问题的局限性。
• 降低传统模态融合策略中因噪声导致的性能下降。
• 通过利用MRI序列之间的互补信息，提升模型的鲁棒性和诊断准确性。
• 开发一种引导机制，利用主要模态作为参考，选择性增强来自次要模态的信息特征。
• 在仅使用原始MRI数据、无需任何预处理的情况下实现高性能。

提出的方法

• 采用ResNet Mix卷积作为轻量化主干网络，实现跨MRI模态的高效特征提取。
• 引入双注意力机制，捕捉空间维度和切片维度上的语义依赖关系。
• 设计跨模态引导模块，使性能最优的模态引导性能较差模态的学习过程。
• 将主要模态的高层特征用于引导次要模态的低层特征，弥合语义鸿沟。
• 采用累积学习策略，在训练阶段保留先前学习到的有价值特征。
• 采用两阶段训练流程，结合引导机制与注意力机制，逐步优化特征学习。

实验结果

研究问题

• RQ1 主要MRI模态能否有效引导次要模态的学习过程，以减少噪声并提升性能？
• RQ2 空间与切片维度上的双注意力机制如何增强肿瘤分级的特征表示？
• RQ3 跨模态引导是否能缓解多模态MRI数据简单融合导致的性能下降？
• RQ4 所提方法是否能在不依赖预处理（如肿瘤ROI标注或放射组学特征提取）的情况下实现最先进性能？
• RQ5 该模型在具有不同数据分布的多个数据集上是否表现出足够的鲁棒性？

主要发现

• 所提方法在BraTS2018数据集上达到0.985 ± 0.019的AUC，显著优于单模态与多模态基线方法。
• 在BraTS2019数据集上，该方法达到0.966 ± 0.021的AUC，展现出优异的泛化能力与鲁棒性。
• 消融实验证实，跨模态引导与双注意力机制组件对实现最优性能均不可或缺。
• 该模型在无需肿瘤ROI真实标签或放射组学特征提取的情况下，仍优于现有最先进方法。
• 通过三折交叉验证，框架展现出更强的鲁棒性，表明在不同数据划分下性能保持一致。
• 尽管性能优异，但模型参数量较大且采用两阶段训练流程，限制了其效率与复现便捷性。