[论文解读] Medical Transformer: Universal Brain Encoder for 3D MRI Analysis

Eunji Jun, Seungwoo Jeong+2 more|arXiv (Cornell University)|Apr 28, 2021

Advanced Neural Network Applications参考文献 24被引用 32

一句话总结

一个自监督、多视角的基于 Transformer 的骨干网络（Medical Transformer），在大规模脑部3D MRI数据上预训练，以实现面向脑疾病诊断、脑年龄预测和脑肿瘤分割的参数高效迁移学习。

ABSTRACT

Transfer learning has gained attention in medical image analysis due to limited annotated 3D medical datasets for training data-driven deep learning models in the real world. Existing 3D-based methods have transferred the pre-trained models to downstream tasks, which achieved promising results with only a small number of training samples. However, they demand a massive amount of parameters to train the model for 3D medical imaging. In this work, we propose a novel transfer learning framework, called Medical Transformer, that effectively models 3D volumetric images in the form of a sequence of 2D image slices. To make a high-level representation in 3D-form empowering spatial relations better, we take a multi-view approach that leverages plenty of information from the three planes of 3D volume, while providing parameter-efficient training. For building a source model generally applicable to various tasks, we pre-train the model in a self-supervised learning manner for masked encoding vector prediction as a proxy task, using a large-scale normal, healthy brain magnetic resonance imaging (MRI) dataset. Our pre-trained model is evaluated on three downstream tasks: (i) brain disease diagnosis, (ii) brain age prediction, and (iii) brain tumor segmentation, which are actively studied in brain MRI research. The experimental results show that our Medical Transformer outperforms the state-of-the-art transfer learning methods, efficiently reducing the number of parameters up to about 92% for classification and

研究动机与目标

• 由于标注数据有限，在3D医学影像中推动迁移学习。
• 提出一个通用、参数高效的骨干网络，将3D MRI建模为多视角2D切片（矢状、冠状、轴位）。
• 通过对大规模健康脑MRI数据进行自监督的掩码编码向量预测进行预训练。
• 证明迁移学习在脑疾病诊断、脑年龄预测和脑肿瘤分割中的应用。
• 在实现或超越现有方法的同时，降低参数数量。

提出的方法

• 从矢状、冠状和轴位平面获得多视角3D转2D切片表示。
• 骨干网络对每个平面使用单独的卷积编码器，连接一个 transformer 以建模切片之间的依赖关系。
• 使用自监督的示例驱动损失和掩码编码向量预测（类似 BERT）在切片令牌上进行预训练。
• 以单尺度（分类/回归）或多尺度（分割）预测进行微调。
• 在 ADNI、BraTS 和相关数据集上与最先进的3D迁移学习和自监督方法进行对比。

实验结果

研究问题

• RQ1 自我监督、多视角的骨干网络在健康脑MRI上预训练后，是否可以作为下游3D MRI任务的通用编码器？
• RQ2 与现有3D迁移学习方法相比，Medical Transformer 是否在保持或提升性能的同时减少模型参数？
• RQ3 脑疾病诊断、脑年龄预测、脑肿瘤分割如何从共享的通用编码器中受益？
• RQ4 多尺度微调策略在分割任务相较于分类/回归任务是否更具优势？

主要发现

• 在三个任务上均超过了最先进的3D迁移学习方法。
• 与竞争者相比，在分类和回归方面参数量最多可减少约92%，在分割方面约97%。
• 在部分训练数据下仍显示出强大性能，使用更少的带标签样本也能保持准确性。
• 脑疾病诊断和脑年龄预测在使用通用骨干网络时表现出显著提升。
• 脑肿瘤分割在 Dice 分数上具有竞争力，在具有挑战性的 ET 区域表现出最显著的改进。