[论文解读] Med3D: Transfer Learning for 3D Medical Image Analysis

Sihong Chen, Kai Ma+1 more|arXiv (Cornell University)|Apr 1, 2019

COVID-19 diagnosis using AI参考文献 40被引用 332

一句话总结

Med3D创建一个大型的多域3D医学分割数据集（3DSeg-8），训练一个带有八个解码器的异构3D编码器，并展示在肺分割、肺结节分类和LiTS肝脏分割等任务上的强大迁移性能，优于Kinetics预训练和从头训练。

ABSTRACT

The performance on deep learning is significantly affected by volume of training data. Models pre-trained from massive dataset such as ImageNet become a powerful weapon for speeding up training convergence and improving accuracy. Similarly, models based on large dataset are important for the development of deep learning in 3D medical images. However, it is extremely challenging to build a sufficiently large dataset due to difficulty of data acquisition and annotation in 3D medical imaging. We aggregate the dataset from several medical challenges to build 3DSeg-8 dataset with diverse modalities, target organs, and pathologies. To extract general medical three-dimension (3D) features, we design a heterogeneous 3D network called Med3D to co-train multi-domain 3DSeg-8 so as to make a series of pre-trained models. We transfer Med3D pre-trained models to lung segmentation in LIDC dataset, pulmonary nodule classification in LIDC dataset and liver segmentation on LiTS challenge. Experiments show that the Med3D can accelerate the training convergence speed of target 3D medical tasks 2 times compared with model pre-trained on Kinetics dataset, and 10 times compared with training from scratch as well as improve accuracy ranging from 3% to 20%. Transferring our Med3D model on state-the-of-art DenseASPP segmentation network, in case of single model, we achieve 94.6\% Dice coefficient which approaches the result of top-ranged algorithms on the LiTS challenge.

研究动机与目标

• 由于标注数据有限，推动大规模3D医学预训练的必要性。
• 创建一个大型多域3D医学分割数据集（3DSeg-8）。
• 设计Med3D，具有共享编码器和多分支解码器，以应对不完整的注释。
• 证明Med3D预训练的编码器能提升分割和分类任务的迁移学习效果。
• 向社区提供预训练的Med3D模型和代码。

提出的方法

• 将八个3D分割数据集合并为3DSeg-8，覆盖多样的模态和目标。
• 对空间间隔和强度进行归一化，以降低域间差异。
• 使用基于ResNet的3D编码器，具有八个并行解码分支，以处理不完整的注释。
• 以多域目标进行训练，分享编码器而解码器对每个数据集进行专业化。
• 将Med3D编码器迁移到下游任务（肺分割、肺结节分类、LiTS肝脏分割）。
• 与在Kinetics上预训练的模型以及从头训练进行比较。

实验结果

研究问题

• RQ1 多域3D医学预训练（Med3D）是否学习到可在器官和模态间通用的3D特征？
• RQ2 与Kinetics或从头训练相比，Med3D预训练是否能加速收敛并提高下游3D医学任务的准确性？
• RQ3 预训练数据的多样性（一个域与八个域）如何影响迁移性能？
• RQ4 Med3D特征是否能同时提升3D医学影像的分割与分类任务？
• RQ5 Med3D在具有挑战性的公共LiTS肝脏分割任务上的表现如何？

主要发现

网络	预训练	分割 Dice系数	分类准确率
3D-ResNet10	TFS	71.30%	79.80%
Med3D	Med3D	87.16%	86.87%
3D-ResNet18	TFS	75.22%	80.80%
Kin	Kin	83.21%	82.83%
Med3D	Med3D	89.31%	89.90%
3D-ResNet34	TFS	76.82%	83.84%
Kin	Kin	85.82%	83.84%
Med3D	Med3D	93.31%	89.90%
3D-ResNet50	TFS	71.75%	84.85%
Kin	Kin	87.11%	74.75%
Med3D	Med3D	93.31%	89.90%
3D-ResNet101	TFS	72.10%	81.82%
Kin	Kin	88.32%	74.75%
Med3D	Med3D	92.79%	90.91%
3D-ResNet152	TFS	73.29%	73.74%
Kin	Kin	88.61%	75.76%
Med3D	Med3D	92.33%	90.91%
3D-PreResNet200	TFS	71.29%	76.77%
Kin	Kin	-	-
Med3D	Med3D	93.82%	91.92%

• Med3D相对于Kinetics和从头训练在目标任务上加速收敛并提升准确性（在某些任务中Dice提升约20%）。
• 当使用全部八个域进行训练时（八域Med3D）的结果最佳，优于单域、双域和四域变体。
• 将Med3D编码器迁移至肺分割和肺部结节分类，在多种ResNet骨干上获得比Kin预训练和从头基线更高的Dice/准确性。
• 在LiTS上，Med3D达到94.6% Dice和1.9 ASSD，超越若干纯3D方法和Kin预训练。
• 在DenseASPP的肝脏分割中，结合Med3D实现的Dice接近状态-of-the-art水平。