[论文解读] Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysis

Zongwei Zhou, Vatsal Sodha+5 more|arXiv (Cornell University)|Aug 19, 2019

AI in cancer detection参考文献 15被引用 44

一句话总结

论文介绍 Generic Autodidactic Models (Models Genesis)，在未标注的三维医学影像上进行自监督训练，在多项三维和二维任务上展现出强大的迁移性能，超过从头训练和二维 ImageNet 基线。该模型已开源，供社区使用。

ABSTRACT

Transfer learning from natural image to medical image has established as one of the most practical paradigms in deep learning for medical image analysis. However, to fit this paradigm, 3D imaging tasks in the most prominent imaging modalities (e.g., CT and MRI) have to be reformulated and solved in 2D, losing rich 3D anatomical information and inevitably compromising the performance. To overcome this limitation, we have built a set of models, called Generic Autodidactic Models, nicknamed Models Genesis, because they are created ex nihilo (with no manual labeling), self-taught (learned by self-supervision), and generic (served as source models for generating application-specific target models). Our extensive experiments demonstrate that our Models Genesis significantly outperform learning from scratch in all five target 3D applications covering both segmentation and classification. More importantly, learning a model from scratch simply in 3D may not necessarily yield performance better than transfer learning from ImageNet in 2D, but our Models Genesis consistently top any 2D approaches including fine-tuning the models pre-trained from ImageNet as well as fine-tuning the 2D versions of our Models Genesis, confirming the importance of 3D anatomical information and significance of our Models Genesis for 3D medical imaging. This performance is attributed to our unified self-supervised learning framework, built on a simple yet powerful observation: the sophisticated yet recurrent anatomy in medical images can serve as strong supervision signals for deep models to learn common anatomical representation automatically via self-supervision. As open science, all pre-trained Models Genesis are available at https://github.com/MrGiovanni/ModelsGenesis.

研究动机与目标

• 证明基于未标注医学影像构建的源模型能够通过迁移学习产生高性能的目标模型。
• 表明 3D Models Genesis 比二维迁移更好地利用了三维解剖信息。
• 在多器官、疾病和模态下评估 Genesis 以确立其通用性。
• 与从头训练和基于 ImageNet 的迁移进行比较，以量化三维自监督的优势。

提出的方法

• 提出一个统一的自监督框架，包含四种变换（非线性强度、局部像素置换、外涂块、内涂块），整合为一个单一的图像重建任务。
• 训练一个编码器-解码器网络，从变换后的补丁中恢复原始补丁，从而实现可迁移的特征学习。
• 在未标注的胸部CT数据（LIDC-IDRI）上对 Genesis Chest CT 进行预训练，在 ChestX-ray83 上对 Genesis Chest X-ray 进行预训练；为切片任务生成一个二维版本（Genesis Chest CT 2D）。
• 在下游任务上微调预训练的编码器或编码器-解码器，以评估分割和分类性能。
• 评估跨越七个目标任务，覆盖 3D 和 2D 的分类与分割（如 NCC、NCS、ECC、LCS、BMS、DXC、IUC）。
• 与从头训练、ImageNet 微调（2D 和 3D）以及 NiftyNet 基线进行比较；分析同域迁移与跨域迁移。

实验结果

研究问题

• RQ1 未标注医学影像的自监督预训练是否能为多样化的三维医学影像任务提供可迁移的表征？
• RQ2 Genesis 中利用三维解剖信息是否优于来自 ImageNet 的二维迁移学习？
• RQ3 相较于单一代理任务，统一的多任务自监督框架对医学影像表征是否更有利？
• RQ4 在疾病、器官和模态下，使用 Genesis 时同域迁移与跨域迁移有何比较？

主要发现

• Genesis 模型在七个目标任务上始终优于从头训练。
• Genesis Chest CT（3D）在同任务上通常优于经过 2D ImageNet 微调的模型，突出3D解剖信息的价值。
• 降级的2D版本（Genesis Chest CT 2D 和 Genesis X-ray）在同一模态下达到与 ImageNet 微调等效的性能。
• 在各任务中，同域迁移优选，跨域迁移有潜力但通常不如同域迁移。
• 统一的自监督框架在多项任务上带来稳健提升，且优于单一代理方案。
• 所有预训练的 Models Genesis 已公开发布，以支持开放科学。