[论文解读] 3D Self-Supervised Methods for Medical Imaging

Aiham Taleb, Winfried Loetzsch+5 more|arXiv (Cornell University)|Jun 6, 2020

Medical Image Segmentation Techniques参考文献 67被引用 128

一句话总结

引入五个3D自监督代理任务（3D-CPC、3D-Rot、3D-Jig、3D-RPL、3D-Exe），从未标注的3D医疗影像中学习，在脑肿瘤、胰腺肿瘤和糖网病变任务中提升数据效率与下游性能，并提供开源代码。

ABSTRACT

Self-supervised learning methods have witnessed a recent surge of interest after proving successful in multiple application fields. In this work, we leverage these techniques, and we propose 3D versions for five different self-supervised methods, in the form of proxy tasks. Our methods facilitate neural network feature learning from unlabeled 3D images, aiming to reduce the required cost for expert annotation. The developed algorithms are 3D Contrastive Predictive Coding, 3D Rotation prediction, 3D Jigsaw puzzles, Relative 3D patch location, and 3D Exemplar networks. Our experiments show that pretraining models with our 3D tasks yields more powerful semantic representations, and enables solving downstream tasks more accurately and efficiently, compared to training the models from scratch and to pretraining them on 2D slices. We demonstrate the effectiveness of our methods on three downstream tasks from the medical imaging domain: i) Brain Tumor Segmentation from 3D MRI, ii) Pancreas Tumor Segmentation from 3D CT, and iii) Diabetic Retinopathy Detection from 2D Fundus images. In each task, we assess the gains in data-efficiency, performance, and speed of convergence. Interestingly, we also find gains when transferring the learned representations, by our methods, from a large unlabeled 3D corpus to a small downstream-specific dataset. We achieve results competitive to state-of-the-art solutions at a fraction of the computational expense. We publish our implementations for the developed algorithms (both 3D and 2D versions) as an open-source library, in an effort to allow other researchers to apply and extend our methods on their datasets.

研究动机与目标

• 减少对在3D医学影像中昂贵专家标注的依赖。
• 使用自监督代理任务从未标注数据中学习丰富的3D 表征。
• 实现对下游任务（如分割和检测）的数据高效微调。
• 展示从大规模未标注语料中学习的3D表示向小型下游数据集的迁移能力。

提出的方法

• 提出五个3D自监督任务：3D Contrastive Predictive Coding (3D-CPC)、3D Rotation prediction (3D-Rot)、3D Jigsaw puzzles (3D-Jig)、Relative 3D patch location (3D-RPL)、和3D Exemplar networks (3D-Exe)。
• 3D-CPC 使用编码器和上下文网络，在倒金字塔式的3D上下文中利用 InfoNCE 损失预测未来补丁的潜在表示。
• 3D-RPL 训练在3D网格中对查询补丁的相对位置进行分类，含抖动以避免捷径。
• 3D-Jig 将3D补丁网格的排列解为P类分类任务。
• 3D-Rot 预测离散化的3D旋转（10类），以强制语义理解。
• 3D-Exe 使用三元组损失，在嵌入空间中将变换后的正样本拉近、将负样本推远。

实验结果

研究问题

• RQ1 3D自监督代理任务是否能够相较于2D或从零开始训练，提升来自未标注3D医学影像的学习表征？
• RQ2 是否能通过从大型未标注语料库进行迁移学习，将3D表示迁移到下游任务和更小的数据集？
• RQ3 在脑肿瘤分割、胰腺肿瘤分割和糖网检测中，3D代理任务是否提供数据效率、性能提升和更快收敛？
• RQ4 在现实医学影像基准上，3D方法与2D对比和有监督基线表现如何？

主要发现

模型	ET	WT	TC
3D-From scratch	76.38	87.82	83.11
3D Supervised	78.88	90.11	84.92
2D-CPC	76.60	86.27	82.41
2D-RPL	77.53	87.91	82.56
2D-Jigsaw	76.12	86.28	83.26
2D-Rotation	76.60	88.78	82.41
2D-Exemplar	75.22	84.82	81.87
Popli et al. [66]	74.39	89.41	82.48
Baid et al. [67]	74.80	87.80	82.66
Chandra et al. [68]	74.06	87.19	79.89
Isensee et al. [65]	80.36	90.80	84.32
3D-CPC	80.83	89.88	85.11
3D-RPL	81.28	90.71	86.12
3D-Jigsaw	79.66	89.20	82.52
3D-Rotation	80.21	89.63	84.75
3D-Exemplar	79.46	90.80	83.87

• 3D自监督预训练在数据效率方面具有优越性，尤其在少样本情境中优于从零开始和2D切片基线。
• 在各个任务中，3D方法的Dice分数与最先进的BraTS基线相当或更优，并且常需更少的下游训练轮次。
• 3D表示能从大型未标注3D语料库（如UK Biobank）有效迁移到较小的下游数据集。
• 在BraTS脑肿瘤分割任务中，3D方法优于其2D对比，凸显了全3D上下文的价值。
• 开源实现同时覆盖3D和2D变体，便于采用与扩展。