[论文解读] Deep learning with noisy labels: exploring techniques and remedies in medical image analysis

Davood Karimi, Haoran Dou+2 more|arXiv (Cornell University)|Dec 5, 2019

Machine Learning and Data Classification被引用 4

一句话总结

本文研究了医学影像数据集中由观察者间差异、人为错误及自动标注引起的标签噪声，并评估了减轻其对深度学习模型影响的技术。通过在三个医学影像数据集上的实验，作者表明，建模标注者混淆并采用不确定性感知训练可显著提升性能，优于简单的标签过滤或忽略高损失样本的方法。

ABSTRACT

Supervised training of deep learning models requires large labeled datasets. There is a growing interest in obtaining such datasets for medical image analysis applications. However, the impact of label noise has not received sufficient attention. Recent studies have shown that label noise can significantly impact the performance of deep learning models in many machine learning and computer vision applications. This is especially concerning for medical applications, where datasets are typically small, labeling requires domain expertise and suffers from high inter- and intra-observer variability, and erroneous predictions may influence decisions that directly impact human health. In this paper, we first review the state-of-the-art in handling label noise in deep learning. Then, we review studies that have dealt with label noise in deep learning for medical image analysis. Our review shows that recent progress on handling label noise in deep learning has gone largely unnoticed by the medical image analysis community. To help achieve a better understanding of the extent of the problem and its potential remedies, we conducted experiments with three medical imaging datasets with different types of label noise, where we investigated several existing strategies and developed new methods to combat the negative effect of label noise. Based on the results of these experiments and our review of the literature, we have made recommendations on methods that can be used to alleviate the effects of different types of label noise on deep models trained for medical image analysis. We hope that this article helps the medical image analysis researchers and developers in choosing and devising new techniques that effectively handle label noise in deep learning.

研究动机与目标

• 研究医学影像数据集中标签噪声的普遍性及其影响，特别是由观察者间差异和自动标注引起的噪声。
• 评估现有及新型深度学习技术在医学图像分析中处理标签噪声的性能。
• 为临床深度学习流程中选择和应用抗噪声训练策略提供基于证据的建议。
• 证明建模标注者不确定性和混淆矩阵可优于标签过滤或忽略高损失样本的方法。

提出的方法

• 评估了多种标签噪声缓解策略，包括损失校正、标签平滑和不确定性感知训练。
• 提出了一种方法，通过利用卷积神经网络（CNN）预测的不确定性来识别标注不足的病灶，从而检测并纠正缺失的标注。
• 在端到端训练过程中，采用基于混淆矩阵的方法来建模单个专家标注错误。
• 利用来自相关成像序列（如T2加权像到弥散加权MRI）的计算机生成的噪声标签作为弱监督信号，以提高模型泛化能力。
• 比较了不同方法的性能，包括标签过滤、损失加权和基于集成的标签融合。
• 在具有不同噪声类型的多个数据集上训练模型：专家标注的病灶中存在缺失的小结构、多位专家分割存在观察者间差异、基于图谱的噪声分割。

实验结果

研究问题

• RQ1 由观察者间差异引起的标签噪声在医学图像分析中如何影响深度学习模型的性能？
• RQ2 针对医学影像中不同类型标签噪声，哪些标签噪声缓解技术最为有效？
• RQ3 在训练过程中建模标注者混淆是否优于传统的标签融合或过滤策略？
• RQ4 当真实标签稀缺时，来自相关成像模态的弱监督标签能在多大程度上提升模型性能？
• RQ5 基于不确定性的缺失标注检测是否能提升病灶分割任务中的泛化能力？

主要发现

• 在多位专家分割任务中，训练过程中建模标注者混淆显著优于标签融合和损失过滤方法。
• 利用CNN预测的不确定性可有效检测并纠正小、微弱或孤立病灶中的缺失标注。
• 使用与DW-MRI匹配的T2加权图像生成的计算机噪声标签进行训练，其性能优于仅使用65个体素手动分割的DW-MRI图像进行训练。
• 忽略高损失样本的效果不如建模标签不确定性，尤其是在存在系统性标注偏差的数据集中。
• 若自动系统生成的标签过于不准确，则其有害；但若与真实解剖结构足够一致，则具有积极作用。
• 结合混淆建模的多位专家标注带来的性能提升，高于单专家标注或简单平均方法。