[论文解读] Less is More: Simultaneous View Classification and Landmark Detection for Abdominal Ultrasound Images

Zhoubing Xu, Yuankai Huo+5 more|arXiv (Cornell University)|May 25, 2018

Radiomics and Machine Learning in Medical Imaging参考文献 11被引用 7

一句话总结

本文提出了一种统一的多任务深度学习框架，通过单一卷积神经网络同时执行腹部超声图像视图分类与器官标志点检测，利用全局卷积核、坐标约束和条件对抗模块。在187,219张图像上进行评估，该方法的视图分类准确率超过放射科专家间的一致性水平，基于标志点的测量误差与人眼观察者间差异水平相当。

ABSTRACT

An abdominal ultrasound examination, which is the most common ultrasound examination, requires substantial manual efforts to acquire standard abdominal organ views, annotate the views in texts, and record clinically relevant organ measurements. Hence, automatic view classification and landmark detection of the organs can be instrumental to streamline the examination workflow. However, this is a challenging problem given not only the inherent difficulties from the ultrasound modality, e.g., low contrast and large variations, but also the heterogeneity across tasks, i.e., one classification task for all views, and then one landmark detection task for each relevant view. While convolutional neural networks (CNN) have demonstrated more promising outcomes on ultrasound image analytics than traditional machine learning approaches, it becomes impractical to deploy multiple networks (one for each task) due to the limited computational and memory resources on most existing ultrasound scanners. To overcome such limits, we propose a multi-task learning framework to handle all the tasks by a single network. This network is integrated to perform view classification and landmark detection simultaneously; it is also equipped with global convolutional kernels, coordinate constraints, and a conditional adversarial module to leverage the performances. In an experimental study based on 187,219 ultrasound images, with the proposed simplified approach we achieve (1) view classification accuracy better than the agreement between two clinical experts and (2) landmark-based measurement errors on par with inter-user variability. The multi-task approach also benefits from sharing the feature extraction during the training process across all tasks and, as a result, outperforms the approaches that address each task individually.

研究动机与目标

• 为通过减少手动标注与测量工作量，实现腹部超声检查流程的自动化。
• 克服在超声扫描仪上部署多个深度学习模型所面临的计算与内存限制。
• 通过在不同任务间共享特征联合学习，提升视图分类与标志点检测的性能。
• 在视图分类中达到临床水平的准确率，测量误差与人类观察者间差异水平相当。
• 开发一种轻量化、单网络解决方案，其性能优于针对单一任务的专用模型。

提出的方法

• 设计了一种多任务学习框架，利用单一深度神经网络联合执行视图分类与标志点检测。
• 引入全局卷积核，以增强图像整体区域的特征表示，适用于两项任务。
• 应用坐标约束对标志点预测进行正则化，确保解剖学上合理的空间位置。
• 引入条件对抗模块，以提升特征解耦与领域泛化能力。
• 通过任务间共享特征提取，实现参数效率，并在训练过程中提升泛化能力。
• 在包含187,219张腹部超声图像的大规模数据集上端到端训练该模型。

实验结果

研究问题

• RQ1 单一深度神经网络能否有效同时完成腹部超声图像的视图分类与标志点检测？
• RQ2 在这些异质性任务间进行联合学习，是否能提升性能，优于分别训练独立模型？
• RQ3 所提出的架构能否在视图分类中达到临床水平的准确率，超过临床专家间的一致性水平？
• RQ4 标志点检测误差能否降低至与人类观察者间差异水平相当的水平？
• RQ5 使用全局卷积核与坐标约束是否能提升定位与分类性能？

主要发现

• 所提出的多任务模型在视图分类中的准确率超过了两名临床专家之间的一致性水平。
• 基于标志点的测量误差与放射科医生之间观察者间差异水平相当。
• 任务间共享特征学习相比为每项任务单独训练模型，显著提升了泛化能力与性能。
• 全局卷积核与坐标约束的集成显著提升了定位准确率与模型鲁棒性。
• 条件对抗模块有助于实现更好的特征解耦，并在具有挑战性的病例中提升性能。
• 单网络架构展现出适合在资源受限的超声扫描仪上部署的计算效率。