[论文解读] The Impact of Scanner Domain Shift on Deep Learning Performance in Medical Imaging: an Experimental Study

Brian H.W. Guo, Darui Lu+5 more|arXiv (Cornell University)|Sep 6, 2024

Radiomics and Machine Learning in Medical ImagingMedicine被引用 3

一句话总结

本研究系统评估了在磁共振成像（MRI）、计算机断层扫描（CT）和X射线等多种医学影像模态中，深度学习面临的扫描仪领域偏移问题。研究结果表明，由于扫描仪参数和后处理方法的差异，模型性能显著下降——其中MRI模态下降最严重，X射线模态中度下降，CT模态下降最轻微，凸显了在临床应用的模型开发与部署过程中必须考虑扫描仪差异性的重要性。

ABSTRACT

Purpose: Medical images acquired using different scanners and protocols can differ substantially in their appearance. This phenomenon, scanner domain shift, can result in a drop in the performance of deep neural networks which are trained on data acquired by one scanner and tested on another. This significant practical issue is well-acknowledged, however, no systematic study of the issue is available across different modalities and diagnostic tasks. Materials and Methods: In this paper, we present a broad experimental study evaluating the impact of scanner domain shift on convolutional neural network performance for different automated diagnostic tasks. We evaluate this phenomenon in common radiological modalities, including X-ray, CT, and MRI. Results: We find that network performance on data from a different scanner is almost always worse than on same-scanner data, and we quantify the degree of performance drop across different datasets. Notably, we find that this drop is most severe for MRI, moderate for X-ray, and quite small for CT, on average, which we attribute to the standardized nature of CT acquisition systems which is not present in MRI or X-ray. We also study how injecting varying amounts of target domain data into the training set, as well as adding noise to the training data, helps with generalization. Conclusion: Our results provide extensive experimental evidence and quantification of the extent of performance drop caused by scanner domain shift in deep learning across different modalities, with the goal of guiding the future development of robust deep learning models for medical image analysis.

研究动机与目标

• 调查扫描仪领域偏移对多种医学影像模态中深度学习性能的影响。
• 量化在某一扫描仪上训练的模型在另一扫描仪数据上测试时的性能退化程度。
• 评估数据注入和数据增强在缓解领域偏移影响方面的有效性。
• 理解不同模态对扫描仪领域偏移的敏感性差异，特别是与采集标准化相关的因素。
• 为在具有异构扫描仪的现实临床环境中开发稳健的深度学习模型提供实证指导。

提出的方法

• 在来自单一扫描仪的源域数据上训练卷积神经网络，并在来自不同扫描仪的目标域数据上评估性能。
• 使用了涵盖MRI、CT和X射线模态的七个数据集，覆盖不同解剖区域和诊断任务。
• 在训练集中逐步注入目标域数据的比例，以评估领域泛化能力。
• 向训练图像中添加高斯噪声，以评估其对域外性能的影响。
• 比较不同扫描仪制造商和型号之间的性能表现，重点关注图像采集参数和后处理差异。
• 分析图像特征分布及其对扫描仪特定参数（如TR、TE和层厚）的敏感性。

Figure 1: Seeded Training and Testing Procedure: See Section 3.1 for more details.

实验结果

研究问题

• RQ1 扫描仪领域偏移如何影响不同医学影像模态（MRI、CT、X射线）中深度学习的性能？
• RQ2 当在某一扫描仪上训练的模型在另一扫描仪的数据上测试时，性能退化的程度有多大？
• RQ3 在训练过程中包含目标域数据如何影响模型在不同扫描仪间的泛化能力？
• RQ4 向训练数据中添加噪声是否能提升模型对域外扫描仪数据的泛化能力？
• RQ5 为何MRI比CT对扫描仪领域偏移更敏感，尽管两者均为MRI和CT模态？

主要发现

• 在14项评估设置（数据集、任务、源/目标域）中的13项中，当模型在另一台扫描仪的数据上测试时，性能均出现下降。
• 平均而言，MRI表现出最严重的性能下降，其次是X射线，CT的下降幅度最小，这归因于CT采集协议更高的标准化程度。
• MRI的性能下降最为显著，主要由于不同扫描仪制造商之间在回波时间（TE）、重复时间（TR）和翻转角等参数上存在高度变异性。
• 将目标域数据注入训练集可提升泛化能力，但其效果因模态和任务而异，MRI在使用此类数据时受益最大。
• 向训练数据中添加高斯噪声并未提升对域外扫描仪数据的泛化性能，表明简单数据增强不足以缓解领域偏移。
• 扫描仪特有的后处理和采集参数会显著改变图像特征，即使图像差异在视觉上细微，也会导致模型性能下降。

Figure 2: Visualization of the change in network performance due to the test scanner domain shifting away from the training scanner domain (see Table 2 for reference). Each pair of bars is for a single modality and training set scanner manufacturer.