[论文解读] Computer-Vision Benchmark Segment-Anything Model (SAM) in Medical Images: Accuracy in 12 Datasets

Sheng He, Rina Bao+5 more|arXiv (Cornell University)|Apr 18, 2023

Radiomics and Machine Learning in Medical Imaging被引用 65

一句话总结

本研究在12个公开医学图像分割数据集上对零-shot SAM进行评估，发现SAM的性能不及5个数据集特定的医学分割模型，性能受维度、目标大小、对比度等因素影响。

ABSTRACT

Background: The segment-anything model (SAM), introduced in April 2023, shows promise as a benchmark model and a universal solution to segment various natural images. It comes without previously-required re-training or fine-tuning specific to each new dataset. Purpose: To test SAM's accuracy in various medical image segmentation tasks and investigate potential factors that may affect its accuracy in medical images. Methods: SAM was tested on 12 public medical image segmentation datasets involving 7,451 subjects. The accuracy was measured by the Dice overlap between the algorithm-segmented and ground-truth masks. SAM was compared with five state-of-the-art algorithms specifically designed for medical image segmentation tasks. Associations of SAM's accuracy with six factors were computed, independently and jointly, including segmentation difficulties as measured by segmentation ability score and by Dice overlap in U-Net, image dimension, size of the target region, image modality, and contrast. Results: The Dice overlaps from SAM were significantly lower than the five medical-image-based algorithms in all 12 medical image segmentation datasets, by a margin of 0.1-0.5 and even 0.6-0.7 Dice. SAM-Semantic was significantly associated with medical image segmentation difficulty and the image modality, and SAM-Point and SAM-Box were significantly associated with image segmentation difficulty, image dimension, target region size, and target-vs-background contrast. All these 3 variations of SAM were more accurate in 2D medical images, larger target region sizes, easier cases with a higher Segmentation Ability score and higher U-Net Dice, and higher foreground-background contrast.

研究动机与目标

• 评估 Segment Anything Model (SAM) 在12个公开医学图像分割数据集上的零-shot准确性。
• 将SAM与最新、数据集特定的医学分割算法进行比较。
• 研究影响SAM在医学图像分割准确性的因素（维度、目标区域大小、对比度、模态等）。
• 分析哪种提示模式（SAM-Semantic、SAM-Point、SAM-Box）在医学图像中取得更好结果。

提出的方法

• 在不对任何医学数据集进行再训练或微调的情况下，将SAM应用于三种提示模式（SAM-Semantic、SAM-Point、SAM-Box）。
• 使用 Dice 重叠作为准确性指标，在涵盖10个器官和6种成像模态的12个公开数据集上评估SAM。
• 将SAM变体与五种最先进的医学图像分割模型（U-Net、U-Net++、Attention U-Net、Trans U-Net、UCTransNet）在各自数据集上进行比较。
• 将3D图像视为一系列2D切片进行分割，并将切片结果拼接以获得个体级 Dice 分数。
• 使用单因素和多因素分析计算SAM准确性与六个潜在因素（Segmentation Ability分、U-Net Dice、图像维度、目标区域大小、模态和对比度）的关联。
• 使用广义线性模型(GLM)评估六个因素对SAM Dice分数的联合影响。

实验结果

研究问题

• RQ1 零-shot SAM在12个医学图像分割数据集上的表现如何与专用医学分割模型相比？
• RQ2 哪些SAM提示模式（语义、点、框）在医学图像中取得更好准确性？
• RQ3 哪些因素（难度、维度、目标大小、对比度、模态）显著影响SAM在医学图像中的分割准确性？
• RQ4 多因素模型能否解释SAM在不同数据集上的Dice性能？

主要发现

• SAM在所有12个数据集中均不如五种医学图像特定算法，Dice差距范围为0.1–0.5，在某些情况下甚至达0.6–0.7。
• SAM-Semantic、SAM-Point和SAM-Box表现各不相同；总体上都不如基于U-Net的方法，尤其在3D和小型或低对比度区域。
• SAM的Dice与分割难度（由U-Net Dice衡量）相关；在二维图像和较大目标区域的情况下表现更好，并且前景-背景对比度越高越好。
• 二维图像（ Dermoscopy、Colonoscopy、X-ray）和较大目标区域可获得更好SAM性能；3D图像和小型、低对比度目标存在挑战。
• 联合GLM分析证实这六个因素共同对SAM Dice分数具有显著预测力（p < 2.2e-16）。
• 研究建议通过在医学数据上微调或开发医学影像特定基准模型来调整SAM以适应医学成像。