[논문 리뷰] SAM.MD: Zero-shot medical image segmentation capabilities of the Segment Anything Model

Saikat Roy, Tassilo Wald외 6명|arXiv (Cornell University)|2023. 04. 10.

Artificial Intelligence in Healthcare and Education인용 수 64

한 줄 요약

이 논문은 샘(SAM)의 제로샷 의료 영상 분할 성능을 망측 CT 데이터에서 포인트 및 바운딩 박스 프롬프트를 사용해 평가하고, nnU-Net 베이스라인과 비교하며, 인터랙티브 반자동 분할에 대한 가능성을 논의합니다.

ABSTRACT

Foundation models have taken over natural language processing and image generation domains due to the flexibility of prompting. With the recent introduction of the Segment Anything Model (SAM), this prompt-driven paradigm has entered image segmentation with a hitherto unexplored abundance of capabilities. The purpose of this paper is to conduct an initial evaluation of the out-of-the-box zero-shot capabilities of SAM for medical image segmentation, by evaluating its performance on an abdominal CT organ segmentation task, via point or bounding box based prompting. We show that SAM generalizes well to CT data, making it a potential catalyst for the advancement of semi-automatic segmentation tools for clinicians. We believe that this foundation model, while not reaching state-of-the-art segmentation performance in our investigations, can serve as a highly potent starting point for further adaptations of such models to the intricacies of the medical domain. Keywords: medical image segmentation, SAM, foundation models, zero-shot learning

연구 동기 및 목표

• Segment Anything Model (SAM)의 의료 CT 데이터에 대한 기본 제공 제로샷 분할 능력 평가.
• 다양한 시각적 프롬프트(포인트 및 바운딩 박스)가 SAM의 분할 정확도에 미치는 영향 조사.
• SAM의 제로샷 결과를 다기관 복부 CT 데이터셋에서 강력한 자동 기반선(nnU-Net)과 비교.
• CT 강도 범위에 따른 바운딩 박스 프롬프팅의 강건성 확인.
• 임상 분할 워크플로우에서 SAM의 인터랙티브 적용에 대한 가이드 제공.

제안 방법

• AMOS22 복부 CT 장기 분할 데이터셋의 축 방향 2D 단면을 평가 데이터로 사용.
• 평가 프롬프트를 생성: (i) 임의 포인트 프롬프트(세분화 마스크당 1, 3, 10개) 및 (ii) 마스크 주변의 jittered 바운딩 박스(0.01–0.5).
• Ground-truth 마스크에 대한 Dice Similarity Coefficient (DSC)로 분할 정확도 계산.
• 동일한 슬라이스에서 SAM 프롬프트를 nnU-Net 2D 및 3D 베이스라인과 비교."],
• research_questions:["Can SAM perform zero-shot segmentation of unseen abdominal organs in CT images using simple prompts?", "How do point-based prompts compare to bounding box prompts in achieving accurate segmentations?", "Is bounding box prompting robust to intensity variations in CT data?", "How does SAM zero-shot performance compare to nnU-Net baselines?", "Can SAM accelerate interactive semi-automatic segmentation workflows in practice?"]
• research_questions_translation_placeholder:[]
• key_findings:["Box prompting (even with moderate jitter) yields high DSCs and is competitive with baselines.", "Single positive bounding boxes outperform multiple point prompts (e.g., 1 point vs 10 points).", "Performance remains robust on raw CT value ranges (AVG* similar to bounded prompts).", "SAM demonstrates strong zero-shot segmentation potential when paired with expert prompts in interactive workflows.", "While not reaching state-of-the-art fully automatic methods, SAM can speed up semi-automatic clinician workflows on most structures."]
• table_headers:["Method","Organs","AVG","AVG*","Spl.","R.Kid.","L.Kid.","GallBl.","Esoph.","Liver","Stom.","Aorta","Postc.","Pancr.","R.AG.","L.AG.","Duod.","Blad."]
• table_rows:[["1 Point","",0.632,0.759,0.770,0.616,0.382,0.577,0.508,0.720,0.453,0.317,0.085,0.196,0.339,0.542,0.493,0.347],["3 Points","",0.733,0.784,0.786,0.683,0.448,0.658,0.577,0.758,0.493,0.343,0.129,0.240,0.325,0.631,0.542,0.397],["10 Points","",0.857,0.855,0.857,0.800,0.643,0.811,0.759,0.842,0.637,0.538,0.405,0.516,0.480,0.789,0.699,0.560],["Boxes, 0.01","",0.926,0.884,0.889,0.883,0.820,0.902,0.823,0.924,0.867,0.727,0.618,0.754,0.811,0.909,0.838,0.826],["Boxes, 0.05","",0.920,0.883,0.894,0.879,0.814,0.883,0.818,0.923,0.862,0.727,0.609,0.746,0.805,0.907,0.834,0.819],["Boxes, 0.1","",0.890,0.870,0.874,0.859,0.806,0.813,0.796,0.919,0.845,0.702,0.594,0.733,0.785,0.862,0.810,0.795],["Boxes, 0.25","",0.553,0.601,0.618,0.667,0.656,0.490,0.561,0.747,0.687,0.481,0.478,0.558,0.655,0.561,0.594,0.612],["Boxes, 0.5","",0.202,0.275,0.257,0.347,0.356,0.164,0.252,0.381,0.335,0.239,0.234,0.308,0.343,0.205,0.278,0.289]]} respect_body_placeholder:0} {
• } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } }
• }]]}]]}]]}]} } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } }

실험 결과

주요 결과

Method	Organs	AVG	AVG*	Spl.	R.Kid.	L.Kid.	GallBl.	Esoph.	Liver	Stom.	Aorta	Postc.	Pancr.	R.AG.	L.AG.	Duod.	Blad.
1 Point		0.632	0.759	0.770	0.616	0.382	0.577	0.508	0.720	0.453	0.317	0.085	0.196	0.339	0.542	0.493	0.347
3 Points		0.733	0.784	0.786	0.683	0.448	0.658	0.577	0.758	0.493	0.343	0.129	0.240	0.325	0.631	0.542	0.397
10 Points		0.857	0.855	0.857	0.800	0.643	0.811	0.759	0.842	0.637	0.538	0.405	0.516	0.480	0.789	0.699	0.560
Boxes, 0.01		0.926	0.884	0.889	0.883	0.820	0.902	0.823	0.924	0.867	0.727	0.618	0.754	0.811	0.909	0.838	0.826
Boxes, 0.05		0.920	0.883	0.894	0.879	0.814	0.883	0.818	0.923	0.862	0.727	0.609	0.746	0.805	0.907	0.834	0.819
Boxes, 0.1		0.890	0.870	0.874	0.859	0.806	0.813	0.796	0.919	0.845	0.702	0.594	0.733	0.785	0.862	0.810	0.795
Boxes, 0.25		0.553	0.601	0.618	0.667	0.656	0.490	0.561	0.747	0.687	0.481	0.478	0.558	0.655	0.561	0.594	0.612
Boxes, 0.5		0.202	0.275	0.257	0.347	0.356	0.164	0.252	0.381	0.335	0.239	0.234	0.308	0.343	0.205	0.278	0.289

• Box 프롬프팅은(중간 수준의 지터링에도) 높은 DSC를 낳고 베이스라인과 경쟁적입니다.
• 단일 양성 바운딩 박스가 다중 포인트 프롬프팅보다 더 우수합니다(예: 1 포인트 대 10 포인트).
• 성능은 원시 CT 값 범위에서도 강건하게 유지됩니다(AVG*가 경계 프로프트와 유사).
• 전문가 프롬프트와 결합된 인터랙티브 워크플로우에서 SAM은 강력한 제로샷 분할 가능성을 보여줍니다.
• 최첨단 Fully Automatic 방법에 비해 다소 미치지 못하지만, 대부분의 구조물에서 반자동 임상의 워크플로우를 가속화할 수 있습니다.