[Paper Review] Polyp-SAM: Transfer SAM for Polyp Segmentation

Yuheng Li, Mingzhe Hu+1 more|arXiv (Cornell University)|Apr 29, 2023

Colorectal Cancer Screening and Detection15 citations

TL;DR

Polyp-SAM finetunes the Segment Anything Model (SAM) for colon polyp segmentation and compares two transfer strategies, achieving state-of-the-art results on several datasets with Dice scores above 88%.

ABSTRACT

Colon polyps are considered important precursors for colorectal cancer. Automatic segmentation of colon polyps can significantly reduce the misdiagnosis of colon cancer and improve physician annotation efficiency. While many methods have been proposed for polyp segmentation, training large-scale segmentation networks with limited colonoscopy data remains a challenge. Recently, the Segment Anything Model (SAM) has recently gained much attention in both natural and medical image segmentation. SAM demonstrates superior performance in several image benchmarks and therefore shows great potential for medical image segmentation. In this study, we propose Poly-SAM, a finetuned SAM model for polyp segmentation, and compare its performance to several state-of-the-art polyp segmentation models. We also compare two transfer learning strategies of SAM with and without finetuning its encoders. Evaluated on five public datasets, our Polyp-SAM achieves state-of-the-art performance on two datasets and impressive performance on three datasets, with dice scores all above 88%. This study demonstrates the great potential of adapting SAM to medical image segmentation tasks. We plan to release the code and model weights for this paper at: https://github.com/ricklisz/Polyp-SAM.

Motivation & Objective

• Motivate automatic polyp segmentation to reduce misdiagnosis and improve annotation efficiency in colonoscopy.
• Leverage the Segment Anything Model (SAM) for medical image segmentation by finetuning on polyp data.
• Compare transfer learning strategies for SAM with and without encoder finetuning.
• Evaluate Polyp-SAM across multiple public polyp datasets to assess generalization.

Proposed method

• Finite-tuning of SAM for polyp segmentation (Polyp-SAM).
• Comparison of two transfer learning strategies: SAM with and without encoder finetuning.
• Evaluation on five public polyp datasets with Dice score as primary metric.
• Comparison against state-of-the-art polyp segmentation models.

Experimental results

Research questions

• RQ1 Can SAM be effectively adapted to polyp segmentation through finetuning?
• RQ2 Which transfer strategy (finetuning encoders vs. fixed encoders) yields better polyp segmentation performance?
• RQ3 How does Polyp-SAM perform across diverse public datasets compared to existing methods?

Key findings

• Polyp-SAM achieves state-of-the-art performance on two datasets.
• Polyp-SAM shows impressive performance on three datasets.
• Dice scores on evaluated datasets exceed 88%.