[论文解读] I-MedSAM: Implicit Medical Image Segmentation with Segment Anything

Xiaobao Wei, Jiajun Cao+4 more|arXiv (Cornell University)|Nov 28, 2023

Radiomics and Machine Learning in Medical Imaging被引用 4

一句话总结

I-MedSAM 提出了一种新颖的隐式医学图像分割框架，通过将分割一切模型（SAM）与隐式神经表示（INRs）相结合，实现了更好的泛化能力和边界精度。通过在粗到细的 INR 解码器中集成频率感知适配器和不确定性引导采样，I-MedSAM 仅使用 1.6M 可训练参数即在 2D 医学图像分割基准上取得了最先进性能，优于所有离散与连续方法。

ABSTRACT

With the development of Deep Neural Networks (DNNs), many efforts have been made to handle medical image segmentation. Traditional methods such as nnUNet train specific segmentation models on the individual datasets. Plenty of recent methods have been proposed to adapt the foundational Segment Anything Model (SAM) to medical image segmentation. However, they still focus on discrete representations to generate pixel-wise predictions, which are spatially inflexible and scale poorly to higher resolution. In contrast, implicit methods learn continuous representations for segmentation, which is crucial for medical image segmentation. In this paper, we propose I-MedSAM, which leverages the benefits of both continuous representations and SAM, to obtain better cross-domain ability and accurate boundary delineation. Since medical image segmentation needs to predict detailed segmentation boundaries, we designed a novel adapter to enhance the SAM features with high-frequency information during Parameter-Efficient Fine-Tuning (PEFT). To convert the SAM features and coordinates into continuous segmentation output, we utilize Implicit Neural Representation (INR) to learn an implicit segmentation decoder. We also propose an uncertainty-guided sampling strategy for efficient learning of INR. Extensive evaluations on 2D medical image segmentation tasks have shown that our proposed method with only 1.6M trainable parameters outperforms existing methods including discrete and implicit methods. The code will be available at: https://github.com/ucwxb/I-MedSAM.

研究动机与目标

• 为解决医学图像分割中离散表示的局限性，如空间灵活性差和边界划分不清晰。
• 通过连续表示提升医学图像分割的跨域泛化能力与分辨率可扩展性。
• 通过新型适配器将频域信息融入 SAM 特征，提升 SAM 在医学图像上的性能。
• 通过粗到细的 INR 解码器结合不确定性引导采样，优化隐式神经表示学习，实现高效且精确的推理。

提出的方法

• I-MedSAM 使用频率适配器在参数高效微调（PEFT）过程中将 DFT 谱的高频信息注入 SAM 特征，提升边界敏感性。
• 该模型采用两阶段隐式神经表示（INR）解码器：粗 INR 预测初始分割，随后在选定点上进行精细 INR 优化。
• 不确定性引导采样从粗 INR 输出中选择 Top-K 方差特征点，聚焦于模糊或复杂区域以优化细化过程。
• INR 解码器将图像特征与坐标网格映射为连续分割输出，实现分辨率无关的推理。
• 整个框架端到端训练，仅包含 1.6M 可训练参数，利用 LoRA 实现 SAM 的高效微调。
• 该方法整合空间与频域特征，增强表示能力，同时在领域分布变化下保持泛化性能。

Figure 2 : The overall pipeline of I-MedSAM. First, given the medical images and a coarse bounding box as a prompt, I-MedSAM utilizes the medical image encoder and the prompt encoder to generate discrete features. For the medical image encoder, we design low-rank adapters and frequency adapters to e

实验结果

研究问题

• RQ1 隐式神经表示能否提升医学图像分割模型的泛化能力与分辨率可扩展性？
• RQ2 将频域信息融入 SAM 特征对分割边界精度有何影响？
• RQ3 不确定性引导采样能否提升医学图像中隐式分割的效率与精度？
• RQ4 具有选择性采样的粗到细 INR 解码器是否优于标准 INR 或离散预测头？
• RQ5 轻量级、参数高效的适配器在多样化医学成像领域中能在多大程度上提升 SAM 的性能？

主要发现

• I-MedSAM 在 BCV 数据集上取得 86.28% 的 Dice 分数，在 AMOS 数据集上取得 79.86% 的 Dice 分数，超越最先进离散与连续方法。
• 消融实验表明，LoRA 秩为 4 时性能最优且参数最少，证实了 PEFT 策略的高效性。
• 使用幅度谱时，频率适配器使性能提升 0.61% Dice（86.80% 对比无适配器的 86.19%），表明幅度比相位更为关键。
• 采用 Top-12.5% 特征点的不确定性引导采样达到 86.80% Dice，优于全量采样，证明选择性采样兼具高效与有效性。
• I-MedSAM 在领域分布偏移下表现出稳健泛化能力，如在 Kvasir-Sessile 和 BCV 数据集上的定性对比显示边界划分更优。
• 该模型仅使用 1.6M 可训练参数即保持高性能，显著少于基线方法如 nnUNet（126.6M）和 SAMed（3.9M）。

Figure 3 : The illustration of the proposed frequency adapters and LoRA in the image encoder. With these adapters, I-MedSAM can be trained with minorities of parameters while obtaining high-quality segmentation boundaries.