[论文解读] A Comprehensive Survey of Mamba Architectures for Medical Image Analysis: Classification, Segmentation, Restoration and Beyond

Shubhi Bansal, A Sreeharish+7 more|arXiv (Cornell University)|Oct 3, 2024

AI in cancer detection被引用 9

一句话总结

对用于医学影像分析的 Mamba 状态空间模型的综合综述，详细介绍架构、优化、扫描机制、多模态能力，以及在分割、分类和重建中的应用。

ABSTRACT

Mamba, a special case of the State Space Model, is gaining popularity as an alternative to template-based deep learning approaches in medical image analysis. While transformers are powerful architectures, they have drawbacks, including quadratic computational complexity and an inability to address long-range dependencies efficiently. This limitation affects the analysis of large and complex datasets in medical imaging, where there are many spatial and temporal relationships. In contrast, Mamba offers benefits that make it well-suited for medical image analysis. It has linear time complexity, which is a significant improvement over transformers. Mamba processes longer sequences without attention mechanisms, enabling faster inference and requiring less memory. Mamba also demonstrates strong performance in merging multimodal data, improving diagnosis accuracy and patient outcomes. The organization of this paper allows readers to appreciate the capabilities of Mamba in medical imaging step by step. We begin by defining core concepts of SSMs and models, including S4, S5, and S6, followed by an exploration of Mamba architectures such as pure Mamba, U-Net variants, and hybrid models with convolutional neural networks, transformers, and Graph Neural Networks. We also cover Mamba optimizations, techniques and adaptations, scanning, datasets, applications, experimental results, and conclude with its challenges and future directions in medical imaging. This review aims to demonstrate the transformative potential of Mamba in overcoming existing barriers within medical imaging while paving the way for innovative advancements in the field. A comprehensive list of Mamba architectures applied in the medical field, reviewed in this work, is available at Github.

研究动机与目标

• 解释 Mamba 与结构化状态空间序列模型（SSMs）及其与医学影像的相关性。
• 编目并分析用于医学影像任务的纯 Mamba、U-Net 变体及混合架构。
• 讨论在医疗保健领域中对 Mamba 的优化、训练策略及多模态/自适应技术。
• 总结在分割、分类和重建等方面的数据集、实验和应用。
• 突出在医学影像中 Mamba 的局限性与未来发展方向。

提出的方法

• 定义核心的 SSM 概念（A、B、C、D 矩阵；离散化）及关键变体（S4、S5、S6）。
• 描述 Mamba 架构组件，包括选择性扫描和面向硬件优化的实现。
• 调查纯 Mamba、ViM、VMamba、Plain Mamba、VSS/SS2D 块，以及大量基于 Mamba 的 UNet 变体。
• 讨论扫描机制、优化（轻量/高效设计）及学习范式（弱监督/半监督/自监督、多模态）。
• 对架构家族进行分类（纯 Mamba、UNet 变体、混合模型），并总结在分割、分类、重建和配准等方面的实验发现。

Figure 1 . Evolution of Mamba from State Space Models (SSMs)

实验结果

研究问题

• RQ1 支撑 Mamba 的核心状态空间模型概念是什么，以及在可扩展性和处理长程依赖方面与 Transformers 相比如何？
• RQ2 Mamba 架构已如何被改编（纯、UNet 变体、混合）以用于分割、分类和重建等医学影像分析任务？
• RQ3 哪些优化和训练策略使 Mamba 在医学影像场景中高效？
• RQ4 哪些数据集和实验结果展示了 Mamba 在医学影像领域的有效性？
• RQ5 为医疗保健领域中的 Mamba 指出的局限性与未来方向是什么？

主要发现

• Mamba 将二次注意力的复杂度转化为线性时间，使在不使用注意力机制的情况下也能处理更长的序列。
• 提出了广泛的架构（纯 Mamba、ViM、VMamba、VSS/SS2D 块）及 UNet 混合模型，用于医学影像任务。
• 将CNN、视觉变换器和 Mamba 块结合的混合模型显示出在内存占用降低的同时具有竞争力的性能。
• 扫描、选择性机制以及高阶 SS2D 变体有助于高效建模二维/三维医学图像。
• 对多样的数据集和应用进行了综述，并讨论了在医学环境中的局限性与未来方向。

Figure 3 . Publication Trend of Mamba Research in the Medical Domain