[论文解读] A Recent Survey of Vision Transformers for Medical Image Segmentation

Asifullah Khan, Zunaira Rauf+14 more|arXiv (Cornell University)|Dec 1, 2023

Advanced Neural Network Applications被引用 11

一句话总结

对 Vision Transformers (ViTs) 和 Hybrid Vision Transformers (HVTs) 在 medical image segmentation 领域的综述，详细介绍优点、局限性、架构，以及实时模态应用。

ABSTRACT

Medical image segmentation plays a crucial role in various healthcare applications, enabling accurate diagnosis, treatment planning, and disease monitoring. Traditionally, convolutional neural networks (CNNs) dominated this domain, excelling at local feature extraction. However, their limitations in capturing long-range dependencies across image regions pose challenges for segmenting complex, interconnected structures often encountered in medical data. In recent years, Vision Transformers (ViTs) have emerged as a promising technique for addressing the challenges in medical image segmentation. Their multi-scale attention mechanism enables effective modeling of long-range dependencies between distant structures, crucial for segmenting organs or lesions spanning the image. Additionally, ViTs' ability to discern subtle pattern heterogeneity allows for the precise delineation of intricate boundaries and edges, a critical aspect of accurate medical image segmentation. However, they do lack image-related inductive bias and translational invariance, potentially impacting their performance. Recently, researchers have come up with various ViT-based approaches that incorporate CNNs in their architectures, known as Hybrid Vision Transformers (HVTs) to capture local correlation in addition to the global information in the images. This survey paper provides a detailed review of the recent advancements in ViTs and HVTs for medical image segmentation. Along with the categorization of ViT and HVT-based medical image segmentation approaches, we also present a detailed overview of their real-time applications in several medical image modalities. This survey may serve as a valuable resource for researchers, healthcare practitioners, and students in understanding the state-of-the-art approaches for ViT-based medical image segmentation.

研究动机与目标

• 解释为什么 Vision Transformers 对 medical image segmentation 相关，以及它们在建模长程依赖关系方面的能力。
• 对最近的 ViT- 与 HVT-based 分割方法进行分类，并比较它们的特征。
• 讨论 ViT 基础的医学分割在实时可用性与模态特异性考量方面的内容。

提出的方法

• 评述并对最近的 ViT-based 和 HVT-based 医学图像分割方法进行分类。
• 分析 ViTs 的优点与局限性，包括长程依赖建模和缺乏归纳偏置。
• 突出结合 CNN 与 ViT 组件以捕捉局部和全局信息的混合架构。
• 提供跨多种医学影像模态的实时应用概述。

实验结果

研究问题

• RQ1 用于医学图像分割的主要 ViT-based 策略有哪些？
• RQ2 Hybrid Vision Transformers 如何在局部特征提取与全局注意力之间取得平衡？
• RQ3 对于 ViT-based 分割，实时应用的考量与模态特定挑战有哪些？
• RQ4 ViTs 在医学影像中的主要局限性及潜在缓解方法有哪些？

主要发现

• ViTs 使对大型或互连结构的分割中的长程依赖建模变得有效。
• Hybrid Vision Transformers 将 CNN 与 ViT 组件结合，以同时捕捉局部相关性和全局信息。
• 该综述对最近的 ViT/HVT 方法进行了分类，并将其映射到特定的医学影像模态。
• 在跨多种医学图像模态的实时应用背景下讨论了基于 ViT 的方法。