[논문 리뷰] A Recent Survey of Vision Transformers for Medical Image Segmentation

Asifullah Khan, Zunaira Rauf외 14명|arXiv (Cornell University)|2023. 12. 01.

Advanced Neural Network Applications인용 수 11

한 줄 요약

의료 영상 분할을 위한 Vision Transformers (ViTs)와 Hybrid Vision Transformers (HVTs)에 대한 고찰로, 장점, 한계, 아키텍처 및 실시간 모달리티 응용을 자세히 다룬다.

ABSTRACT

Medical image segmentation plays a crucial role in various healthcare applications, enabling accurate diagnosis, treatment planning, and disease monitoring. Traditionally, convolutional neural networks (CNNs) dominated this domain, excelling at local feature extraction. However, their limitations in capturing long-range dependencies across image regions pose challenges for segmenting complex, interconnected structures often encountered in medical data. In recent years, Vision Transformers (ViTs) have emerged as a promising technique for addressing the challenges in medical image segmentation. Their multi-scale attention mechanism enables effective modeling of long-range dependencies between distant structures, crucial for segmenting organs or lesions spanning the image. Additionally, ViTs' ability to discern subtle pattern heterogeneity allows for the precise delineation of intricate boundaries and edges, a critical aspect of accurate medical image segmentation. However, they do lack image-related inductive bias and translational invariance, potentially impacting their performance. Recently, researchers have come up with various ViT-based approaches that incorporate CNNs in their architectures, known as Hybrid Vision Transformers (HVTs) to capture local correlation in addition to the global information in the images. This survey paper provides a detailed review of the recent advancements in ViTs and HVTs for medical image segmentation. Along with the categorization of ViT and HVT-based medical image segmentation approaches, we also present a detailed overview of their real-time applications in several medical image modalities. This survey may serve as a valuable resource for researchers, healthcare practitioners, and students in understanding the state-of-the-art approaches for ViT-based medical image segmentation.

연구 동기 및 목표

• Why Vision Transformers are relevant for medical image segmentation and their ability to model long-range dependencies.
• Categorize recent ViT- and HVT-based segmentation approaches and compare their characteristics.
• Discuss real-time applicability and modality-specific considerations in ViT-based medical segmentation.

제안 방법

• Review and categorize recent ViT-based and HVT-based medical image segmentation approaches.
• Analyze the strengths and limitations of ViTs, including long-range dependency modeling and lack of inductive bias.
• Highlight hybrid architectures that combine CNNs with ViT components to capture local and global information.
• Provide an overview of real-time applications across multiple medical imaging modalities.

실험 결과

연구 질문

• RQ1 What are the main ViT-based strategies used for medical image segmentation?
• RQ2 How do Hybrid Vision Transformers address local feature extraction versus global attention?
• RQ3 What are the real-time application considerations and modality-specific challenges for ViT-based segmentation?
• RQ4 What are the primary limitations of ViTs in medical imaging and potential mitigation approaches?

주요 결과

• ViTs enable effective long-range dependency modeling for segmentation of large or interconnected structures.
• Hybrid Vision Transformers combine CNNs with ViT components to capture local correlations alongside global information.
• The survey categorizes recent ViT/HVT approaches and maps them to specific medical imaging modalities.
• ViT-based methods are discussed in the context of real-time applications across multiple medical image modalities.