[Paper Review] MSA$^2$Net: Multi-scale Adaptive Attention-guided Network for Medical Image Segmentation

Sina Ghorbani Kolahi, S. Kamal Chaharsooghi+6 more|arXiv (Cornell University)|Jul 31, 2024

Brain Tumor Detection and Classification5 citations

TL;DR

MSA2Net introduces MASAG, a multi-scale adaptive spatial attention gate within a hybrid CNN-Transformer framework to fuse encoder-decoder features for precise medical image segmentation, achieving state-of-the-art results on Synapse and ISIC2018 datasets.

ABSTRACT

Medical image segmentation involves identifying and separating object instances in a medical image to delineate various tissues and structures, a task complicated by the significant variations in size, shape, and density of these features. Convolutional neural networks (CNNs) have traditionally been used for this task but have limitations in capturing long-range dependencies. Transformers, equipped with self-attention mechanisms, aim to address this problem. However, in medical image segmentation it is beneficial to merge both local and global features to effectively integrate feature maps across various scales, capturing both detailed features and broader semantic elements for dealing with variations in structures. In this paper, we introduce MSA$^2$Net, a new deep segmentation framework featuring an expedient design of skip-connections. These connections facilitate feature fusion by dynamically weighting and combining coarse-grained encoder features with fine-grained decoder feature maps. Specifically, we propose a Multi-Scale Adaptive Spatial Attention Gate (MASAG), which dynamically adjusts the receptive field (Local and Global contextual information) to ensure that spatially relevant features are selectively highlighted while minimizing background distractions. Extensive evaluations involving dermatology, and radiological datasets demonstrate that our MSA$^2$Net outperforms state-of-the-art (SOTA) works or matches their performance. The source code is publicly available at https://github.com/xmindflow/MSA-2Net.

Motivation & Objective

• Address variabilities in organ size, shape, and density in medical images.
• Merge local and global features via adaptive skip connections between encoder and decoder.
• Develop a module (MASAG) to dynamically recalibrate receptive fields and highlight spatially relevant features.
• Validate the approach on multi-organ Synapse and ISIC2018 datasets with boundary-aware loss.

Proposed method

• Proposes MASAG (Multi-Scale Adaptive Spatial Attention Gate) to fuse encoder and decoder features with dynamic receptive fields.
• Uses a hybrid decoder where shallow layers employ Large Kernel Attention (LKA) and deeper layers use Dual Attention Enhanced Transformer (DAE-Former) blocks.
• Implements multi-scale feature fusion combining Local Context Extraction and Global Context Extraction.
• Incorporates spatial selection, spatial interaction, cross-modulation, and recalibration to refine feature maps for accurate segmentation.
• Employs MaxViT-based encoder with pretrained weights and boundary-aware BDoU loss for boundary delineation.
• Evaluates on Synapse (multi-organ CT) and ISIC2018 (skin lesion) datasets.

Figure 1: Our proposed segmentation network, called MSA ${}^{\text{2}}$ Net, is composed of an encoder (using pretrained MaxViT block) and a decoder (comprising DAE-Former blocks in shallow layers and LKA blocks in deeper ones). The encoding-decoding feature fusion is performed via our novel MASAG m

Experimental results

Research questions

• RQ1 Can MASAG dynamically recalibrate receptive fields to improve segmentation across objects of varying scales?
• RQ2 Does the hybrid encoder-decoder with MASAG outperform state-of-the-art CNN-Transformer models in medical image segmentation?
• RQ3 How do local and global context fusion, along with adaptive skip connections, affect boundary precision and overall DSC/HD95 metrics?
• RQ4 Is the approach robust across radiological and dermatological imaging modalities?

Key findings

• MSA2Net achieves DSC of 84.75 and HD95 of 13.29 on the Synapse dataset, surpassing several SOTA baselines.
• On ISIC2018, MSA2Net reaches DSC of 0.9129, SE 0.8840, SP 0.9557, ACC 0.9640, outperforming multiple prior methods.
• Ablation studies show that including MASAG, LKA, and DAE-Former in various configurations yields progressive gains, with all three components together producing the best Dice/HD95 trade-off.
• MSA2Net demonstrates notable improvements in pancreas and aorta segmentation on Synapse, illustrating the benefit of dynamic receptive field recalibration for both small and large organs.

Figure 2: A comparative visual examination of the proposed approach in contrast to different methods employing the Synapse multi-organ segmentation dataset.