[论文解读] BEFUnet: A Hybrid CNN-Transformer Architecture for Precise Medical Image Segmentation

Omid Nejati Manzari, Javad Mirzapour Kaleybar+2 more|arXiv (Cornell University)|Feb 13, 2024

Brain Tumor Detection and Classification被引用 9

一句话总结

BEFUnet 将边缘聚焦的 CNN 编码与主体聚焦的 Swin Transformer 编码通过 Local Cross-Attention Fusion 和 Double-Level Fusion 模块融合，以在多个数据集上提升医疗图像分割性能。

ABSTRACT

The accurate segmentation of medical images is critical for various healthcare applications. Convolutional neural networks (CNNs), especially Fully Convolutional Networks (FCNs) like U-Net, have shown remarkable success in medical image segmentation tasks. However, they have limitations in capturing global context and long-range relations, especially for objects with significant variations in shape, scale, and texture. While transformers have achieved state-of-the-art results in natural language processing and image recognition, they face challenges in medical image segmentation due to image locality and translational invariance issues. To address these challenges, this paper proposes an innovative U-shaped network called BEFUnet, which enhances the fusion of body and edge information for precise medical image segmentation. The BEFUnet comprises three main modules, including a novel Local Cross-Attention Feature (LCAF) fusion module, a novel Double-Level Fusion (DLF) module, and dual-branch encoder. The dual-branch encoder consists of an edge encoder and a body encoder. The edge encoder employs PDC blocks for effective edge information extraction, while the body encoder uses the Swin Transformer to capture semantic information with global attention. The LCAF module efficiently fuses edge and body features by selectively performing local cross-attention on features that are spatially close between the two modalities. This local approach significantly reduces computational complexity compared to global cross-attention while ensuring accurate feature matching. BEFUnet demonstrates superior performance over existing methods across various evaluation metrics on medical image segmentation datasets.

研究动机与目标

• 动机在于阐明 CNN 和纯 Transformer 在医学图像分割中的局限性，特别是在边缘信息和全局上下文方面，并提出相应解决思路。
• 提出双分支编码器，以同时捕捉边缘与主体特征。
• 引入局部跨注意力融合（LCAF），以高效地融合跨模态特征。
• 开发双层融合（DLF）模块，在尺度之间融合粗粒度与细粒度特征。
• 在多种医学图像数据集上展示出色的分割性能。

提出的方法

• 提出 BEFUnet，具备双分支编码器：基于 CNN 的边缘编码器使用 Pixel Different Convolution (PDC) 块，和一个 Swin Transformer 的主体编码器。
• 引入局部跨注意力融合（LCAF），利用局部跨注意力融合边缘与主体特征，与全局跨注意力相比可降低计算量。
• 实现双层融合（DLF）模块，通过跨注意力和基于类别令牌的交互融合最小尺度和最大尺度的特征。
• 使用两个损失分量对边缘和主体进行监督：边缘监督损失和主体监督损失（二元交叉熵 + Dice 损失）。
• 使用组合的最终损失 L = L_body + gamma * L_edge 来同时优化两个分支。

Figure 1 : The structure of our proposed BEFUnet. BEFUnet consists of a dual-encoder that simultaneously extracts edge and body features. Moreover, it incorporates an efficient fusion module called LCAF, which facilitates the merging of edge and body features. Additionally, there is a DLF module int

实验结果

研究问题

• RQ1 将边缘和主体语义分开建模的双分支编码器，是否能在分割精度上优于单分支 CNN 或 Transformer 模型？
• RQ2 局部跨注意力融合是否在保持精度的同时高效融合跨模态特征并减少计算量？
• RQ3 双层融合模块是否能有效整合多尺度特征以提升分割性能？
• RQ4 与最先进方法相比，BEFUnet 在多个不同的医学成像数据集上的性能提升如何？

主要发现

• BEFUnet 在 Synapse 上的分割性能优越，DSC 80.47，HD 16.26，超越了若干 SOTA 方法。
• 在 SegPC 数据集上，BEFUnet 达到 95.1% 的准确率和 0.871 的 F1 分数，超越多种基线。
• 在 ISIC 2017 皮肤病变分割中，BEFUnet 取得最佳 Dice 0.868、SE 0.853、SP 0.985、ACC 0.946、IoU 0.768。
• 由于带有 PDC 块的边缘编码器，BEFUnet 展现出强大的边缘分割能力，改善边界预测。
• 消融结果（见表 4）显示边缘与主体分支及融合模块对总体性能的贡献。

Figure 2 : A block diagram of the LCAF.