[论文解读] Cooperation Learning Enhanced Colonic Polyp Segmentation Based on Transformer-CNN Fusion

Yuanyuan Wang, Zhaohong Deng+4 more|arXiv (Cornell University)|Jan 17, 2023

Colorectal Cancer Screening and DetectionMedicine被引用 3

一句话总结

该论文提出了一种新型的Transformer-CNN混合神经网络Fu-TransHNet，用于结肠息肉分割，通过多视角协同学习和双分支融合模块增强特征学习。该方法在ETIS-LaribPolypDB数据集上实现了最先进性能，相较于HardNet-MSEG的mDice提升12.4%，相较于TransFuse-s提升6.2%，尤其在小息肉分割方面表现优异。

ABSTRACT

Traditional segmentation methods for colonic polyps are mainly designed based on low-level features. They could not accurately extract the location of small colonic polyps. Although the existing deep learning methods can improve the segmentation accuracy, their effects are still unsatisfied. To meet the above challenges, we propose a hybrid network called Fusion-Transformer-HardNetMSEG (i.e., Fu-TransHNet) in this study. Fu-TransHNet uses deep learning of different mechanisms to fuse each other and is enhanced with multi-view collaborative learning techniques. Firstly, the Fu-TransHNet utilizes the Transformer branch and the CNN branch to realize the global feature learning and local feature learning, respectively. Secondly, a fusion module is designed to integrate the features from two branches. The fusion module consists of two parts: 1) the Global-Local Feature Fusion (GLFF) part and 2) the Dense Fusion of Multi-scale features (DFM) part. The former is built to compensate the feature information mission from two branches at the same scale; the latter is constructed to enhance the feature representation. Thirdly, the above two branches and fusion modules utilize multi-view cooperative learning techniques to obtain their respective weights that denote their importance and then make a final decision comprehensively. Experimental results showed that the Fu-TransHNet network was superior to the existing methods on five widely used benchmark datasets. In particular, on the ETIS-LaribPolypDB dataset containing many small-target colonic polyps, the mDice obtained by Fu-TransHNet were 12.4% and 6.2% higher than the state-of-the-art methods HardNet-MSEG and TransFuse-s, respectively.

研究动机与目标

• 为解决基于低级特征的传统方法在小结肠息肉分割中不准确的问题。
• 通过融合Transformer的全局上下文与CNN的局部细节，提升分割精度。
• 通过多尺度融合模块和多视角协同学习增强特征表示。
• 在基准数据集上实现卓越性能，尤其在小息肉检测方面。

提出的方法

• 网络采用双分支架构：Transformer分支用于全局特征学习，CNN分支用于局部特征学习。
• 全局-局部特征融合（GLFF）模块在相同尺度上整合双分支的特征，以弥补信息缺失。
• 多尺度特征密集融合（DFM）模块通过跨分支聚合多尺度特征，增强特征表示。
• 应用多视角协同学习，动态分配各分支和融合模块的重要性权重，实现综合的最终决策。
• 融合模块通过加权集成，结合基于注意力的全局上下文与卷积的局部模式。
• 整个网络通过反向传播优化的分割损失函数进行端到端训练。

实验结果

研究问题

• RQ1 与现有方法相比，混合的Transformer-CNN架构是否能提升小结肠息肉分割的准确性？
• RQ2 多视角协同学习在息肉分割中的特征融合方面是否具有高效性？
• RQ3 所提出的融合模块是否在保留全局与局部特征方面优于标准的拼接或逐元素操作？
• RQ4 该模型在具有不同息肉大小和外观的多样化基准数据集上泛化能力如何？

主要发现

• 在ETIS-LaribPolypDB数据集上，Fu-TransHNet相较于最先进方法HardNet-MSEG的mDice得分高出12.4%。
• 在包含大量小息肉的同一挑战性数据集上，该模型相较TransFuse-s的mDice提升了6.2%。
• Fu-TransHNet在五个广泛使用的基准数据集上均优于现有方法，展现出一致的泛化能力。
• 消融实验确认，GLFF与DFM模块均对性能提升有显著贡献。
• 多视角协同学习机制有效平衡了Transformer与CNN分支的贡献，提升了模型鲁棒性。