[Paper Review] LCAUnet: A skin lesion segmentation network with enhanced edge and body fusion

Qisen Ma, Keming Mao+3 more|arXiv (Cornell University)|May 1, 2023

Cutaneous Melanoma Detection and ManagementMedicine3 citations

TL;DR

LCAUnet proposes a U-shaped segmentation network that enhances skin lesion segmentation by fusing edge and body features through a dual-branch encoder, a local cross-attention fusion module (LCAF), and a prior-guided multi-scale fusion (PGMF). It achieves state-of-the-art performance on ISIC 2017, ISIC 2018, and PH2 datasets, with improvements of up to 8.87% in IoU and 7.03% in Dice over baseline models.

ABSTRACT

Accurate segmentation of skin lesions in dermatoscopic images is crucial for the early diagnosis of skin cancer and improving the survival rate of patients. However, it is still a challenging task due to the irregularity of lesion areas, the fuzziness of boundaries, and other complex interference factors. In this paper, a novel LCAUnet is proposed to improve the ability of complementary representation with fusion of edge and body features, which are often paid little attentions in traditional methods. First, two separate branches are set for edge and body segmentation with CNNs and Transformer based architecture respectively. Then, LCAF module is utilized to fuse feature maps of edge and body of the same level by local cross-attention operation in encoder stage. Furthermore, PGMF module is embedded for feature integration with prior guided multi-scale adaption. Comprehensive experiments on public available dataset ISIC 2017, ISIC 2018, and PH2 demonstrate that LCAUnet outperforms most state-of-the-art methods. The ablation studies also verify the effectiveness of the proposed fusion techniques.

Motivation & Objective

• Address the challenge of inaccurate segmentation of skin lesions with irregular shapes and blurry boundaries in dermatoscopic images.
• Overcome the limitation of existing methods that focus primarily on lesion body while neglecting edge features.
• Improve segmentation robustness in the presence of image interferences such as hair, rulers, and staining.
• Enhance feature representation by fusing multi-scale and complementary edge-body features using attention and guided fusion mechanisms.
• Develop a unified deep learning framework that leverages both CNNs and Transformers for superior segmentation performance.

Proposed method

• Employ a dual-branch encoder: a CNN-based branch for edge feature extraction and a Transformer-based branch for body feature learning.
• Introduce the Local Cross-Attention Fusion (LCAF) module to fuse edge and body features at the same encoder level using local cross-attention.
• Apply the Prior-Guided Multi-Scale Fusion (PGMF) module in the decoder to integrate features across adjacent scales using prior knowledge.
• Use skip connections with concatenation and element-wise operations to preserve spatial details and enhance feature propagation.
• Train the network end-to-end using binary cross-entropy and Dice loss to optimize segmentation accuracy.
• Leverage Grad-CAM visualization to validate that the edge encoder focuses on lesion boundaries while the body encoder captures global semantics.

Experimental results

Research questions

• RQ1 Can a dual-branch architecture that separately processes edge and body features improve skin lesion segmentation accuracy compared to single-branch models?
• RQ2 How effective is local cross-attention fusion (LCAF) in integrating complementary edge and body features for better boundary delineation?
• RQ3 To what extent does prior-guided multi-scale fusion (PGMF) enhance feature integration across scales in the decoder?
• RQ4 Does the combination of CNN and Transformer architectures in a unified network architecture lead to better generalization and robustness on challenging dermatoscopic images?
• RQ5 How do edge and body features contribute differently to segmentation performance, and can their fusion mitigate limitations of individual modality representations?

Key findings

• LCAUnet achieves state-of-the-art performance on ISIC 2017, ISIC 2018, and PH2 datasets, outperforming existing SOTA methods in Dice and IoU metrics.
• The ablation study shows that adding the EdgeEncoder alone improves Dice by 5.27% and IoU by 6.74% on ISIC 2017, demonstrating the importance of edge features.
• The LCAF module further boosts performance by 1.02% in Dice and 0.63% in IoU over the EdgeEncoder-only baseline, confirming its effectiveness in cross-modality fusion.
• The PGMF module contributes significantly, improving Dice by 2.52% and IoU by 2.63% over the baseline, indicating its role in multi-scale feature integration.
• The full LCAUnet model achieves 8.87% higher IoU and 7.03% higher Dice than the baseline on ISIC 2017, with consistent improvements on ISIC 2018 and PH2.
• Grad-CAM visualization confirms that the edge encoder focuses on lesion boundaries while the body encoder captures global lesion semantics, validating the complementary design of the dual-branch architecture.