[Paper Review] CS2-Net: Deep Learning Segmentation of Curvilinear Structures in Medical Imaging

Lei Mou, Yitian Zhao+10 more|arXiv (Cornell University)|Oct 15, 2020

Retinal Imaging and Analysis69 references36 citations

TL;DR

CS2-Net is a unified 2D/3D deep learning model with dual self-attention (channel and spatial) and boundary-aware convolutions for segmenting curvilinear structures across multiple medical imaging modalities, achieving state-of-the-art results on nine datasets.

ABSTRACT

Automated detection of curvilinear structures, e.g., blood vessels or nerve fibres, from medical and biomedical images is a crucial early step in automatic image interpretation associated to the management of many diseases. Precise measurement of the morphological changes of these curvilinear organ structures informs clinicians for understanding the mechanism, diagnosis, and treatment of e.g. cardiovascular, kidney, eye, lung, and neurological conditions. In this work, we propose a generic and unified convolution neural network for the segmentation of curvilinear structures and illustrate in several 2D/3D medical imaging modalities. We introduce a new curvilinear structure segmentation network (CS2-Net), which includes a self-attention mechanism in the encoder and decoder to learn rich hierarchical representations of curvilinear structures. Two types of attention modules - spatial attention and channel attention - are utilized to enhance the inter-class discrimination and intra-class responsiveness, to further integrate local features with their global dependencies and normalization, adaptively. Furthermore, to facilitate the segmentation of curvilinear structures in medical images, we employ a 1x3 and a 3x1 convolutional kernel to capture boundary features. ...

Motivation & Objective

• Automate the segmentation of curvilinear structures (e.g., vessels, nerve fibers) across diverse medical imaging modalities.
• Develop a unified encoder–decoder network capable of handling both 2D and 3D data.
• Incorporate channel and spatial attention to learn rich hierarchical representations and long-range dependencies.
• Enhance boundary feature capture with specialized 1x3 and 3x1 convolutions and extend attention to 3D.
• Demonstrate robust performance against state-of-the-art methods across multiple datasets.

Proposed method

• Proposes CS2-Net with an encoder–decoder backbone and four ResNet-based blocks with skip connections.
• Introduces a Channel and Spatial Attention Module (CSAM) consisting of a 2D/3D Spatial Attention Block (SAB) and Channel Attention Block (CAB) to produce attention-aware features.
• Implements boundary-focused convolutions (1x3 and 3x1 in 2D; extended 3D equivalents) to better capture curvilinear boundaries.
• Extends the attention mechanism to 3D to aggregate depth information across slices.
• Uses a loss combining weighted cross-entropy and Dice coefficient (L = alpha*LWCE + (1-alpha)*LDice, with alpha = 0.6).
• Adapts the CSAM for 3D by incorporating additional directional convolutions (e.g., 1x1x3) to model width, height, and depth relationships.

Experimental results

Research questions

• RQ1 Can CS2-Net effectively segment curvilinear structures in both 2D and 3D medical images across multiple modalities?
• RQ2 Do channel and spatial attention mechanisms improve discrimination between curvilinear structures and background, and capture long-range dependencies in these structures?
• RQ3 Does incorporating boundary-focused convolutions and extending attention to 3D improve segmentation performance across datasets?
• RQ4 How does CS2-Net compare to state-of-the-art methods on nine diverse datasets?

Key findings

• CS2-Net achieves overall state-of-the-art performance in detecting curvilinear structures across diverse biomedical imaging modalities.
• The framework is validated on six 2D datasets and three 3D datasets, demonstrating versatility across imaging types.
• Dual attention (channel and spatial) enhances feature discrimination and global context modeling for curvilinear structures.
• Boundary-aware convolutions help capture edge information essential for thin, elongated structures.
• A unified 2D/3D architecture with skip connections and a principled loss improves segmentation robustness.