[Paper Review] DilatedSegNet: A Deep Dilated Segmentation Network for Polyp Segmentation

Nikhil Kumar Tomar, Debesh Jha+1 more|arXiv (Cornell University)|Oct 24, 2022

Colorectal Cancer Screening and Detection4 citations

TL;DR

DilatedSegNet is a real-time, lightweight deep learning architecture for polyp segmentation in colonoscopy images, leveraging a pre-trained ResNet50 encoder with dilated convolution pooling (DCP) blocks and CBAM attention modules to enhance feature extraction and boundary detection. It achieves 33.68 FPS inference speed with a 0.90 Dice coefficient and 0.83 mIoU on public datasets, demonstrating high accuracy and generalizability for clinical deployment.

ABSTRACT

Colorectal cancer (CRC) is the second leading cause of cancer-related death worldwide. Excision of polyps during colonoscopy helps reduce mortality and morbidity for CRC. Powered by deep learning, computer-aided diagnosis (CAD) systems can detect regions in the colon overlooked by physicians during colonoscopy. Lacking high accuracy and real-time speed are the essential obstacles to be overcome for successful clinical integration of such systems. While literature is focused on improving accuracy, the speed parameter is often ignored. Toward this critical need, we intend to develop a novel real-time deep learning-based architecture, DilatedSegNet, to perform polyp segmentation on the fly. DilatedSegNet is an encoder-decoder network that uses pre-trained ResNet50 as the encoder from which we extract four levels of feature maps. Each of these feature maps is passed through a dilated convolution pooling (DCP) block. The outputs from the DCP blocks are concatenated and passed through a series of four decoder blocks that predicts the segmentation mask. The proposed method achieves a real-time operation speed of 33.68 frames per second with an average dice coefficient of 0.90 and mIoU of 0.83. Additionally, we also provide heatmap along with the qualitative results that shows the explanation for the polyp location, which increases the trustworthiness of the method. The results on the publicly available Kvasir-SEG and BKAI-IGH datasets suggest that DilatedSegNet can give real-time feedback while retaining a high \ac{DSC}, indicating high potential for using such models in real clinical settings in the near future. The GitHub link of the source code can be found here: \url{https://github.com/nikhilroxtomar/DilatedSegNet}.

Motivation & Objective

• To address the clinical challenge of missed polyps during colonoscopy, which contribute to interval colorectal cancer.
• To develop a real-time, accurate, and generalizable polyp segmentation model suitable for integration into clinical endoscopy workflows.
• To improve segmentation performance by enhancing feature representation in the encoder through dilated convolutions and attention mechanisms.
• To ensure model interpretability by generating heatmaps that highlight salient polyp regions.
• To balance high accuracy with computational efficiency for deployment on standard hardware.

Proposed method

• The network uses a pre-trained ResNet50 as the encoder to extract multi-scale feature maps at four levels.
• Each feature map is processed through a Dilated Convolution Pooling (DCP) block using parallel dilated convolutions with rates 1, 3, 6, and 9 to expand the receptive field.
• Outputs from the DCP blocks are concatenated and passed to a decoder composed of upsampling, skip-connection concatenation, residual blocks, and a CBAM attention module.
• The final segmentation mask is produced via a 1×1 convolution layer after the last decoder block.
• The architecture is trained using standard binary cross-entropy and dice loss, with data augmentation applied during training.
• Model interpretability is enhanced through Grad-CAM-style heatmaps to visualize attention on polyp regions.

Experimental results

Research questions

• RQ1 Can a lightweight, real-time segmentation network achieve high accuracy in polyp detection without sacrificing inference speed?
• RQ2 How does the integration of dilated convolutions and attention mechanisms improve segmentation of small and flat polyps?
• RQ3 To what extent does the proposed architecture generalize to unseen datasets from different clinical settings?
• RQ4 Does the attention mechanism and DCP block contribute significantly to boundary delineation and model robustness?
• RQ5 Can heatmaps generated from the network provide reliable visual explanations for polyp localization?

Key findings

• DilatedSegNet achieves a mean Dice coefficient (DSC) of 0.90 and mIoU of 0.83 on the Kvasir-SEG dataset, outperforming nine benchmark methods.
• The model operates in real time at 33.68 frames per second, with only 18.11 million parameters and 27.1 GMac FLOPs, enabling deployment on standard hardware.
• On cross-dataset evaluation, DilatedSegNet achieves 0.8278 DSC and 0.7545 mIoU when trained on Kvasir-SEG and tested on CVC-ClinicDB, outperforming DeepLabv3+ by 1.36% in DSC.
• When tested on the unseen BKAI-IGH dataset, DilatedSegNet achieves 2.47% higher DSC and 2.97% higher mIoU than PraNet, the previous SOTA model.
• Abllation studies confirm that removing either the DCP block or CBAM attention module reduces DSC by 3.3% and mIoU by 3.9%, proving their critical role in performance.
• Heatmap visualization confirms that the model focuses on polyp regions and boundaries, enhancing interpretability and clinical trust.