[Paper Review] COTR: Convolution in Transformer Network for End to End Polyp Detection

Zhi-Qiang Shen, Chaonan Lin+1 more|arXiv (Cornell University)|May 23, 2021

Radiomics and Machine Learning in Medical Imaging4 citations

TL;DR

This paper proposes COTR, a novel end-to-end polyp detection model that integrates convolutional layers within transformer encoder blocks to accelerate convergence and improve feature reconstruction. Evaluated on ETIS-LARIB and CVC-ColonDB, COTR achieved 91.49% precision, 82.69% sensitivity, and 86.87% F1-score on ETIS-LARIB, and 91.67% precision, 93.54% sensitivity, and 92.60% F1-score on CVC-ColonDB, demonstrating state-of-the-art performance without handcrafted pre- or post-processing steps.

ABSTRACT

Purpose: Colorectal cancer (CRC) is the second most common cause of cancer mortality worldwide. Colonoscopy is a widely used technique for colon screening and polyp lesions diagnosis. Nevertheless, manual screening using colonoscopy suffers from a substantial miss rate of polyps and is an overwhelming burden for endoscopists. Computer-aided diagnosis (CAD) for polyp detection has the potential to reduce human error and human burden. However, current polyp detection methods based on object detection framework need many handcrafted pre-processing and post-processing operations or user guidance that require domain-specific knowledge. Methods: In this paper, we propose a convolution in transformer (COTR) network for end-to-end polyp detection. Motivated by the detection transformer (DETR), COTR is constituted by a CNN for feature extraction, transformer encoder layers interleaved with convolutional layers for feature encoding and recalibration, transformer decoder layers for object querying, and a feed-forward network for detection prediction. Considering the slow convergence of DETR, COTR embeds convolution layers into transformer encoder for feature reconstruction and convergence acceleration. Results: Experimental results on two public polyp datasets show that COTR achieved 91.49\% precision, 82.69% sensitivity, and 86.87% F1-score on the ETIS-LARIB, and 91.67% precision, 93.54% sensitivity, and 92.60% F1-score on the CVC-ColonDB. Conclusion: This study proposed an end to end detection method based on detection transformer for colorectal polyp detection. Experimental results on ETIS-LARIB and CVC-ColonDB dataset demonstrated that the proposed model achieved comparable performance against state-of-the-art methods.

Motivation & Objective

• To address the high miss rate and workload in colonoscopy by developing an end-to-end computer-aided diagnosis (CAD) system for polyp detection.
• To overcome the slow convergence and poor feature organization in standard transformers when applied to polyp detection, especially given limited and complex polyp datasets.
• To eliminate reliance on handcrafted pre-processing and post-processing operations such as anchor generation and non-maximum suppression in existing object detection frameworks.
• To improve detection robustness for polyps with diverse textures, shapes, and low contrast against background tissues.

Proposed method

• COTR employs a CNN backbone for initial feature extraction from colonoscopy images.
• Transformer encoder layers are interleaved with convolutional layers to reconstruct high-level image features and stabilize training dynamics.
• The model uses a transformer decoder with learnable object queries to predict bounding boxes and class scores directly.
• A feed-forward network generates final detection predictions from the decoder output.
• The integration of convolution into the transformer encoder helps preserve spatial structure and accelerates convergence compared to standard DETR.
• The architecture is trained end-to-end using AdamW optimizer with a total of 300 epochs and data augmentation to improve generalization.

Experimental results

Research questions

• RQ1 Can a transformer-based object detector achieve competitive performance in end-to-end polyp detection without relying on handcrafted components?
• RQ2 How does inserting convolutional layers into the transformer encoder affect convergence speed and feature representation in polyp detection?
• RQ3 Does the proposed COTR model outperform existing state-of-the-art methods on public polyp detection benchmarks?
• RQ4 Can the model effectively detect small or sessile polyps that are challenging due to low contrast and irregular shapes?

Key findings

• On the ETIS-LARIB dataset, COTR achieved 91.49% precision, 82.69% sensitivity, and 86.87% F1-score, outperforming most existing methods in precision and F1-score.
• On the CVC-ColonDB dataset, COTR achieved 91.67% precision, 93.54% sensitivity, and 92.60% F1-score, setting new state-of-the-art performance on this benchmark.
• The training loss of COTR converged significantly faster than DETR, confirming that convolutional layers in the encoder improve convergence by preserving spatial feature structure.
• COTR demonstrated robustness in detecting polyps with varied textures, shapes, and colors, including small polyps, though it occasionally struggled with sessile polyps due to low confidence predictions.
• The model’s performance was stable across diverse colonoscopy conditions, including those with residual liquid or low-contrast backgrounds.
• The ablation study confirmed that the integration of convolution into the transformer encoder was critical for convergence acceleration and improved detection accuracy.