[Paper Review] Colonoscopy Polyp Detection and Classification: Dataset Creation and Comparative Evaluations

Kaidong Li, Mohammad Isyroqi Fathan+7 more|arXiv (Cornell University)|Apr 21, 2021

Colorectal Cancer Screening and Detection90 references120 citations

TL;DR

This paper introduces a large-scale, multi-source colonoscopy dataset with expert-annotated polyp locations and histological classifications, enabling benchmark evaluation of deep learning models. It evaluates eight state-of-the-art object detection models for polyp detection and classification, demonstrating that YOLOv4, ATSS, and RefineDet achieve the best balance of precision, recall, and consistency, with Faster R-CNN achieving the highest recall (93.3%) for adenomatous polyps.

ABSTRACT

Colorectal cancer (CRC) is one of the most common types of cancer with a high mortality rate. Colonoscopy is the preferred procedure for CRC screening and has proven to be effective in reducing CRC mortality. Thus, a reliable computer-aided polyp detection and classification system can significantly increase the effectiveness of colonoscopy. In this paper, we create an endoscopic dataset collected from various sources and annotate the ground truth of polyp location and classification results with the help of experienced gastroenterologists. The dataset can serve as a benchmark platform to train and evaluate the machine learning models for polyp classification. We have also compared the performance of eight state-of-the-art deep learning-based object detection models. The results demonstrate that deep CNN models are promising in CRC screening. This work can serve as a baseline for future research in polyp detection and classification.

Motivation & Objective

• To create a standardized, large-scale endoscopic dataset with ground-truth polyp annotations for colorectal cancer screening.
• To evaluate and compare the performance of eight state-of-the-art deep learning-based object detectors on polyp detection and classification.
• To establish a benchmark for future research in computer-aided polyp diagnosis using endoscopic images.
• To investigate the impact of localization refinement on classification performance in polyp detection.

Proposed method

• Collected colonoscopy images from diverse clinical sources to form a comprehensive dataset.
• Annotated polyp locations with bounding boxes and histological types (adenomatous, hyperplastic) using expert gastroenterologists.
• Trained and evaluated eight state-of-the-art deep learning object detectors: Faster R-CNN, SSD, YOLOv3, RetinaNet, DetNet, RefineDet, YOLOv4, and ATSS.
• Performed frame-based one-class detection and sequence-based two-class classification to assess model robustness across video sequences.
• Used metrics including precision, recall, F1-score, average precision (AP), and inference time to compare models.
• Applied focal loss in RetinaNet and anchor refinement techniques to improve detection of small polyps.

Experimental results

Research questions

• RQ1 Which deep learning object detector achieves the highest detection recall for adenomatous polyps in colonoscopy images?
• RQ2 How do single-stage and two-stage detectors compare in terms of precision, recall, and inference speed for polyp detection?
• RQ3 What is the impact of localization refinement on polyp classification performance?
• RQ4 How consistent are the models in predicting the same polyp class across multiple frames in a video sequence?
• RQ5 Which models show the most robust and stable performance across both detection and classification tasks?

Key findings

• Faster R-CNN achieved the highest recall (93.3%) for adenomatous polyps, outperforming most models in detection sensitivity.
• RefineDet achieved the highest F1-score (88.6) and AP (88.5) in frame-based one-class detection, indicating the best overall detection performance.
• YOLOv4, ATSS, and RefineDet showed the most balanced performance across precision, recall, and F1-score in both detection and classification tasks.
• SSD and YOLOv3 achieved the fastest inference times (17ms), enabling over 60 FPS, making them suitable for real-time applications.
• DetNet showed improved precision (85.8%) over Faster R-CNN (63.9%) but had lower recall (81.8%), indicating a trade-off in detection strategy.
• Models like RefineDet, SSD, YOLOv4, and ATSS demonstrated higher consistency in class prediction across video sequences, with over 70% of sequences showing a dominant predicted class.