[Paper Review] PolypDB: A Curated Multi-Center Dataset for Development of AI Algorithms in Colonoscopy

Debesh Jha, Nikhil Kumar Tomar+20 more|arXiv (Cornell University)|Aug 19, 2024

Colorectal Cancer Screening and DetectionMedicine3 citations

TL;DR

PolypDB is a large-scale, multi-center, multi-modality dataset of 3,934 endoscopic polyp images across WLI, NBI, and LCI modalities, curated to improve AI-based polyp detection and segmentation. The study benchmarks state-of-the-art models, demonstrating superior performance—especially SSFormer-L and YOLOv10—with mIoU up to 0.8821 and F2 score of 0.8941, establishing a robust benchmark for clinical AI development.

ABSTRACT

Colonoscopy is the primary method for examination, detection, and removal of polyps. However, challenges such as variations among the endoscopists' skills, bowel quality preparation, and the complex nature of the large intestine contribute to high polyp miss-rate. These missed polyps can develop into cancer later, underscoring the importance of improving the detection methods. To address this gap of lack of publicly available, multi-center large and diverse datasets for developing automatic methods for polyp detection and segmentation, we introduce PolypDB, a large scale publicly available dataset that contains 3934 still polyp images and their corresponding ground truth from real colonoscopy videos. PolypDB comprises images from five modalities: Blue Light Imaging (BLI), Flexible Imaging Color Enhancement (FICE), Linked Color Imaging (LCI), Narrow Band Imaging (NBI), and White Light Imaging (WLI) from three medical centers in Norway, Sweden, and Vietnam. We provide a benchmark on each modality and center, including federated learning settings using popular segmentation and detection benchmarks. PolypDB is public and can be downloaded at \url{https://osf.io/pr7ms/}. More information about the dataset, segmentation, detection, federated learning benchmark and train-test split can be found at \url{https://github.com/DebeshJha/PolypDB}.

Motivation & Objective

• Address the critical need for diverse, real-world data to train robust AI models for polyp detection and segmentation in colonoscopy.
• Overcome limitations of existing datasets by incorporating multi-center, multi-modality (WLI, NBI, LCI) endoscopic images from geographically diverse clinical settings.
• Provide a standardized benchmark to evaluate and compare state-of-the-art segmentation and detection models across different imaging modalities.
• Enhance model generalizability and clinical applicability by including varied polyp types, including challenging diminutive and flat polyps.
• Support the development of AI systems that reduce polyp miss rates and improve early colorectal cancer detection.

Proposed method

• Curated a multi-center dataset of 3,934 polyp images from 10 medical centers across the USA, Sweden, Vietnam, and Vietnam, ensuring diversity in patient demographics and imaging protocols.
• Collected images across three endoscopic modalities: white light imaging (WLI), narrow-band imaging (NBI), and linked-endoscopically (LCI), to reflect real-world clinical practice.
• Performed rigorous data curation, including expert-verified polyp annotations and quality control to ensure high annotation accuracy.
• Benchmarked state-of-the-art segmentation and detection models (e.g., SSFormer-L, YOLOv10, PVT-CASCADE) on all three modalities using standard metrics: mIoU, mDSC, precision, recall, and F2 score.
• Employed the MiT-PLD-B4 backbone in SSFormer-L to enhance multi-scale feature learning and robustness across diverse visual features.
• Evaluated model performance across modalities to identify modality-specific strengths and guide clinical deployment decisions.

Figure 1: Examples of polyps in BLI, FICE, LCI, NBI, and WLI modalities from the PolypDB dataset, showcasing variations in shape, size, color, and appearance. Each image includes polyp bounding boxes and color-coded segmentation masks to show polyp ground truth.

Experimental results

Research questions

• RQ1 Can a large-scale, multi-center, multi-modality dataset improve the generalization and robustness of AI models for polyp detection and segmentation in colonoscopy?
• RQ2 How do state-of-the-art segmentation and detection models perform across different endoscopic imaging modalities (WLI, NBI, LCI) when trained and evaluated on PolypDB?
• RQ3 Which model architectures achieve the highest performance in terms of mIoU, mDSC, recall, and F2 score across diverse polyp types and imaging conditions?
• RQ4 What are the key performance differences between models in detecting subtle or challenging polyp structures, such as diminutive or flat polyps?
• RQ5 How can modality-specific benchmarks guide the selection of optimal AI models for clinical deployment in real-time colonoscopy settings?

Key findings

• SSFormer-L achieved the highest mIoU (0.8821) and mDSC (0.9294) on the WLI dataset, with strong recall (0.9314) and precision (0.9438), indicating high segmentation accuracy.
• PVT-CASCADE achieved the highest F2 score (0.8941) and recall (0.9385) on the NBI dataset, demonstrating superior performance in detecting subtle polyp structures.
• YOLOv10 and SSFormer-L outperformed other models across all modalities, establishing strong baselines for future research on polyp detection and segmentation.
• The MiT-PLD-B4 backbone significantly enhanced model robustness by enabling effective multi-scale feature extraction across diverse imaging conditions.
• PolypDB’s multi-center and multi-modality design enables training of models with improved generalization, reducing domain shift and enhancing real-world clinical applicability.
• The dataset and benchmarks provide a foundation for future development of video-based AI systems capturing dynamic polyp detection during real-time colonoscopy procedures.

Figure 2: Qualitative results for the different modalities in the PolypDB dataset.