[Paper Review] Colorectal Polyp Segmentation in the Deep Learning Era: A Comprehensive Survey

Zhenyu Wu, Fengmao Lv+3 more|arXiv (Cornell University)|Jan 22, 2024

Colorectal Cancer Screening and Detection8 citations

TL;DR

This paper conducts a systematic survey of deep-learning based colorectal polyp segmentation (CPS) methods from 2014 to 2023, introducing a taxonomy, datasets, metrics, and a benchmarking analysis of 40 state-of-the-art models.

ABSTRACT

Colorectal polyp segmentation (CPS), an essential problem in medical image analysis, has garnered growing research attention. Recently, the deep learning-based model completely overwhelmed traditional methods in the field of CPS, and more and more deep CPS methods have emerged, bringing the CPS into the deep learning era. To help the researchers quickly grasp the main techniques, datasets, evaluation metrics, challenges, and trending of deep CPS, this paper presents a systematic and comprehensive review of deep-learning-based CPS methods from 2014 to 2023, a total of 115 technical papers. In particular, we first provide a comprehensive review of the current deep CPS with a novel taxonomy, including network architectures, level of supervision, and learning paradigm. More specifically, network architectures include eight subcategories, the level of supervision comprises six subcategories, and the learning paradigm encompasses 12 subcategories, totaling 26 subcategories. Then, we provided a comprehensive analysis the characteristics of each dataset, including the number of datasets, annotation types, image resolution, polyp size, contrast values, and polyp location. Following that, we summarized CPS's commonly used evaluation metrics and conducted a detailed analysis of 40 deep SOTA models, including out-of-distribution generalization and attribute-based performance analysis. Finally, we discussed deep learning-based CPS methods' main challenges and opportunities.

Motivation & Objective

• Provide a structured overview of deep CPS methods and their evolution from 2014 to 2023.
• Catalog network architectures, supervision levels, and learning paradigms in CPS.
• Analyze CPS datasets and evaluation metrics to benchmark performance.
• Summarize current challenges and identify promising directions for CPS research.
• Facilitate rapid understanding and future work for researchers entering CPS.

Proposed method

• Propose a novel taxonomy dividing deep CPS approaches into network architectures, supervision levels, and learning paradigms (26 subcategories).
• Systematically review 115 CPS papers from 2014 to 2023.
• Characterize 14 CPS datasets by annotations, resolution, polyp size, contrast, and location.
• Benchmark 40 deep CPS models, including out-of-distribution generalization and attribute-based analyses.
• Compare methods using the PolypGen and SUN-SEG datasets to assess generalization and attribute performance.
• Discuss challenges and opportunities in CPS, including interpretability, generalization, privacy, and potential integration with large models.

Experimental results

Research questions

• RQ1 What are the core architectural, supervisory, and learning paradigm patterns that define modern CPS methods?
• RQ2 How do CPS datasets differ in annotations, resolution, polyp size, contrast, and location, and how do these differences affect model performance?
• RQ3 What is the landscape of deep CPS models from 2014 to 2023, and how do the 40 SOTA models compare in generalization and attribute-based performance?
• RQ4 What are the main challenges facing CPS, and what future directions show the most promise?
• RQ5 How can CPS methods be benchmarked in a standardized, open framework across multiple datasets?

Key findings

• A comprehensive taxonomy organizes CPS methods into 26 subcategories across architectures, supervision, and learning paradigms.
• The survey covers 115 deep CPS papers from 2014 to 2023 and analyzes 40 deep SOTA models.
• Fourteen CPS datasets and twelve common evaluation metrics are discussed.
• The analysis includes out-of-distribution generalization (PolypGen) and attribute-based performance (SUN-SEG).
• The paper identifies challenges such as interpretability, generalization, robustness, and data privacy, and highlights opportunities like unsupervised anomaly localization and integration with large visual/language models.