[Paper Review] CARE: A Large Scale CT Image Dataset and Clinical Applicable Benchmark Model for Rectal Cancer Segmentation

Hantao Zhang, Weidong Guo+5 more|arXiv (Cornell University)|Aug 16, 2023

Radiomics and Machine Learning in Medical Imaging4 citations

TL;DR

This paper introduces CARE, a large-scale, clinically annotated CT dataset with pixel-level segmentation for normal and cancerous rectum, and proposes U-SAM, a promptable U-shaped transformer-based model that integrates convolutional feature extraction and skip-connections to improve segmentation accuracy. U-SAM achieves state-of-the-art performance on both CARE and WORD datasets, with a 4.10% mean Dice gain using three point prompts.

ABSTRACT

Rectal cancer segmentation of CT image plays a crucial role in timely clinical diagnosis, radiotherapy treatment, and follow-up. Although current segmentation methods have shown promise in delineating cancerous tissues, they still encounter challenges in achieving high segmentation precision. These obstacles arise from the intricate anatomical structures of the rectum and the difficulties in performing differential diagnosis of rectal cancer. Additionally, a major obstacle is the lack of a large-scale, finely annotated CT image dataset for rectal cancer segmentation. To address these issues, this work introduces a novel large scale rectal cancer CT image dataset CARE with pixel-level annotations for both normal and cancerous rectum, which serves as a valuable resource for algorithm research and clinical application development. Moreover, we propose a novel medical cancer lesion segmentation benchmark model named U-SAM. The model is specifically designed to tackle the challenges posed by the intricate anatomical structures of abdominal organs by incorporating prompt information. U-SAM contains three key components: promptable information (e.g., points) to aid in target area localization, a convolution module for capturing low-level lesion details, and skip-connections to preserve and recover spatial information during the encoding-decoding process. To evaluate the effectiveness of U-SAM, we systematically compare its performance with several popular segmentation methods on the CARE dataset. The generalization of the model is further verified on the WORD dataset. Extensive experiments demonstrate that the proposed U-SAM outperforms state-of-the-art methods on these two datasets. These experiments can serve as the baseline for future research and clinical application development.

Motivation & Objective

• To address the lack of large-scale, finely annotated CT datasets for rectal cancer segmentation.
• To improve segmentation precision in complex anatomical structures like the rectum, which feature irregular shapes and thin walls.
• To develop a clinically applicable benchmark model that generalizes well across datasets and supports accurate tumor delineation.
• To integrate prompt-based localization with U-Net-like skip-connections and convolutional modules to enhance feature learning in medical images.
• To establish a new baseline for rectal cancer segmentation using real clinical data and advanced deep learning.

Proposed method

• The CARE dataset comprises 1,000 clinical CT volumes with pixel-level annotations for both normal rectum and rectal tumors, created through consensus from expert radiologists.
• U-SAM employs a U-shaped architecture combining a vision transformer encoder with a convolutional module for local detail capture and skip-connections to preserve spatial information.
• The model uses promptable inputs such as points to guide localization, enabling flexible and accurate segmentation with minimal user input.
• Skip-connections are applied at four resolution scales (1/8, 1/4, 1/2, and 1) to optimize feature recovery and boundary delineation.
• The model is trained end-to-end on the CARE dataset and evaluated on both CARE and the external WORD dataset to verify generalization.
• Ablation studies systematically evaluate the impact of prompt count and skip-connection configuration on performance.

Experimental results

Research questions

• RQ1 Can a large-scale, clinically annotated CT dataset significantly improve the performance of rectal cancer segmentation models?
• RQ2 How does integrating prompt-based localization with a U-shaped architecture enhance segmentation accuracy in complex abdominal anatomy?
• RQ3 What is the optimal number of point prompts for achieving high segmentation performance in rectal cancer detection?
• RQ4 How do skip-connections at multiple scales affect the recovery of fine spatial details in rectal segmentation?
• RQ5 To what extent does U-SAM generalize to external datasets like WORD compared to existing state-of-the-art models?

Key findings

• U-SAM achieves a 4.10% improvement in mean Dice score and a 4.94% improvement in mean IoU when using three point prompts per class, outperforming baseline models.
• The addition of a convolution module and skip-connections to the SAM framework improves mean Dice by 3.3% and mean IoU by 3.67%.
• Using five point prompts does not yield further gains, indicating diminishing returns beyond three prompts.
• The optimal configuration uses skip-connections at all four resolution scales (1/8, 1/4, 1/2, and 1), maximizing performance on both normal rectum and tumor segmentation.
• Visualizations confirm that U-SAM produces more coherent boundaries and reduces false positives compared to baseline models.
• U-SAM generalizes effectively to the WORD dataset, demonstrating robustness and clinical applicability beyond the training distribution.