[Paper Review] Performance of a Deep Learning-Based Segmentation Model for Pancreatic Tumors on Public Endoscopic Ultrasound Datasets

Pankaj Kumar Gupta, Priya Mudgil+12 more|arXiv (Cornell University)|Jan 9, 2026

Pancreatic and Hepatic Oncology Research0 citations

TL;DR

A Vision Transformer-based segmentation model (HVITBackbone4Seg) was trained on public EUS datasets and externally evaluated, achieving moderate Dice scores around 0.65 with high specificity, highlighting generalizability but with some failure cases.

ABSTRACT

Background: Pancreatic cancer is one of the most aggressive cancers, with poor survival rates. Endoscopic ultrasound (EUS) is a key diagnostic modality, but its effectiveness is constrained by operator subjectivity. This study evaluates a Vision Transformer-based deep learning segmentation model for pancreatic tumors. Methods: A segmentation model using the USFM framework with a Vision Transformer backbone was trained and validated with 17,367 EUS images (from two public datasets) in 5-fold cross-validation. The model was tested on an independent dataset of 350 EUS images from another public dataset, manually segmented by radiologists. Preprocessing included grayscale conversion, cropping, and resizing to 512x512 pixels. Metrics included Dice similarity coefficient (DSC), intersection over union (IoU), sensitivity, specificity, and accuracy. Results: In 5-fold cross-validation, the model achieved a mean DSC of 0.651 +/- 0.738, IoU of 0.579 +/- 0.658, sensitivity of 69.8%, specificity of 98.8%, and accuracy of 97.5%. For the external validation set, the model achieved a DSC of 0.657 (95% CI: 0.634-0.769), IoU of 0.614 (95% CI: 0.590-0.689), sensitivity of 71.8%, and specificity of 97.7%. Results were consistent, but 9.7% of cases exhibited erroneous multiple predictions. Conclusions: The Vision Transformer-based model demonstrated strong performance for pancreatic tumor segmentation in EUS images. However, dataset heterogeneity and limited external validation highlight the need for further refinement, standardization, and prospective studies.

Motivation & Objective

• Motivate automated, standardized pancreatic tumor segmentation in endoscopic ultrasound (EUS) to reduce operator variability.
• Develop and evaluate a Vision Transformer-based segmentation model on large public EUS datasets.
• Assess generalizability through external validation on an independent public dataset.

Proposed method

• Use USFM framework with Vision Transformer backbone (HVITBackbone4Seg) for 2-class segmentation (foreground/background).
• Preprocess EUS images by grayscale conversion, cropping, and resizing to 512x512 pixels.
• Train with 50 epochs in 5-fold cross-validation using AdamW optimizer and cosine learning rate, selecting best Dice score as early stopping criterion.
• Evaluate using Dice similarity coefficient (DSC), IoU, sensitivity, specificity, and accuracy with 95% CIs; report qualitative failure analysis.
• Test on external LEP dataset subset (350 images) with no post-processing beyond argmax to obtain binary masks.

Experimental results

Research questions

• RQ1 Can a Vision Transformer-based segmentation model achieve robust delineation of pancreatic tumors in publicly available EUS datasets?
• RQ2 How well does the model generalize to an independent external EUS dataset?
• RQ3 What are common failure modes in segmentation performance on EUS images?

Key findings

• Mean Dice Similarity Coefficient (DSC) during 5-fold cross-validation: 0.651 (95% CI: 0.615–0.738).
• IoU during cross-validation: 0.579 (95% CI: 0.557–0.658).
• Specificity during cross-validation: 98.8%; Sensitivity: 69.8%; Overall accuracy: 97.5%.
• External test set (350 images): DSC 0.657 (95% CI: 0.634–0.769); IoU 0.614 (95% CI: 0.590–0.689).
• External test set: Sensitivity 71.8% (95% CI: 69.1–79.3); Specificity 97.7% (95% CI: 95.1–99.2).
• 9.7% of cases showed erroneous multiple predictions, indicating some failure modes.