[Paper Review] From CNN to Transformer: A Review of Medical Image Segmentation Models

Wenjian Yao, Jiajun Bai+4 more|arXiv (Cornell University)|Aug 10, 2023

Radiomics and Machine Learning in Medical Imaging8 citations

TL;DR

This paper surveys four representative medical image segmentation models (U-Net, UNet++, TransUNet, Swin-Unet), analyzes their characteristics, and compares their performance on Tuberculosis Chest X-rays and Ovarian Tumors datasets, highlightingTransformer-based models' advantages in global context modeling.

ABSTRACT

Medical image segmentation is an important step in medical image analysis, especially as a crucial prerequisite for efficient disease diagnosis and treatment. The use of deep learning for image segmentation has become a prevalent trend. The widely adopted approach currently is U-Net and its variants. Additionally, with the remarkable success of pre-trained models in natural language processing tasks, transformer-based models like TransUNet have achieved desirable performance on multiple medical image segmentation datasets. In this paper, we conduct a survey of the most representative four medical image segmentation models in recent years. We theoretically analyze the characteristics of these models and quantitatively evaluate their performance on two benchmark datasets (i.e., Tuberculosis Chest X-rays and ovarian tumors). Finally, we discuss the main challenges and future trends in medical image segmentation. Our work can assist researchers in the related field to quickly establish medical segmentation models tailored to specific regions.

Motivation & Objective

• Assess the evolution and characteristics of CNN- and Transformer-based medical image segmentation models.
• Quantitatively compare four representative models on benchmark datasets.
• Identify challenges and future trends in medical image segmentation.
• Provide experimental configurations and open-source code to aid replication and extension.

Proposed method

• Review four representative models: U-Net, UNet++, TransUNet, Swin-Unet.
• Theoretically analyze each model's strengths and limitations in feature representation and long-range dependency modeling.
• Experimentally evaluate models on Tuberculosis Chest X-rays and Ovarian Tumors datasets using standard segmentation metrics.
• Report Dice, HD95, IoU, Accuracy, Precision, and Recall to compare performance.
• Discuss challenges like data labeling and class imbalance, and suggest mitigation strategies.
• Share code and configuration parameters on GitHub for reproducibility.

Experimental results

Research questions

• RQ1 How do traditional CNN-based architectures compare to Transformer-based architectures for medical image segmentation?
• RQ2 What is the impact of incorporating Transformer modules (or Swin Transformer) on segmentation performance across different datasets?
• RQ3 Which model provides the best balance of accuracy, boundary quality, and computational efficiency on the tested tasks?
• RQ4 What are the practical challenges in applying supervised segmentation to medical images, and how can they be mitigated?

Key findings

• TransUNet achieves the best overall performance on Tuberculosis Chest X-rays across all metrics examined.
• On the TB dataset, TransUNet attains DSC 96.45, HD95 10.75, IoU 93.25, Accuracy 98.16, Precision 97.36, Recall 95.72.
• For the ovarian tumors dataset, TransUNet again leads in Dice, HD95, IoU, Accuracy, and Precision, with Recall second to Swin-Unet.
• All four methods achieve mean IoU values above 91% for the lung segmentation task, indicating strong performance in easy-to-segment regions.
• Introducing Transformer modules reduces instances of extremely poor Dice scores (<20%) compared to pure CNNs.
• The study concludes that Transformer-enabled architectures (notably TransUNet) effectively capture global information and improve segmentation accuracy.