[Paper Review] Efficient automatic segmentation for multi-level pulmonary arteries: The PARSE challenge

Gongning Luo, Kuanquan Wang+28 more|arXiv (Cornell University)|Apr 7, 2023

Medical Imaging and Pathology Studies10 citations

TL;DR

PARSE benchmarks methods for simultaneous main and branch pulmonary artery segmentation in CTPA, balancing accuracy with inference efficiency using a multi-level evaluation and open dataset. Top teams combined novel architectures, data strategies, and efficiency tricks to optimize Dice and HD95 while minimizing runtime and GPU memory.

ABSTRACT

Efficient automatic segmentation of multi-level (i.e. main and branch) pulmonary arteries (PA) in CTPA images plays a significant role in clinical applications. However, most existing methods concentrate only on main PA or branch PA segmentation separately and ignore segmentation efficiency. Besides, there is no public large-scale dataset focused on PA segmentation, which makes it highly challenging to compare the different methods. To benchmark multi-level PA segmentation algorithms, we organized the first extbf{P}ulmonary extbf{AR}tery extbf{SE}gmentation (PARSE) challenge. On the one hand, we focus on both the main PA and the branch PA segmentation. On the other hand, for better clinical application, we assign the same score weight to segmentation efficiency (mainly running time and GPU memory consumption during inference) while ensuring PA segmentation accuracy. We present a summary of the top algorithms and offer some suggestions for efficient and accurate multi-level PA automatic segmentation. We provide the PARSE challenge as open-access for the community to benchmark future algorithm developments at \url{https://parse2022.grand-challenge.org/Parse2022/}.

Motivation & Objective

• Motivate and establish a public, large-scale benchmark for simultaneous main and branch pulmonary artery segmentation.
• Evaluate accuracy and efficiency together to reflect clinical deployment needs.
• Provide insights and guidelines from top-performing methods to advance efficient multi-level PA segmentation.

Proposed method

• Provide a two-level PA segmentation benchmark (main PA outside lungs; branch PA inside lungs) with weighted scores (80% branch, 20% main).
• Use DSC and HD95 for accuracy; running time and GPU memory for efficiency.
• Organize training, validation, and testing phases with strict non-use of external data or pre-trained models; require Docker submission for final results.
• Compute a four-mMetric ranking by combining weighted DSC, weighted HD95, RT, and GPU into a final score.
• Publish the PARSE dataset with 203 subjects (training, validation, testing) and 1 mm CT slices across multiple centers and vendors.

Experimental results

Research questions

• RQ1 Can automatic segmentation reliably segment both main PA and branch PA in CTPA while maintaining clinical-level efficiency?
• RQ2 What are the most effective strategies to balance accuracy and speed for multi-level vascular segmentation?
• RQ3 How do different algorithmic approaches (e.g., U-Net variants, skeleton-based decoding, and multi-stage pipelines) impact performance on main vs branch PA?
• RQ4 What insights can be drawn about dataset variability, annotation noise, and top-performing techniques to guide future work?

Key findings

• Top methods achieve mean DSC around 89–91% for main PA, and 68–77% for branch PA, with weighted PA DSC around 79–80%.
• HD95 favors branch PA over main PA across teams, highlighting higher difficulty in the main PA region.
• Top performers balance accuracy and efficiency; some prioritize speed (RT) or memory (GPU) while others favor accuracy, illustrating ranking stability depends on metric emphasis.
• Final ranking integrates four metrics (DSC, HD95, RT, GPU) with branch PA weighted more (80%), enabling robust multi-level evaluation.
• There is a trade-off between segmentation quality and inference efficiency; some teams achieve near-best accuracy but at high GPU/memory costs, while others optimize runtime with modest accuracy changes.