[Paper Review] Benchmarking the CoW with the TopCoW Challenge: Topology-Aware Anatomical Segmentation of the Circle of Willis for CTA and MRA

Kaiyuan Yang, Fabio Musio+111 more|PubMed|Dec 29, 2023

Cerebrovascular and Carotid Artery Diseases45 references9 citations

TL;DR

The paper presents the TopCoW challenge and dataset for topology-aware multiclass segmentation of the Circle of Willis in paired CTA and MRA images, reporting near 90% Dice on many components and analyzing topology-based metrics.

ABSTRACT

The Circle of Willis (CoW) is an important network of arteries connecting major circulations of the brain. Its vascular architecture is believed to affect the risk, severity, and clinical outcome of serious neurovascular diseases. However, characterizing the highly variable CoW anatomy is still a manual and time-consuming expert task. The CoW is usually imaged by two non-invasive angiographic imaging modalities, magnetic resonance angiography (MRA) and computed tomography angiography (CTA), but there exist limited datasets with annotations on CoW anatomy, especially for CTA. Therefore, we organized the TopCoW challenge with the release of an annotated CoW dataset. The TopCoW dataset is the first public dataset with voxel-level annotations for 13 CoW vessel components, enabled by virtual reality technology. It is also the first large dataset using 200 pairs of MRA and CTA from the same patients. As part of the benchmark, we invited submissions worldwide and attracted over 250 registered participants from six continents. The submissions were evaluated on both internal and external test datasets of 226 scans from over five centers. The top performing teams achieved over 90% Dice scores at segmenting the CoW components, over 80% F1 scores at detecting key CoW components, and over 70% balanced accuracy at classifying CoW variants for nearly all test sets. The best algorithms also showed clinical potential in classifying fetal-type posterior cerebral artery and locating aneurysms with CoW anatomy. TopCoW demonstrated the utility and versatility of CoW segmentation algorithms for a wide range of downstream clinical applications with explainability. The annotated datasets and best performing algorithms have been released as public Zenodo records to foster further methodological development and clinical tool building.

Motivation & Objective

• Provide a public, voxel-level annotated dataset of Circle of Willis anatomy for both CTA and MRA.
• Formalize CoW segmentation as a multiclass task with topology-aware evaluation.
• Evaluate and benchmark global and topological performance of automated segmentation methods.
• Analyze inter-rater reliability and modality-specific agreements across MRA and CTA.
• Highlight open challenges and future directions for CoW topology-aware analysis.

Proposed method

• Release a paired CTA–MRA dataset with voxel-level multiclass annotations for thirteen CoW vessel components.
• Annotate using VR to enable efficient 3D labeling and verification by clinical experts.
• Define a CoW ROI and perform segmentation within this ROI for both modalities.
• Evaluate submissions using Dice, centerline Dice (clDice), and Betti-0 errors to capture morphology and topology.
• Provide both multiclass and binary segmentation tasks across two modality tracks (CTA and MRA).
• Allow external training data and report inter-modality and inter-rater analyses as part of the benchmark.

Experimental results

Research questions

• RQ1 Can automated methods accurately segment thirteen CoW vessel components in both MRA and CTA while preserving topological integrity?
• RQ2 How do voxel-level segmentation performance and topology-based metrics compare across modalities and variant CoW anatomies?
• RQ3 What are the inter-rater reliability levels for multiclass CoW annotations in VR-based labeling?
• RQ4 What are the common failure modes in topology matching for CoW variants in automated predictions?

Key findings

• TopCoW attracted 146 registered participants across four continents and 27 teams, with 18 teams contributing to the final challenge paper.
• TopCoW training set includes 90 patients, validation set 5, and test set 35, totaling 130 paired MRA–CTA cases.
• Inter-rater Dice on a subset of five test cases averaged about 90%+ for most of the 13 classes, with lower scores for R-Pcom, L-Pcom, Acom, and 3rd-A2.
• Binary Dice between raters averaged around 95%, indicating strong overall agreement on the merged CoW mask.
• Inter-modality agreement was high for anterior variants, and more variable for posterior variants due to CTA proximity to bone regions.
• Top-performing methods achieved Dice scores around 90% for many vessel components, but showed weaknesses in communicating arteries and rare variants, and topology errors even with high Dice.