[Paper Review] The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification

Ujjwal Baid, Satyam Ghodasara+101 more|arXiv (Cornell University)|Jul 5, 2021

Radiomics and Machine Learning in Medical Imaging24 references88 citations

TL;DR

BraTS 2021 benchmarks segmentation of glioma sub-regions and MGMT promoter methylation status from pre-operative mpMRI across 2,040 patients, via Task 1 segmentation and Task 2 radiogenomic classification.

ABSTRACT

The BraTS 2021 challenge celebrates its 10th anniversary and is jointly organized by the Radiological Society of North America (RSNA), the American Society of Neuroradiology (ASNR), and the Medical Image Computing and Computer Assisted Interventions (MICCAI) society. Since its inception, BraTS has been focusing on being a common benchmarking venue for brain glioma segmentation algorithms, with well-curated multi-institutional multi-parametric magnetic resonance imaging (mpMRI) data. Gliomas are the most common primary malignancies of the central nervous system, with varying degrees of aggressiveness and prognosis. The RSNA-ASNR-MICCAI BraTS 2021 challenge targets the evaluation of computational algorithms assessing the same tumor compartmentalization, as well as the underlying tumor's molecular characterization, in pre-operative baseline mpMRI data from 2,040 patients. Specifically, the two tasks that BraTS 2021 focuses on are: a) the segmentation of the histologically distinct brain tumor sub-regions, and b) the classification of the tumor's O[6]-methylguanine-DNA methyltransferase (MGMT) promoter methylation status. The performance evaluation of all participating algorithms in BraTS 2021 will be conducted through the Sage Bionetworks Synapse platform (Task 1) and Kaggle (Task 2), concluding in distributing to the top ranked participants monetary awards of $60,000 collectively.

Motivation & Objective

• Provide a standardized, multi-institution mpMRI dataset for brain glioma segmentation and MGMT promoter methylation prediction.
• Establish robust annotation and preprocessing pipelines to enable fair cross-method comparison.
• Evaluate state-of-the-art segmentation methods and radiogenomic classification approaches on preoperative MRI.

Proposed method

• Multi-institutional mpMRI data (T1, T1Gd, T2, T2-FLAIR) with standardized preprocessing (DICOM→NIfTI, co-registration to SRI24, 1 mm3 resampling, skull-stripping).
• Ground-truth tumor sub-region annotations generated via STAPLE fusion of nnU-Net, DeepScan, and DeepMedic, followed by expert neuroradiologist refinement.
• Task 1 evaluation using Dice similarity, Hausdorff distance (95%), Sensitivity, Specificity for ET, TC, WT regions.
• Task 2 MGMT promoter methylation status prediction evaluated with AUC, accuracy, F1-score, and Matthew’s Correlation Coefficient.
• MGMT labels provided as binary ground truth; preprocessing and conversion maintain patient-space integrity; data handling restricts external data use for fair ranking.

Experimental results

Research questions

• RQ1 Can automated methods accurately segment glioma sub-regions (ET, TC, WT) in pre-operative mpMRI across diverse institutions?
• RQ2 Can MGMT promoter methylation status be predicted from pre-operative mpMRI using radiogenomic techniques?
• RQ3 How do segmentation and radiogenomic classification methods generalize to out-of-distribution test cohorts?
• RQ4 What preprocessing, annotation, and evaluation protocols support fair comparison across competing methods?

Key findings

• The BraTS 2021 dataset comprises 2,000 glioma cases with 8,000 mpMRI scans from multiple institutions.
• Two tasks are defined: tumor sub-region segmentation (ET, TC, WT) and MGMT promoter methylation classification.
• Ground-truth segmentations are produced by STAPLE fusion of top BraTS methods (nnU-Net, DeepScan, DeepMedic) and refined by expert neuroradiologists.
• Common evaluation metrics for Task 1 include Dice, 95th percentile Hausdorff distance, Sensitivity, and Specificity.
• Task 2 evaluation uses AUC, accuracy, F1-score, and Matthews Correlation Coefficient to assess MGMT status prediction.
• Publicly available preprocessing pipelines (CaPTk, FeTS) support reproducible data processing and annotation.
• Participants’ methods are tested on out-of-distribution data to assess generalizability.