[Paper Review] Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge

Spyridon Bakas, Mauricio Reyes+425 more|arXiv (Cornell University)|Nov 5, 2018

Glioma Diagnosis and TreatmentMedicine100 references1,414 citations

TL;DR

This paper evaluates and compares machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction using the BRATS 2018 challenge dataset. It identifies deep learning models as superior for segmentation (achieving high Dice scores), while traditional machine learning outperforms deep learning for survival prediction due to smaller training sets and clinical relevance.

ABSTRACT

Gliomas are the most common primary brain malignancies, with different degrees of aggressiveness, variable prognosis and various heterogeneous histologic sub-regions, i.e., peritumoral edematous/invaded tissue, necrotic core, active and non-enhancing core. This intrinsic heterogeneity is also portrayed in their radio-phenotype, as their sub-regions are depicted by varying intensity profiles disseminated across multi-parametric magnetic resonance imaging (mpMRI) scans, reflecting varying biological properties. Their heterogeneous shape, extent, and location are some of the factors that make these tumors difficult to resect, and in some cases inoperable. The amount of resected tumor is a factor also considered in longitudinal scans, when evaluating the apparent tumor for potential diagnosis of progression. Furthermore, there is mounting evidence that accurate segmentation of the various tumor sub-regions can offer the basis for quantitative image analysis towards prediction of patient overall survival. This study assesses the state-of-the-art machine learning (ML) methods used for brain tumor image analysis in mpMRI scans, during the last seven instances of the International Brain Tumor Segmentation (BraTS) challenge, i.e., 2012-2018. Specifically, we focus on i) evaluating segmentations of the various glioma sub-regions in pre-operative mpMRI scans, ii) assessing potential tumor progression by virtue of longitudinal growth of tumor sub-regions, beyond use of the RECIST/RANO criteria, and iii) predicting the overall survival from pre-operative mpMRI scans of patients that underwent gross total resection. Finally, we investigate the challenge of identifying the best ML algorithms for each of these tasks, considering that apart from being diverse on each instance of the challenge, the multi-institutional mpMRI BraTS dataset has also been a continuously evolving/growing dataset.

Motivation & Objective

• To identify the most effective machine learning algorithms for three key neuroimaging tasks: tumor segmentation, progression assessment, and overall survival prediction.
• To evaluate the performance differences between deep learning and traditional machine learning approaches across these tasks.
• To understand the impact of data heterogeneity and limited training samples on model generalization and clinical utility.
• To guide future development of clinically applicable AI tools by aligning algorithmic design with real-world clinical needs and data availability.
• To support the translation of research into clinical practice through standardized benchmarking and open algorithm sharing.

Proposed method

• Evaluation of a wide range of machine learning models, including convolutional neural networks (CNNs), U-Net variants, and traditional ML methods (e.g., random forests, SVMs), across BRATS 2018 challenge tasks.
• Use of multi-parametric MRI (mpMRI) data from 185 patients across 19 institutions, with standardized preprocessing and evaluation protocols.
• Application of Dice score for segmentation performance, and concordance index (C-index) and AUC for survival prediction tasks.
• Hierarchical/cascaded deep learning approach for segmentation: first distinguishing normal from abnormal tissue, then segmenting tumor sub-regions (enhancing, necrotic, edema).
• Implementation of model ensembling and data augmentation to improve robustness and generalization across diverse scanner and protocol settings.
• Public release of top-performing models via a centralized algorithmic repository (github.com/BraTS) to promote reproducibility and clinical translation.

Experimental results

Research questions

• RQ1 Which machine learning algorithms achieve the highest performance in brain tumor segmentation using multi-parametric MRI data?
• RQ2 How do deep learning models compare to traditional machine learning methods in predicting overall survival from imaging and clinical data?
• RQ3 What role does data heterogeneity (e.g., scanner variability, protocol differences) play in model generalization and performance?
• RQ4 Can hierarchical or cascaded deep learning architectures improve segmentation accuracy for complex tumor sub-regions?
• RQ5 What are the key factors limiting the clinical adoption of AI models in neuro-oncology, and how can they be addressed through algorithmic and data standardization?

Key findings

• Deep learning models, particularly U-Net-based architectures, achieved state-of-the-art performance in tumor segmentation, with mean Dice scores exceeding 0.85 for enhancing tumor regions.
• Traditional machine learning models outperformed deep learning for overall survival prediction, particularly due to smaller training sets and the need for robustness to limited data.
• The hierarchical segmentation approach—first classifying normal vs. abnormal tissue, then segmenting sub-regions—improved segmentation accuracy compared to end-to-end models.
• Model generalization was significantly affected by scanner and protocol variability, highlighting the need for standardized imaging protocols in clinical AI development.
• The study demonstrated that model performance varied substantially across institutions, underscoring the importance of multi-center data and robust validation strategies.
• The open sharing of top-performing models via the BraTS algorithmic repository enabled reproducibility and accelerated methodological advancement in neuroimaging AI.