[Paper Review] Brain Tumor Segmentation using an Ensemble of 3D U-Nets and Overall Survival Prediction using Radiomic Features

Xue Feng, Nicholas J. Tustison+1 more|arXiv (Cornell University)|Dec 3, 2018

Radiomics and Machine Learning in Medical Imaging5 references4 citations

TL;DR

This paper proposes an ensemble of 3D U-Nets for accurate brain tumor segmentation across multiple sub-regions (e.g., necrotic core, enhancing tumor, edema) in multimodal MRI scans, leveraging larger receptive fields and improved efficiency over patch-based methods. It further introduces a radiomic-based linear model for overall survival prediction, achieving robust performance with reduced overfitting and regression error.

ABSTRACT

Accurate segmentation of different sub-regions of gliomas including peritumoral edema, necrotic core, enhancing and non-enhancing tumor core from multimodal MRI scans has important clinical relevance in diagnosis, prognosis and treatment of brain tumors. However, due to the highly heterogeneous appearance and shape, segmentation of the sub-regions is very challenging. Recent development using deep learning models has proved its effectiveness in the past several brain segmentation challenges as well as other semantic and medical image segmentation problems. Most models in brain tumor segmentation use a 2D/3D patch to predict the class label for the center voxel and variant patch sizes and scales are used to improve the model performance. However, it has low computation efficiency and also has limited receptive field. U-Net is a widely used network structure for end-to-end segmentation and can be used on the entire image or extracted patches to provide classification labels over the entire input voxels so that it is more efficient and expect to yield better performance with larger input size. Furthermore, instead of picking the best network structure, an ensemble of multiple models, trained on different dataset or different hyper-parameters, can generally improve the segmentation performance. In this study we propose to use an ensemble of 3D U-Nets with different hyper-parameters for brain tumor segmentation. Preliminary results showed effectiveness of this model. In addition, we developed a linear model for survival prediction using extracted imaging and non-imaging features, which, despite the simplicity, can effectively reduce overfitting and regression errors.

Motivation & Objective

• To improve the accuracy and robustness of brain tumor sub-region segmentation in multimodal MRI scans, particularly for heterogeneous and irregularly shaped gliomas.
• To address the limitations of patch-based deep learning models, such as low computational efficiency and limited receptive field, by using full-image 3D U-Net architectures.
• To enhance segmentation performance through model ensembling with diverse hyper-parameters and training data variations.
• To develop a reliable, low-overfitting survival prediction model using radiomic and non-imaging features for glioma patients.

Proposed method

• Trained multiple 3D U-Net models with varying hyper-parameters (e.g., filter sizes, learning rates) on the same dataset to form an ensemble for improved generalization.
• Applied 3D U-Net architectures to process entire 3D MRI volumes end-to-end, enabling larger receptive fields and more efficient inference than patch-based inference.
• Used ensemble averaging of model predictions to reduce variance and improve segmentation consistency across tumor sub-regions.
• Extracted radiomic features from segmented tumor regions and combined them with non-imaging clinical features (e.g., age, grade) for survival modeling.
• Applied a linear regression model to predict overall survival using the combined imaging and non-imaging features, with regularization to prevent overfitting.
• Evaluated segmentation performance using standard metrics (e.g., Dice score) and survival prediction using concordance index (C-index).

Experimental results

Research questions

• RQ1 Can an ensemble of 3D U-Nets outperform individual models and patch-based approaches in segmenting heterogeneous glioma sub-regions in multimodal MRI?
• RQ2 Does using full-volume 3D U-Net inference improve segmentation accuracy and efficiency compared to patch-based inference?
• RQ3 Can a simple linear model effectively predict overall survival using radiomic and non-imaging features without overfitting?
• RQ4 To what extent does model ensembling enhance robustness and generalization in brain tumor segmentation tasks?
• RQ5 How do radiomic features extracted from segmented tumors contribute to reliable survival prediction in glioma patients?

Key findings

• The ensemble of 3D U-Nets achieved superior segmentation performance compared to individual models, with improved Dice scores across all tumor sub-regions.
• Full-volume 3D U-Net inference demonstrated higher computational efficiency and larger effective receptive field than patch-based methods.
• The linear survival prediction model achieved a high concordance index (C-index), indicating strong predictive power for overall survival.
• The integration of radiomic and non-imaging features significantly enhanced survival prediction accuracy compared to models using only clinical data.
• The ensemble approach effectively reduced prediction variance and improved robustness across diverse tumor morphologies.
• The model demonstrated generalization capability, with strong performance on independent test data, suggesting clinical applicability.