[Paper Review] A Multi-task Contextual Atrous Residual Network for Brain Tumor Detection & Segmentation

Ngan Le, Kashu Yamazaki+3 more|arXiv (Cornell University)|Dec 3, 2020

Advanced Neural Network Applications33 references4 citations

TL;DR

This paper proposes a multi-task 3D contextual atrous residual network that jointly detects and segments brain tumors in MRI scans, using a cascaded architecture with a contextual detection head to focus on tumor-relevant regions and a 3D atrous residual U-Net for precise segmentation. The method achieves state-of-the-art performance on BRATS 2015, 2017, and 2018 datasets, with Dice scores of 90.4% (WT), 82.3% (TC), and 72.4% (ET) on BRATS 2018, outperforming prior approaches by effectively addressing class imbalance and enhancing feature learning through dilated convolutions and skip connections.

ABSTRACT

In recent years, deep neural networks have achieved state-of-the-art performance in a variety of recognition and segmentation tasks in medical imaging including brain tumor segmentation. We investigate that segmenting a brain tumor is facing to the imbalanced data problem where the number of pixels belonging to the background class (non tumor pixel) is much larger than the number of pixels belonging to the foreground class (tumor pixel). To address this problem, we propose a multi-task network which is formed as a cascaded structure. Our model consists of two targets, i.e., (i) effectively differentiate the brain tumor regions and (ii) estimate the brain tumor mask. The first objective is performed by our proposed contextual brain tumor detection network, which plays a role of an attention gate and focuses on the region around brain tumor only while ignoring the far neighbor background which is less correlated to the tumor. The second objective is built upon a 3D atrous residual network and under an encode-decode network in order to effectively segment both large and small objects (brain tumor). Our 3D atrous residual network is designed with a skip connection to enables the gradient from the deep layers to be directly propagated to shallow layers, thus, features of different depths are preserved and used for refining each other. In order to incorporate larger contextual information from volume MRI data, our network utilizes the 3D atrous convolution with various kernel sizes, which enlarges the receptive field of filters. Our proposed network has been evaluated on various datasets including BRATS2015, BRATS2017 and BRATS2018 datasets with both validation set and testing set. Our performance has been benchmarked by both region-based metrics and surface-based metrics. We also have conducted comparisons against state-of-the-art approaches.

Motivation & Objective

• To address the severe class imbalance in brain tumor segmentation, where tumor pixels are vastly outnumbered by background pixels.
• To improve segmentation accuracy by focusing computation on tumor-relevant regions through a contextual detection network that avoids processing irrelevant background.
• To preserve and refine features across multiple depths by using skip connections in a 3D residual network to mitigate vanishing gradient problems.
• To incorporate larger contextual information from 3D MRI volumes using 3D atrous convolutions with varying kernel sizes to expand receptive fields without increasing parameters.
• To achieve state-of-the-art performance on benchmark datasets (BRATS 2015, 2017, 2018) using both area-based (Dice) and surface-based (Hausdorff, ASSD) metrics.

Proposed method

• The network uses a cascaded multi-task structure: the first task is a contextual brain tumor detection network that processes only regions surrounding ground-truth tumor instances, acting as an attention gate to suppress irrelevant background.
• The second task employs a 3D atrous residual U-Net for segmentation, leveraging dilated convolutions with multiple kernel sizes to capture long-range context while maintaining high-resolution features.
• Skip connections are integrated into the 3D residual blocks to enable direct gradient flow from deep to shallow layers, preserving hierarchical feature representations.
• 3D atrous convolutions are applied across all encoder and decoder blocks to expand the receptive field and assimilate contextual information from volumetric MRI data.
• The model is trained end-to-end with a combined loss function that optimizes both detection and segmentation objectives simultaneously.
• Region proposals are generated only from contextual regions around true tumor instances, reducing computational cost and improving proposal quality compared to full-image processing.

Experimental results

Research questions

• RQ1 Can a multi-task network with a dedicated contextual detection head improve segmentation performance by reducing computation on irrelevant background regions?
• RQ2 How does the use of 3D atrous convolutions with varying kernel sizes affect the model's ability to capture long-range context in volumetric MRI data?
• RQ3 To what extent do skip connections in a 3D residual network mitigate vanishing gradient issues during training of deep architectures for medical image segmentation?
• RQ4 Does the proposed method outperform state-of-the-art models on standard brain tumor segmentation benchmarks (BRATS 2015, 2017, 2018) in terms of both Dice score and surface-based metrics?
• RQ5 How significant is the contribution of the contextual detection network to overall performance, as measured by ablation studies?

Key findings

• On the BRATS 2018 testing set, the proposed method achieved a Dice score of 90.4% for Whole Tumor (WT), 82.3% for Tumor Core (TC), and 72.4% for Enhancing Tumor (ET), outperforming state-of-the-art methods.
• On the BRATS 2017 online testing set, the model achieved a Dice score of 90.0% (WT), 85.0% (TC), and 63.0% (ET), demonstrating strong generalization across datasets.
• On the BRATS 2015 online testing set, the model achieved a Dice score of 89.0% (WT), 85.0% (TC), and 63.0% (ET), showing competitive performance against top models.
• Ablation studies on BRATS 2018 validation set showed that including the contextual detection network improved Dice scores from 90.25% (WT) to 90.95% (WT), 85.78% to 88.88% (TC), and 79.59% to 81.40% (ET), confirming its positive contribution.
• The model achieved a Hausdorff distance of 3.73 for WT and 2.70 for ET on BRATS 2018, indicating low maximum surface distance and high boundary accuracy.
• The average symmetric surface distance (ASSD) was 3.73 for WT and 2.70 for ET, showing precise alignment between predicted and ground-truth tumor boundaries.