[Paper Review] Multi-stream 3D FCN with Multi-scale Deep Supervision for Multi-modality Isointense Infant Brain MR Image Segmentation

Guodong Zeng, Guoyan Zheng|arXiv (Cornell University)|Nov 28, 2017

Advanced Neural Network Applications20 references3 citations

TL;DR

This paper proposes a multi-stream 3D fully convolutional network with multi-scale deep supervision and context guidance for segmenting multi-modality isointense infant brain MR images into CSF, GM, and WM. The method achieves state-of-the-art performance with 95.4% Dice score for CSF, 91.6% for GM, and 89.6% for WM on the iSeg-2017 test set, demonstrating robustness to low tissue contrast in 6-month-old infants.

ABSTRACT

We present a method to address the challenging problem of segmentation of multi-modality isointense infant brain MR images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF). Our method is based on context-guided, multi-stream fully convolutional networks (FCN), which after training, can directly map a whole volumetric data to its volume-wise labels. In order to alleviate the poten-tial gradient vanishing problem during training, we designed multi-scale deep supervision. Furthermore, context infor-mation was used to further improve the performance of our method. Validated on the test data of the MICCAI 2017 Grand Challenge on 6-month infant brain MRI segmentation (iSeg-2017), our method achieved an average Dice Overlap Coefficient of 95.4%, 91.6% and 89.6% for CSF, GM and WM, respectively.

Motivation & Objective

• To address the challenge of segmenting multi-modality isointense infant brain MR images, where T1 and T2 contrast is minimal due to ongoing myelination.
• To improve segmentation accuracy in the absence of strong tissue contrast by leveraging multi-stream 3D fully convolutional networks (FCN).
• To mitigate gradient vanishing during training using multi-scale deep supervision, especially with limited labeled data.
• To enhance performance by integrating spatial context information into the network architecture.
• To achieve high-accuracy, end-to-end volumetric segmentation of CSF, GM, and WM in 6-month-old infant brains.

Proposed method

• Uses a multi-stream 3D FCN that processes T1 and T2 MR images independently before fusing features for segmentation.
• Applies multi-scale deep supervision by adding auxiliary classifiers at multiple decoder stages to stabilize training and improve gradient flow.
• Incorporates context-guided feature learning to enhance spatial consistency and reduce noise in predictions.
• Employs partial transfer learning to accelerate training on small datasets, initializing with pre-trained ImageNet weights.
• Uses overlapping 64×64×64 patches for inference, with final probability maps obtained by averaging overlapping predictions.
• Applies morphological operations to refine results by removing isolated voxels and filling small holes.

Experimental results

Research questions

• RQ1 Can a multi-stream 3D FCN with multi-scale deep supervision achieve robust segmentation of isointense infant brain MR images with low tissue contrast?
• RQ2 How does multi-scale deep supervision improve training stability and segmentation accuracy in low-data regimes?
• RQ3 To what extent does incorporating spatial context information enhance the precision of tissue segmentation in 3D volumes?
• RQ4 How does the proposed method compare to state-of-the-art approaches in the iSeg-2017 challenge for 6-month infant brain MRI segmentation?
• RQ5 Can context-guided feature learning improve boundary delineation and reduce false positives in CSF and GM segmentation?

Key findings

• The proposed method achieved a mean Dice Overlap Coefficient (DOC) of 95.4% for CSF, 91.6% for GM, and 89.6% for WM on the iSeg-2017 test set.
• Adding context information improved the DOC by 0.4% for CSF, 0.5% for GM, and 0.8% for WM compared to the baseline without context.
• The method ranked 3rd out of 21 teams in the iSeg-2017 challenge, with performance very close to the top two teams.
• The Average Surface Distance (ASD) was 0.127 mm for CSF, 0.341 mm for GM, and 0.40 mm for WM, indicating high spatial accuracy.
• The Modified Hausdorff Distance (MHD) was 9.62 mm for CSF, 6.46 mm for GM, and 6.78 mm for WM, showing robustness to outliers.
• The method required only 8 seconds per subject on a standard desktop GPU, demonstrating efficient inference.