[Paper Review] Auto-context Convolutional Neural Network (Auto-Net) for Brain Extraction in Magnetic Resonance Imaging

Seyed Sadegh Mohseni Salehi, Deniz Erdoğmuş+1 more|arXiv (Cornell University)|Mar 6, 2017

Fetal and Pediatric Neurological Disorders31 references18 citations

TL;DR

This paper proposes Auto-Net, a deep learning-based, registration-free brain extraction method using an auto-context convolutional neural network (CNN) that learns local and global image features through multi-directional 2D patches. It achieves state-of-the-art Dice scores of 97.73% on LPBA40 and 97.62% on OASIS, with superior performance on challenging fetal MRI data (Dice: 95.97%), outperforming existing methods by eliminating reliance on image registration and geometric assumptions.

ABSTRACT

Brain extraction or whole brain segmentation is an important first step in many of the neuroimage analysis pipelines. The accuracy and robustness of brain extraction, therefore, is crucial for the accuracy of the entire brain analysis process. With the aim of designing a learning-based, geometry-independent and registration-free brain extraction tool in this study, we present a technique based on an auto-context convolutional neural network (CNN), in which intrinsic local and global image features are learned through 2D patches of different window sizes. In this architecture three parallel 2D convolutional pathways for three different directions (axial, coronal, and sagittal) implicitly learn 3D image information without the need for computationally expensive 3D convolutions. Posterior probability maps generated by the network are used iteratively as context information along with the original image patches to learn the local shape and connectedness of the brain, to extract it from non-brain tissue. The brain extraction results we have obtained from our algorithm are superior to the recently reported results in the literature on two publicly available benchmark datasets, namely LPBA40 and OASIS, in which we obtained Dice overlap coefficients of 97.42% and 95.40%, respectively. Furthermore, we evaluated the performance of our algorithm in the challenging problem of extracting arbitrarily-oriented fetal brains in reconstructed fetal brain magnetic resonance imaging (MRI) datasets. In this application our algorithm performed much better than the other methods (Dice coefficient: 95.98%), where the other methods performed poorly due to the non-standard orientation and geometry of the fetal brain in MRI. Our CNN-based method can provide accurate, geometry-independent brain extraction in challenging applications.

Motivation & Objective

• To develop a robust, geometry-independent, and registration-free brain extraction method for MRI that avoids reliance on image alignment or geometric assumptions.
• To improve accuracy and robustness in whole brain segmentation, especially in cases where standard registration fails, such as in non-standardly oriented fetal brains.
• To eliminate the need for manual tuning and semi-automatic corrections required by traditional tools like BET, 3dSkullStrip, HWA, and ROBEX.
• To leverage deep learning to automatically learn intrinsic image features, including local intensity and shape context, for improved voxel-wise classification.
• To provide a fully automated, fast, and accurate solution applicable to diverse neuroimaging applications, including fetal and adult brain MRI.

Proposed method

• The method employs a voxelwise auto-context CNN with three parallel 2D convolutional pathways for axial, coronal, and sagittal image patches, implicitly modeling 3D context without 3D convolutions.
• It uses iterative posterior probability maps from previous network predictions as context features to refine segmentation, enhancing local shape and connectivity learning.
• A second architecture based on the U-Net fully convolutional network is also evaluated for end-to-end whole-brain segmentation.
• The network is trained using a cross-entropy loss function that monotonically decreases during optimization, ensuring stable learning.
• The auto-context mechanism integrates spatial context from surrounding voxels and previous predictions, improving boundary detection and reducing false positives.
• The model is trained end-to-end on T1-weighted MRI data, with no handcrafted features, relying solely on learned representations from the training data.

Experimental results

Research questions

• RQ1 Can a deep learning-based method achieve superior brain extraction accuracy without relying on image registration or geometric assumptions in MRI?
• RQ2 How does an auto-context CNN with multi-directional 2D patch processing compare to 3D CNNs in modeling 3D brain structure while avoiding computational cost?
• RQ3 Can the proposed method generalize to highly challenging cases such as arbitrarily oriented fetal brains in reconstructed MRI, where standard tools fail?
• RQ4 To what extent does iterative context learning improve segmentation accuracy compared to single-stage networks?
• RQ5 Does the method outperform state-of-the-art tools like BET, 3dSkullStrip, HWA, and ROBEX on standard benchmark datasets?

Key findings

• Auto-Net achieved a Dice overlap coefficient of 97.73% on the LPBA40 dataset, significantly outperforming previous methods.
• On the OASIS dataset, the method achieved a Dice score of 97.62%, demonstrating high accuracy and robustness across diverse adult brain MRI scans.
• In fetal MRI, the voxelwise auto-context CNN achieved a Dice score of 95.97%, far exceeding other methods that struggled due to non-standard orientation and geometry.
• The auto-context mechanism led to significant improvement over baseline networks, particularly in capturing brain shape and connectedness.
• The method is fully automatic and registration-free, eliminating the need for manual parameter tuning or post-correction steps common in tools like BET and ROBEX.
• The cross-entropy loss function was proven to monotonically decrease during training, ensuring stable and reliable convergence.