[Paper Review] DeepMRSeg: A convolutional deep neural network for anatomy and abnormality segmentation on MR images

Jimit Doshi, Güray Erus+2 more|arXiv (Cornell University)|Jul 3, 2019

Medical Image Segmentation Techniques18 references25 citations

TL;DR

DeepMRSeg is a modified UNet architecture with residual and multi-scale inception blocks that enables accurate, generic segmentation of brain anatomy and abnormalities directly from raw T1-weighted MRI scans. It outperforms standard UNet across multiple tasks—white matter lesions, deep brain structures, and hippocampal subregions—achieving significantly higher overlap metrics and balanced accuracy with minimal preprocessing.

ABSTRACT

Segmentation has been a major task in neuroimaging. A large number of automated methods have been developed for segmenting healthy and diseased brain tissues. In recent years, deep learning techniques have attracted a lot of attention as a result of their high accuracy in different segmentation problems. We present a new deep learning based segmentation method, DeepMRSeg, that can be applied in a generic way to a variety of segmentation tasks. The proposed architecture combines recent advances in the field of biomedical image segmentation and computer vision. We use a modified UNet architecture that takes advantage of multiple convolution filter sizes to achieve multi-scale feature extraction adaptive to the desired segmentation task. Importantly, our method operates on minimally processed raw MRI scan. We validated our method on a wide range of segmentation tasks, including white matter lesion segmentation, segmentation of deep brain structures and hippocampus segmentation. We provide code and pre-trained models to allow researchers apply our method on their own datasets.

Motivation & Objective

• To develop a generic, deep learning-based segmentation framework applicable to diverse neuroimaging tasks without task-specific re-engineering.
• To address the limitations of traditional methods requiring extensive preprocessing and manual feature engineering.
• To enable high-accuracy segmentation using only minimally processed raw T1 MRI scans.
• To provide a plug-and-play solution for researchers via open-source code and pre-trained models.

Proposed method

• The method employs a modified UNet architecture integrating residual connections (ResNet) and multi-branch inception blocks with varying kernel sizes for multi-scale feature extraction.
• Max-pooling layers are replaced with 1x1 convolutional filters to preserve fine spatial details and boundary accuracy.
• The network is trained end-to-end on raw T1-weighted MRI scans without extensive pre-processing, enabling direct application to new datasets.
• The architecture combines encoding and decoding paths with skip connections to retain spatial context and localization precision.
• It leverages transfer learning principles by using pre-trained components from image classification (Inception-ResNet-A) adapted for medical segmentation.
• The model is fine-tuned on task-specific data, allowing deployment across diverse segmentation tasks with minimal reconfiguration.

Experimental results

Research questions

• RQ1 Can a single deep learning architecture achieve high segmentation accuracy across multiple neuroanatomical and pathological targets without task-specific redesign?
• RQ2 How does multi-scale feature extraction using dilated and parallel convolutional branches improve segmentation performance on diverse MRI tasks?
• RQ3 To what extent does replacing max-pooling with learnable 1x1 convolutions enhance boundary detail preservation in brain segmentation?
• RQ4 Can a model trained on raw T1 scans achieve state-of-the-art performance without extensive pre-processing or modality-specific tuning?
• RQ5 How does DeepMRSeg compare to standard UNet in terms of segmentation accuracy and robustness across different brain structures and pathologies?

Key findings

• DeepMRSeg achieved significantly higher balanced accuracy (BACC) than standard UNet across all deep brain structures, with an average BACC of 0.930 compared to 0.906.
• For hippocampal sub-regions, DeepMRSeg improved mean BACC from 0.913 to 0.922 and F1-score from 0.857 to 0.862.
• In white matter lesion segmentation, DeepMRSeg demonstrated superior performance, with higher overlap metrics and improved sensitivity to small, irregular lesions.
• The model achieved a mean weighted F1-score of 0.858 across all deep brain structures, outperforming UNet’s 0.842.
• The use of 1x1 convolutions instead of max-pooling reduced boundary blur and improved localization accuracy, especially in small, complex structures like the amygdala and hippocampus.
• Pre-trained models and code are publicly available via the IPP platform (https://ipp.cbica.upenn.edu/), enabling direct deployment on new datasets without local installation.