[Paper Review] DeepLesionBrain: Towards a broader deep-learning generalization for multiple sclerosis lesion segmentation

Reda Abdellah Kamraoui, Vinh‐Thong Ta+4 more|arXiv (Cornell University)|Dec 14, 2020

Digital Imaging for Blood Diseases51 references73 citations

TL;DR

DLB proposes a domain-generalizing MS lesion segmentation framework using spatially distributed 3D CNNs with hierarchical specialization learning, image quality data augmentation, and modality-efficient design (T1w+FLAIR) to improve cross-dataset generalization.

ABSTRACT

Recently, segmentation methods based on Convolutional Neural Networks (CNNs) showed promising performance in automatic Multiple Sclerosis (MS) lesions segmentation. These techniques have even outperformed human experts in controlled evaluation conditions such as Longitudinal MS Lesion Segmentation Challenge (ISBI Challenge). However state-of-the-art approaches trained to perform well on highly-controlled datasets fail to generalize on clinical data from unseen datasets. Instead of proposing another improvement of the segmentation accuracy, we propose a novel method robust to domain shift and performing well on unseen datasets, called DeepLesionBrain (DLB). This generalization property results from three main contributions. First, DLB is based on a large group of compact 3D CNNs. This spatially distributed strategy ensures a robust prediction despite the risk of generalization failure of some individual networks. Second, DLB includes a new image quality data augmentation to reduce dependency to training data specificity (e.g., acquisition protocol). Finally, to learn a more generalizable representation of MS lesions, we propose a hierarchical specialization learning (HSL). HSL is performed by pre-training a generic network over the whole brain, before using its weights as initialization to locally specialized networks. By this end, DLB learns both generic features extracted at global image level and specific features extracted at local image level. DLB generalization was validated in cross-dataset experiments on MSSEG'16, ISBI challenge, and in-house datasets. During experiments, DLB showed higher segmentation accuracy, better segmentation consistency and greater generalization performance compared to state-of-the-art methods. Therefore, DLB offers a robust framework well-suited for clinical practice.

Motivation & Objective

• Address the poor generalization of CNN-based MS lesion segmentation across unseen clinical datasets.
• Propose a domain-generalizable architecture that maintains performance without target-domain annotations.
• Enhance robustness to acquisition variability via novel data augmentation and hierarchical feature learning.

Proposed method

• Introduce a large group of compact, spatially distributed 3D CNNs each specialized to a brain region with overlapping receptive fields.
• Apply Image Quality Data Augmentation (IQDA) during training to simulate acquisition and quality variability (blur, edge enhancement, axial subsampling).
• Employ Hierarchical Specialization Learning (HSL): first train a generic network on all brain regions to capture global features, then initialize region-specific networks with the generic weights for local specialization.
• Use a two-modality input setup (T1w and FLAIR) to maximize compatibility across datasets and avoid reliance on additional sequences.
• Train and evaluate on cross-dataset MS segmentation benchmarks to assess domain generalization (ISBI, MSSEG’16, in-house data).
• Optimize with a smooth Generalized Jaccard Loss to address class imbalance and aggregate overlapped region predictions via majority voting.

Experimental results

Research questions

• RQ1 Can DeepLesionBrain achieve robust MS lesion segmentation on unseen domains without target-domain annotations?
• RQ2 Does spatially distributed networking with overlapping receptive fields improve generalization compared to single-network approaches?
• RQ3 Do IQDA and Hierarchical Specialization Learning contribute meaningfully to cross-domain robustness and lesion detection accuracy?
• RQ4 Is reliable MS lesion segmentation feasible using only two MRI modalities (T1w and FLAIR) across datasets with varied acquisition protocols?

Key findings

• DLB achieves higher segmentation accuracy, consistency, and generalization performance compared to state-of-the-art methods in cross-dataset tests.
• The spatially distributed networks with overlapping regions provide a robust consensus, improving stability under domain shifts.
• IQDA reduces sensitivity to training-data-specific acquisition and quality variations, aiding domain generalization.
• HSL enables effective learning of generic global features and specialized local features, outperforming single-network or non-HSL baselines.
• DLB demonstrates compatibility with two modalities (T1w and FLAIR) while maintaining robust performance across diverse datasets.