[Paper Review] Towards broader generalization of deep learning methods for multiple sclerosis lesion segmentation.

Reda Abdellah Kamraoui, Vinh‐Thong Ta+4 more|arXiv (Cornell University)|Dec 14, 2020

Brain Tumor Detection and Classification4 citations

TL;DR

This paper proposes DeepLesionBrain (DLB), a robust 3D CNN ensemble method for multiple sclerosis lesion segmentation that generalizes across diverse clinical datasets. By combining hierarchical specialization learning, data augmentation for image quality variation, and an ensemble of compact networks, DLB achieves state-of-the-art generalization performance—outperforming prior methods on cross-dataset benchmarks while using only half the modalities.

ABSTRACT

Recently, segmentation methods based on Convolutional Neural Networks (CNNs) showed promising performance in automatic Multiple Sclerosis (MS) lesions segmentation. These techniques have even outperformed human experts in controlled evaluation condition. However state-of-the-art approaches trained to perform well on highly-controlled datasets fail to generalize on clinical data from unseen datasets. Instead of proposing another improvement of the segmentation accuracy, we propose a novel method robust to domain shift and performing well on unseen datasets, called DeepLesionBrain (DLB). This generalization property results from three main contributions. First, DLB is based on a large ensemble of compact 3D CNNs. This ensemble strategy ensures a robust prediction despite the risk of generalization failure of some individual networks. Second, DLB includes a new image quality data augmentation to reduce dependency to training data specificity (e.g., acquisition protocol). Finally, to learn a more generalizable representation of MS lesions, we propose a hierarchical specialization learning (HSL). HSL is performed by pre-training a generic network over the whole brain, before using its weights as initialization to locally specialized networks. By this end, DLB learns both generic features extracted at global image level and specific features extracted at local image level. At the time of publishing this paper, DLB is among the Top 3 performing published methods on ISBI Challenge while using only half of the available modalities. DLB generalization has also been compared to other state-of-the-art approaches, during cross-dataset experiments on MSSEG'16, ISBI challenge, and in-house datasets. DLB improves the segmentation performance and generalization over classical techniques, and thus proposes a robust approach better suited for clinical practice.

Motivation & Objective

• To address the poor generalization of deep learning models on unseen clinical MS datasets despite strong performance on controlled training data.
• To reduce dependency on specific acquisition protocols by introducing image quality-aware data augmentation.
• To improve model robustness by learning both global and local lesion representations through hierarchical specialization.
• To develop a clinically viable segmentation method that performs reliably across diverse, real-world imaging data.

Proposed method

• DLB employs an ensemble of compact 3D convolutional neural networks to enhance prediction robustness against individual network failure.
• A novel image quality data augmentation strategy is applied during training to reduce sensitivity to variations in acquisition protocols.
• Hierarchical Specialization Learning (HSL) first pre-trains a generic 3D CNN on the whole brain, then fine-tunes specialized networks on local lesion regions.
• The pre-trained global network weights are used to initialize local networks, enabling transfer of general brain structure knowledge to lesion-specific features.
• The method leverages only half the available MRI modalities, yet maintains high performance, indicating improved data efficiency.
• Ensemble averaging across multiple specialized networks reduces variance and improves reliability on out-of-distribution data.

Experimental results

Research questions

• RQ1 Can a deep learning model achieve superior generalization across diverse, unseen clinical MS datasets compared to state-of-the-art methods?
• RQ2 How does hierarchical specialization learning improve feature representation for MS lesion segmentation?
• RQ3 To what extent can image quality data augmentation reduce model dependency on specific MRI acquisition protocols?
• RQ4 Does an ensemble of compact 3D CNNs enhance robustness against domain shift in MS lesion segmentation?
• RQ5 Can a method achieve high performance with only half the available MRI modalities while maintaining generalization?

Key findings

• DLB achieved top-3 performance on the ISBI 2015 challenge for MS lesion segmentation while using only half the available MRI modalities.
• On cross-dataset evaluation across MSSEG'16, ISBI challenge, and in-house datasets, DLB demonstrated superior generalization compared to classical state-of-the-art methods.
• The hierarchical specialization learning strategy enabled the model to learn both global brain context and local lesion-specific features, improving robustness.
• Image quality data augmentation significantly reduced model sensitivity to variations in MRI acquisition protocols.
• The ensemble of compact 3D CNNs provided consistent performance gains and reduced prediction variance across different datasets.
• DLB's performance was consistently better than baseline methods even when tested on datasets not seen during training, confirming its strong generalization capability.