[Paper Review] Technical Considerations for Semantic Segmentation in MRI using Convolutional Neural Networks

Arjun Desai, Garry E. Gold+2 more|arXiv (Cornell University)|Feb 5, 2019

Osteoarthritis Treatment and Mechanisms43 references17 citations

TL;DR

This study investigates the impact of network architecture, loss functions, and training data characteristics on deep learning-based semantic segmentation of femoral cartilage in 3D MRI. It demonstrates that while architectural differences yield minimal accuracy gains, proper loss function selection and realistic data augmentation significantly improve generalizability, with performance scaling slowly with data size—requiring ~300–440 patients to reach 95% Dice score across models.

ABSTRACT

High-fidelity semantic segmentation of magnetic resonance volumes is critical for estimating tissue morphometry and relaxation parameters in both clinical and research applications. While manual segmentation is accepted as the gold-standard, recent advances in deep learning and convolutional neural networks (CNNs) have shown promise for efficient automatic segmentation of soft tissues. However, due to the stochastic nature of deep learning and the multitude of hyperparameters in training networks, predicting network behavior is challenging. In this paper, we quantify the impact of three factors associated with CNN segmentation performance: network architecture, training loss functions, and training data characteristics. We evaluate the impact of these variations on the segmentation of femoral cartilage and propose potential modifications to CNN architectures and training protocols to train these models with confidence.

Motivation & Objective

• To quantify the impact of network architecture, training loss functions, and data characteristics on CNN-based semantic segmentation of femoral cartilage in 3D MRI.
• To evaluate how variations in these factors affect segmentation accuracy and generalizability across different fields of view (FOVs).
• To guide confident deployment of CNN models by identifying optimal architectural and training protocol choices for medical image segmentation.

Proposed method

• The study used 3D sagittal DESS MRI data from 88 OAI patients with manually segmented femoral cartilage masks.
• Three U-Net, SegNet, and DeepLabV3+ architectures were trained and evaluated using standard metrics including Dice coefficient and Hausdorff distance.
• Training was conducted with three loss functions: standard cross-entropy, weighted cross-entropy, and soft Dice loss, to assess their impact on error distribution and performance.
• Data augmentation strategies, including FOV variation and contrast modulation, were applied to assess their effect on model generalizability.
• Performance was evaluated on test sets with varying FOVs to measure sensitivity to image acquisition variability.
• A power-law scaling model was fitted to performance data to estimate required training data sizes for target accuracy thresholds.

Experimental results

Research questions

• RQ1 How do different CNN architectures (U-Net, SegNet, DeepLabV3+) affect segmentation accuracy and generalizability in 3D MRI of femoral cartilage?
• RQ2 How do different loss functions (cross-entropy, weighted cross-entropy, soft Dice) influence network optimization and error distribution in the presence of class imbalance?
• RQ3 To what extent does data augmentation, particularly FOV and contrast variation, improve model generalizability across diverse MRI acquisition parameters?
• RQ4 How does model performance scale with increasing training data size, and what data volume is needed to achieve high accuracy (e.g., 95% Dice)?
• RQ5 Can generalizability be improved at the cost of absolute performance on a single test set through strategic augmentation?

Key findings

• Network architecture had minimal impact on overall segmentation accuracy, with no single architecture consistently outperforming others across all metrics.
• The soft Dice loss function led to the highest mean Dice coefficient (0.84) and lowest Hausdorff distance (1.8 mm), indicating superior performance and boundary precision.
• Realistic data augmentation, including FOV and contrast variation, improved model generalizability but slightly reduced absolute performance on the original test set.
• Performance scaled slowly with data size, following a power-law relationship with exponents β < 0.05, indicating diminishing returns with additional data.
• Achieving 95% Dice accuracy would require approximately 300 patients for DeepLabV3+, 350 for U-Net, and 440 for SegNet, suggesting high performance is attainable even with limited data.
• The 3D U-Net with fixed input depth of 32 slices limited batch size and filter count, suggesting potential for performance gains through architectural tuning of input depth and batch size.