[Paper Review] Mixed-Supervised Dual-Network for Medical Image Segmentation

Duo Wang, Ming Li+5 more|arXiv (Cornell University)|Jul 24, 2019

Advanced Neural Network Applications18 references4 citations

TL;DR

This paper proposes Mixed-Supervised Dual-Network (MSDN), a dual-stream architecture that jointly trains a segmentation network and a detection network via Squeeze-and-Excitation (sSE) modules to transfer knowledge from weakly-labeled bounding boxes to improve dense segmentation. MSDN outperforms multiple baselines on lung nodule and cochlea segmentation tasks, especially under low-supervision regimes, demonstrating robustness and stability with mixed-supervised learning.

ABSTRACT

Deep learning based medical image segmentation models usually require large datasets with high-quality dense segmentations to train, which are very time-consuming and expensive to prepare. One way to tackle this challenge is by using the mixed-supervised learning framework, in which only a part of data is densely annotated with segmentation label and the rest is weakly labeled with bounding boxes. The model is trained jointly in a multi-task learning setting. In this paper, we propose Mixed-Supervised Dual-Network (MSDN), a novel architecture which consists of two separate networks for the detection and segmentation tasks respectively, and a series of connection modules between the layers of the two networks. These connection modules are used to transfer useful information from the auxiliary detection task to help the segmentation task. We propose to use a recent technique called "Squeeze and Excitation" in the connection module to boost the transfer. We conduct experiments on two medical image segmentation datasets. The proposed MSDN model outperforms multiple baselines.

Motivation & Objective

• Address the high cost and scarcity of fully annotated medical image datasets by leveraging mixed-supervision with both dense segmentation masks and weak bounding box annotations.
• Overcome limitations of existing weakly-supervised methods that rely on iterative refinement or prior knowledge, which hinder scalability in clinical applications.
• Develop a novel dual-network architecture that enables effective knowledge transfer from an auxiliary detection task to a primary segmentation task.
• Improve segmentation performance under low-supervision conditions by integrating attention mechanisms (sSE) into cross-network connections.
• Achieve stable and superior performance compared to multi-stream and single-stream baselines across two medical imaging datasets.

Proposed method

• Design a dual-network architecture with separate streams for segmentation and object detection, avoiding shared backbones to allow task-specific optimization.
• Introduce connection modules between corresponding layers of the two networks to transfer useful features from the detection stream to the segmentation stream.
• Integrate the Binary Squeeze-and-Excitation (sSE) module in the connection modules to dynamically recalibrate feature maps based on channel and spatial attention.
• Use the sSE module to emphasize informative channels and spatial regions in the detection features, enhancing their relevance for segmentation.
• Train the model end-to-end in a multi-task learning setting using cross-entropy and Dice loss for both detection and segmentation heads.
• Apply data augmentation, normalization, dropout, and early stopping to improve generalization and prevent overfitting.

Experimental results

Research questions

• RQ1 Can a dual-network architecture with dedicated streams for detection and segmentation improve performance in mixed-supervised medical image segmentation?
• RQ2 Does incorporating Squeeze-and-Excitation modules in inter-network connection layers enhance feature transfer and segmentation accuracy?
• RQ3 How does the proposed MSDN model perform compared to fully supervised and weakly supervised baselines when only a small fraction of data is densely annotated?
• RQ4 Can the model maintain stable performance across varying ratios of strongly-labeled to weakly-labeled data?
• RQ5 Does the use of binary sSE modules in cross-network connections lead to better feature recalibration than unary sSE or no attention mechanism?

Key findings

• MSDN achieves a mean test Dice score of 83.58% ± 1.20 on the 44-22 split of the cochlea dataset, outperforming U-Net (87.91% ± 0.28) and Variant MS-Net (87.54% ± 0.36) on the same split.
• On the lung nodule dataset with 80-80 split, MSDN achieves 83.01% ± 0.69 Dice score, surpassing U-Net (80.51% ± 0.59) and Variant MS-Net (81.72% ± 1.19).
• In low-supervision regimes (e.g., 11-55 split on cochlea), MSDN maintains a Dice score of 85.60% ± 1.76, significantly outperforming U-Net (80.85% ± 0.42) and Variant MS-Net (82.60% ± 1.43).
• The ablation variant MSDN- (without detection stream) achieves lower performance than full MSDN, confirming the benefit of the detection stream and sSE modules.
• The addition of unary sSE to U-Net improves performance slightly but not as effectively as the full MSDN architecture, indicating the superiority of the dual-network with structured attention.
• Qualitative results show that MSDN produces more accurate and complete segmentation maps than baselines, especially in complex anatomical regions with low contrast.