[Paper Review] DA-RefineNet:A Dual Input Whole Slide Image Segmentation Algorithm Based on Attention

Ziqiang Li, Rentuo Tao+2 more|arXiv (Cornell University)|Jul 15, 2019

Advanced Neural Network Applications26 references4 citations

TL;DR

This paper proposes DA-RefineNet, a dual-input attention-based U-Net variant for whole slide image (WSI) segmentation that fuses local patch-level features with global full-image features to improve receptive field and segmentation accuracy. By leveraging attention-based feature fusion, the model achieves superior performance over single-input methods, especially in distinguishing benign from normal tissue regions with reduced noise and improved contextual understanding.

ABSTRACT

Automatic medical image segmentation has wide applications for disease diagnosing. However, it is much more challenging than natural optical image segmentation due to the high-resolution of medical images and the corresponding huge computation cost. The sliding window is a commonly used technique for whole slide image (WSI) segmentation, however, for these methods based on the sliding window, the main drawback is lacking global contextual information for supervision. In this paper, we propose a dual-inputs attention network (denoted as DA-RefineNet) for WSI segmentation, where both local fine-grained information and global coarse information can be efficiently utilized. Sufficient comparative experiments are conducted to evaluate the effectiveness of the proposed method, the results prove that the proposed method can achieve better performance on WSI segmentation compared to methods relying on single-input.

Motivation & Objective

• Address the limitation of single-input deep learning models in whole slide image (WSI) segmentation, which lack global contextual supervision.
• Improve segmentation accuracy by incorporating coarse global semantic information from the full WSI to guide local patch-level predictions.
• Reduce computational cost while expanding receptive field by down-sampling the full WSI to match the resolution of local patches.
• Evaluate and optimize feature fusion strategies to effectively combine fine-grained local features with coarse global features.
• Demonstrate that attention-based fusion outperforms simple concatenation or addition, enabling better generalization and faster convergence.

Proposed method

• Propose a dual-input encoder-decoder architecture where one branch processes local high-resolution patches and the other processes a down-sampled full WSI.
• Utilize ResNet-50 or ResNet-101 as the backbone for both local and global feature encoders to extract hierarchical features.
• Introduce an attention-based feature fusion module that reweights and reorganizes local features using global context as a guide.
• Apply chained residual pooling and skip connections from RefineNet to preserve spatial details and enhance feature representation.
• Train the model end-to-end using cross-entropy loss on the ICIAR2018 WSI segmentation dataset with standard data augmentation.
• Apply no post-processing techniques to preserve the integrity of the model’s intrinsic performance evaluation.

Experimental results

Research questions

• RQ1 Can the integration of global full-image context improve the segmentation accuracy of local patch-based models in whole slide image analysis?
• RQ2 How does attention-based feature fusion compare to simple concatenation or addition in combining local and global features?
• RQ3 Does the proposed dual-input architecture reduce noise and misclassification in benign and normal tissue regions compared to single-input models?
• RQ4 Can a shallower network (e.g., ResNet-50) outperform a deeper one (e.g., ResNet-101) when using attention-based fusion, indicating improved efficiency?
• RQ5 To what extent does the model’s performance depend on the choice of feature fusion strategy and network depth?

Key findings

• The attention-based dual-input model achieved the highest Dice score (0.892) on the ICIAR2018 test set, significantly outperforming single-input models and multi-size dual-input baselines.
• The attention fusion strategy yielded better results with ResNet-50 than with ResNet-101, indicating that attention enables effective feature reorganization even with less complex feature extractors.
• Visual results showed that the proposed method produced cleaner segmentation masks with significantly less red noise (misclassified normal regions as benign) compared to single-input models.
• The model reduced misclassification of normal and benign regions by leveraging global context, which helped resolve ambiguous local texture discrepancies.
• The attention-based fusion strategy improved model robustness and convergence speed, enabling high performance with reduced computational cost.
• The ablation study confirmed that attention-based fusion consistently outperformed concatenation and addition across different backbone architectures and dataset splits.