[Paper Review] Ship Instance Segmentation From Remote Sensing Images Using Sequence Local Context Module

Yingchao Feng, Wenhui Diao+4 more|arXiv (Cornell University)|Apr 22, 2019

Advanced Neural Network Applications7 references4 citations

TL;DR

This paper proposes SLCMASK-Net, a novel instance segmentation framework for remote sensing images that introduces a Sequence Local Context (SLC) module to enhance multi-scale context learning via stacked dilated convolutions. The SLC module improves mask-level AP by 10.98% over Mask R-CNN, achieving state-of-the-art performance of 78.61% AP on a high-resolution QuickBird dataset, effectively reducing confusion between densely packed ships.

ABSTRACT

The performance of object instance segmentation in remote sensing images has been greatly improved through the introduction of many landmark frameworks based on convolutional neural network. However, the object densely issue still affects the accuracy of such segmentation frameworks. Objects of the same class are easily confused, which is most likely due to the close docking between objects. We think context information is critical to address this issue. So, we propose a novel framework called SLCMASK-Net, in which a sequence local context module (SLC) is introduced to avoid confusion between objects of the same class. The SLC module applies a sequence of dilation convolution blocks to progressively learn multi-scale context information in the mask branch. Besides, we try to add SLC module to different locations in our framework and experiment with the effect of different parameter settings. Comparative experiments are conducted on remote sensing images acquired by QuickBird with a resolution of $0.5m-1m$ and the results show that the proposed method achieves state-of-the-art performance.

Motivation & Objective

• To address the challenge of object confusion in ship instance segmentation caused by dense, closely docked ships in remote sensing images.
• To improve mask-level performance by enhancing context-aware feature learning in the mask branch.
• To design a lightweight, effective context module that captures multi-scale spatial context without increasing computational burden.
• To evaluate the impact of module placement, dilation rates, and architectural configurations on segmentation accuracy.

Proposed method

• The framework builds upon Mask R-CNN with ResNet-101 and FPN as the backbone, using RoIAlign to preserve spatial precision.
• A novel Sequence Local Context (SLC) module is introduced in the mask branch, consisting of three stacked dilated convolution blocks with increasing dilation rates (1, r₁, r₂).
• Each block applies a 1×1 convolution followed by a 3×3 dilated convolution, and outputs are fused via element-wise summation to aggregate multi-scale context.
• The SLC module is designed to progressively learn context at different scales while avoiding information loss from large dilation rates.
• The module is evaluated at different positions in the network, including the classification and regression branches, with ablation studies on dilation rates and number of layers.
• Experiments use data augmentation and optimized anchor settings to ensure fair comparison with baseline models.

Experimental results

Research questions

• RQ1 Can multi-scale context learning reduce confusion between densely packed ships in remote sensing instance segmentation?
• RQ2 How does the sequence-based application of dilated convolutions compare to standard ASPP or inception modules in capturing spatial context?
• RQ3 What is the optimal placement and configuration of the SLC module for maximizing mask AP in ship segmentation?
• RQ4 Does context enhancement have a more significant impact on mask-level AP than on bounding box detection performance?

Key findings

• The proposed SLCMASK-Net achieves a mask-level AP of 78.61%, representing a 10.98% improvement over the baseline Mask R-CNN (67.63%).
• The SLC module outperforms other context modules: it achieves 85.68% recall and 78.61% AP, surpassing ASPP(2,4,6) (73.48% AP) and CCL (73.36% AP).
• Placing the SLC module in the mask branch yields the best performance, while adding it to the classification and regression branches degrades accuracy.
• Using dilation rates of 2 and 3 yields optimal results; higher rates (e.g., 4 and 6) reduce performance due to loss of fine-grained context and degeneration to 1×1 convolutions.
• The ablation study confirms that fusing three layers (layers=3) is superior to fusing only two layers, and the best performance is achieved with r₁=2, r₂=3.
• The SLC module improves mask-level AP by 10.98% but only increases box-level AP by 3.0 points, indicating its primary benefit lies in mask refinement.