[Paper Review] CATNet: Context AggregaTion Network for Instance Segmentation in Remote Sensing Images.

Ye Liu, Huifang Li+4 more|arXiv (Cornell University)|Nov 22, 2021

Advanced Neural Network Applications54 references4 citations

TL;DR

CATNet is a novel context aggregation network for instance segmentation in remote sensing images, introducing three lightweight modules—DenseFPN, SCP, and HRoIE—to enhance feature extraction by aggregating global context across feature, spatial, and instance domains. It achieves state-of-the-art performance on iSAID, DIOR, NWPU VHR-10, and HRSID with comparable computational cost.

ABSTRACT

The task of instance segmentation in remote sensing images, aiming at performing per-pixel labeling of objects at instance level, is of great importance for various civil applications. Despite previous successes, most existing instance segmentation methods designed for natural images encounter sharp performance degradations when directly applied to top-view remote sensing images. Through careful analysis, we observe that the challenges mainly come from lack of discriminative object features due to severe scale variations, low contrasts, and clustered distributions. In order to address these problems, a novel context aggregation network (CATNet) is proposed to improve the feature extraction process. The proposed model exploits three lightweight plug-and-play modules, namely dense feature pyramid network (DenseFPN), spatial context pyramid (SCP), and hierarchical region of interest extractor (HRoIE), to aggregate global visual context at feature, spatial, and instance domains, respectively. DenseFPN is a multi-scale feature propagation module that establishes more flexible information flows by adopting inter-level residual connections, cross-level dense connections, and feature re-weighting strategy. Leveraging the attention mechanism, SCP further augments the features by aggregating global spatial context into local regions. For each instance, HRoIE adaptively generates RoI features for different downstream tasks. We carry out extensive evaluation of the proposed scheme on the challenging iSAID, DIOR, NWPU VHR-10, and HRSID datasets. The evaluation results demonstrate that the proposed approach outperforms state-of-the-arts with similar computational costs. Code is available at https://github.com/yeliudev/CATNet.

Motivation & Objective

• Address the performance degradation of natural image instance segmentation models when applied to top-view remote sensing images.
• Overcome challenges from scale variations, low contrast, and clustered object distributions in remote sensing imagery.
• Improve feature representation by aggregating global visual context at three levels: feature, spatial, and instance.
• Design lightweight, plug-and-play modules that enhance feature extraction without significantly increasing computational cost.

Proposed method

• Proposes DenseFPN, a multi-scale feature pyramid with inter-level residual connections, cross-level dense connections, and feature re-weighting to improve information flow across scales.
• Introduces Spatial Context Pyramid (SCP), which uses attention mechanisms to inject global spatial context into local feature regions.
• Develops Hierarchical Region of Interest Extractor (HRoIE) to adaptively generate RoI features tailored for different downstream tasks per instance.
• Integrates the three modules—DenseFPN, SCP, and HRoIE—into a unified network for end-to-end training and inference.
• Employs attention-based feature re-weighting in DenseFPN to emphasize informative features across feature maps.
• Designs the modules to be lightweight and plug-and-play, enabling easy integration into existing instance segmentation frameworks.

Experimental results

Research questions

• RQ1 How can global visual context be effectively aggregated across different levels—feature, spatial, and instance—to improve instance segmentation in remote sensing?
• RQ2 To what extent can lightweight, plug-and-play modules enhance feature representation in remote sensing images with scale variations and low contrast?
• RQ3 Can a unified network architecture combining multi-scale, spatial, and instance-level context aggregation outperform existing state-of-the-art methods?
• RQ4 How does the proposed CATNet perform under diverse remote sensing benchmarks with varying object distributions and image characteristics?

Key findings

• CATNet achieves state-of-the-art performance on the iSAID dataset, outperforming previous methods under similar computational budgets.
• On the DIOR dataset, CATNet demonstrates superior generalization capability across diverse scene categories and object scales.
• The model achieves consistent improvements on the NWPU VHR-10 and HRSID datasets, particularly in handling clustered and small-scale objects.
• Ablation studies confirm that each component—DenseFPN, SCP, and HRoIE—contributes significantly to the overall performance gain.
• The lightweight design of the modules ensures minimal increase in inference time and parameter count compared to baseline models.
• The attention-based context aggregation in SCP enhances feature discrimination, especially in low-contrast and cluttered regions.