[Paper Review] ShipSRDet: An End-to-End Remote Sensing Ship Detector Using Super-Resolved Feature Representation

Shitian He, Huanxin Zou+3 more|arXiv (Cornell University)|Mar 17, 2021

Advanced Image Processing Techniques12 references4 citations

TL;DR

This paper proposes ShipSRDet, an end-to-end deep learning framework that jointly optimizes image super-resolution (SR) and ship detection by fusing intermediate features from the SR and detection networks. By integrating both super-resolved images and feature representations, ShipSRDet achieves a 64.50% mAP on the HRSC dataset—nearly matching the 68.80% mAP of detection on true high-resolution images—demonstrating that feature-level knowledge transfer from SR enhances detection performance beyond image-level SR alone.

ABSTRACT

High-resolution remote sensing images can provide abundant appearance information for ship detection. Although several existing methods use image super-resolution (SR) approaches to improve the detection performance, they consider image SR and ship detection as two separate processes and overlook the internal coherence between these two correlated tasks. In this paper, we explore the potential benefits introduced by image SR to ship detection, and propose an end-to-end network named ShipSRDet. In our method, we not only feed the super-resolved images to the detector but also integrate the intermediate features of the SR network with those of the detection network. In this way, the informative feature representation extracted by the SR network can be fully used for ship detection. Experimental results on the HRSC dataset validate the effectiveness of our method. Our ShipSRDet can recover the missing details from the input image and achieves promising ship detection performance.

Motivation & Objective

• To address the limitation of existing SR-based ship detection methods that treat super-resolution and detection as separate, non-interactive processes.
• To investigate whether intermediate feature representations from a super-resolution network can enhance ship detection performance beyond just using super-resolved images.
• To design an end-to-end trainable framework that jointly optimizes image super-resolution and ship detection using feature-level fusion.
• To validate the effectiveness of super-resolved feature representation in improving detection accuracy on remote sensing images.
• To demonstrate that feature-level integration between SR and detection networks yields performance gains over image-level SR alone.

Proposed method

• The method employs a dual-branch architecture: a super-resolution (SR) module based on residual dense blocks (RDBs) and a detection module based on SSD with VGG-16 backbones.
• The SR module generates a super-resolved image and two intermediate features: one from the low-resolution input ($\mathcal{F}^{LR}_{out}$) and one from the high-resolution output ($\mathcal{F}^{HR}_{out}$), both of dimension 64 channels.
• The detection module performs feature adaptation by projecting $\mathcal{F}^{LR}_{out}$ and $\mathcal{F}^{HR}_{out}$ to 256 and 64 channels, respectively, to align with VGG-16 feature dimensions.
• The adapted features are added to the 2nd and 7th layers of the VGG-16 network, enabling cross-network feature fusion for improved detection representation.
• The entire network is trained end-to-end with joint optimization of SR and detection objectives, allowing the SR network to adapt to detection-relevant details.
• A sub-pixel convolution layer is used to upsample features, and skip connections are applied to stabilize training and preserve low-level details.

Fig. 1 : Visual results achieved by our ShipSRDet on the HRSC dataset [ 7 ] . Our method recovers missing details in the input image and achieves promising detection performance.

Experimental results

Research questions

• RQ1 Can intermediate feature representations from a super-resolution network improve remote sensing ship detection beyond just using super-resolved images?
• RQ2 How does end-to-end joint training of super-resolution and detection compare to pre-trained SR followed by separate detection fine-tuning?
• RQ3 What is the contribution of feature-level fusion between SR and detection networks to detection mAP on remote sensing images?
• RQ4 How does the performance of the proposed method compare to baseline methods using bicubic upsampling or pre-trained SR networks?
• RQ5 To what extent can feature-integrated SR improve detection accuracy in comparison to the upper bound of detection on true high-resolution images?

Key findings

• ShipSRDet achieves a mean average precision (mAP) of 64.50% on the HRSC dataset, significantly outperforming the baseline Bicubic+SSD (58.80%) and the pre-trained SRnet+SSD (60.10%).
• End-to-end fine-tuning of the SRnet+SSD variant improves mAP by 3.70% to 63.80%, demonstrating the benefit of detection-driven SR learning.
• The proposed feature integration in ShipSRDet yields an additional 0.70% mAP gain over (SRnet+SSD)_ft, showing that intermediate feature fusion further enhances detection performance.
• The method closely approximates the upper bound of 68.80% mAP achieved by detecting on true high-resolution images, indicating strong performance recovery.
• Visual results confirm that ShipSRDet recovers fine details in low-resolution inputs, reducing false positives and missed detections compared to bicubic interpolation.
• The average inference time of ShipSRDet is 190 ms per 128×128 image, which is slightly higher than the 81 ms for direct detection on HR images, indicating a small efficiency trade-off for significant accuracy gains.

Fig. 2 : An overview of our ShipSRDet .