[Paper Review] Large Selective Kernel Network for Remote Sensing Object Detection

Yuxuan Li, Qibin Hou+4 more|arXiv (Cornell University)|Mar 16, 2023

Remote-Sensing Image Classification46 citations

TL;DR

The paper introduces LSKNet, which dynamically expands and selects large receptive fields via a sequence of decomposed depthwise convolutions and a spatial kernel selection mechanism, achieving state-of-the-art results on HRSC2016, DOTA-v1.0, and FAIR1M-v1.0.

ABSTRACT

Recent research on remote sensing object detection has largely focused on improving the representation of oriented bounding boxes but has overlooked the unique prior knowledge presented in remote sensing scenarios. Such prior knowledge can be useful because tiny remote sensing objects may be mistakenly detected without referencing a sufficiently long-range context, and the long-range context required by different types of objects can vary. In this paper, we take these priors into account and propose the Large Selective Kernel Network (LSKNet). LSKNet can dynamically adjust its large spatial receptive field to better model the ranging context of various objects in remote sensing scenarios. To the best of our knowledge, this is the first time that large and selective kernel mechanisms have been explored in the field of remote sensing object detection. Without bells and whistles, LSKNet sets new state-of-the-art scores on standard benchmarks, i.e., HRSC2016 (98.46\% mAP), DOTA-v1.0 (81.85\% mAP) and FAIR1M-v1.0 (47.87\% mAP). Based on a similar technique, we rank 2nd place in 2022 the Greater Bay Area International Algorithm Competition. Code is available at https://github.com/zcablii/Large-Selective-Kernel-Network.

Motivation & Objective

• Leverage prior knowledge in remote sensing that objects require varying contextual information for accurate detection.
• Develop a backbone mechanism that can dynamically expand its receptive field using large, selective kernels.
• Efficiently fuse multi-scale contextual features to improve detection of tiny or context-dependent objects in aerial imagery.
• Demonstrate state-of-the-art performance on standard remote sensing benchmarks with a lightweight, scalable backbone.

Proposed method

• Decompose a large convolution kernel into a sequence of depthwise convolutions with progressively larger kernel sizes and dilations to expand the receptive field.
• Construct a large kernel selection (LK) module that processes multi-scale features and applies a spatial attention-based selection mask for each decomposed kernel.
• Concatenate and pool the outputs from different receptive fields, then generate spatial attention maps to weight and fuse large-kernel features.
• Compute the final LSK feature as an element-wise product between the input feature and the learned attention feature (Y = X · S).
• Integrate the LSK module into a residual block (LK Selection block + FFN) within a backbone for detectors like Oriented RCNN.
• Provide variants (LSKNet-T, LSKNet-S) with different channel dimensions and block counts to balance accuracy and efficiency.

Experimental results

Research questions

• RQ1 Does a large-kernel, selectively fused backbone improve remote sensing object detection across datasets with different object scales and contexts?
• RQ2 How should large receptive fields be decomposed and combined to maximize speed-accuracy trade-offs in aerial imagery?
• RQ3 Does spatial (versus channel) kernel selection better capture spatial context variations inherent in remote sensing data?
• RQ4 How does LSKNet perform when integrated with various detection frameworks (two-stage and one-stage) on standard benchmarks?

Key findings

• LSKNet achieves state-of-the-art mAP on HRSC2016 (98.46%), DOTA-v1.0 (81.85% reported in abstract; 81.64% in results table), and FAIR1M-v1.0 (47.87%).
• LSKNet-S attains 18.1 FPS on 1024×1024 images with a single RTX3090 while delivering strong accuracy.
• A two-kernel decomposition of the large kernel provides a favorable speed–accuracy trade-off on DOTA-v1.0 (e.g., 80.91–81.31 mAP across configurations).
• Spatial selection outperforms channel attention for remote sensing tasks, and the model tends to use larger kernels in shallower layers and smaller ones in deeper layers.
• LSKNet-T/S backbones improve multiple detection frameworks (two-stage and one-stage) with competitive parameter and FLOP counts compared to ResNet-18 baselines.
• Visual analyses corroborate that different object categories require varying context ranges, aligning with the priors motivating LSKNet.