[Paper Review] HS-FPN: High Frequency and Spatial Perception FPN for Tiny Object Detection

Zican Shi, Jing Hu+7 more|arXiv (Cornell University)|Dec 13, 2024

Brain Tumor Detection and Classification6 citations

TL;DR

The paper introduces HS-FPN, adding a High Frequency Perception module (HFP) and a Spatial Dependency Perception module (SDP) to Feature Pyramid Networks to boost tiny object detection, achieving notable AP gains on AI-TOD and DOTA-mini10 datasets.

ABSTRACT

The introduction of Feature Pyramid Network (FPN) has significantly improved object detection performance. However, substantial challenges remain in detecting tiny objects, as their features occupy only a very small proportion of the feature maps. Although FPN integrates multi-scale features, it does not directly enhance or enrich the features of tiny objects. Furthermore, FPN lacks spatial perception ability. To address these issues, we propose a novel High Frequency and Spatial Perception Feature Pyramid Network (HS-FPN) with two innovative modules. First, we designed a high frequency perception module (HFP) that generates high frequency responses through high pass filters. These high frequency responses are used as mask weights from both spatial and channel perspectives to enrich and highlight the features of tiny objects in the original feature maps. Second, we developed a spatial dependency perception module (SDP) to capture the spatial dependencies that FPN lacks. Our experiments demonstrate that detectors based on HS-FPN exhibit competitive advantages over state-of-the-art models on the AI-TOD dataset for tiny object detection.

Motivation & Objective

• Identify three key limitations of standard FPN for tiny object detection.
• Propose HS-FPN with HFP and SDP to address limited feature content and lack of spatial perception.
• Demonstrate compatibility and performance gains when replacing FPN with HS-FPN across multiple detectors and backbones on TOD datasets.

Proposed method

• Design HS-FPN with four lateral connections similar to FPN but each lateral includes an HFP module.
• HFP generates high-frequency responses via a predetermined high-pass filter and uses them in channel and spatial branches to reweight features.
• Channel Path (within HFP) uses high-frequency responses to compute channel attention via GAP and GMP followed by grouped 1x1 convolutions.
• Spatial Path (within HFP) uses high-frequency responses as a spatial mask to compute spatial attention via a 1x1 conv.
• Introduce Spatial Dependency Perception (SDP) module to capture pixel-level spatial dependencies between adjacent C_i and upsampled P_{i+1} using cross-attention on feature blocks.
• Integrate SDP between adjacent pyramid levels to enrich tiny-object features with spatially dependent information.
• Evaluate HS-FPN in Faster R-CNN, Cascade R-CNN, and DetectoRS backbones on AI-TOD and DOTA-mini10 datasets.

Experimental results

Research questions

• RQ1 Does HS-FPN improve tiny object detection performance over standard FPN across multiple detectors?
• RQ2 How do HFP and SDP individually and jointly affect AP metrics on TOD benchmarks?
• RQ3 What is the computational cost impact of replacing FPN with HS-FPN?
• RQ4 How sensitive is HS-FPN to the high-pass filter parameter alpha and to different backbones?

Key findings

• Replacing FPN with HS-FPN yields significant AP gains across detectors on AI-TOD (e.g., Faster R-CNN + HS-FPN: AP 48.4 vs 46.9 with FPN; Cascade R-CNN + HS-FPN: AP 50.9 vs 49.4).
• HS-FPN with both HFP and SDP provides the best performance, achieving higher AP than using either module alone (e.g., with Cascade R-CNN: AP 50.9/76.6/22.2/41.8/59.0 vs baseline 49.4/74.2/18.1/40.0/58.1).
• Ablation shows HFP alone improves AP by about 2.2 points over FPN, and SDP adds about 1.1 points; combined they yield larger gains (AP up to 3.4 points over FPN).
• On DOTA-mini10, HS-FPN improves AP from 20.2 (FPN baseline) to 23.6, with notable gains in tiny-object metrics (AP_t 25.2, AP_s 27.0).
• HS-FPN demonstrates compatibility, improving AP even with stronger backbones (e.g., Faster R-CNN + HS-FPN with ResNet-101).
• Increasing computational cost is modest relative to the performance gains (Table 5 shows FLOPs/Params increments).