[Paper Review] Background Semantics Matter: Cross-Task Feature Exchange Network for Clustered Infrared Small Target Detection

Mengxuan Xiao, Zhu, Yinfei+6 more|arXiv (Cornell University)|Jul 29, 2024

Infrared Target Detection Methodologies4 citations

TL;DR

This paper proposes BAFE-Net, a multi-task network that jointly performs infrared small target detection and background semantic segmentation via cross-task feature hard-exchange, significantly improving detection accuracy and reducing false alarms in clustered target scenarios. The method is validated on DenseSIRST, a new sky-annotated dataset with per-pixel background semantics, and includes a novel data augmentation strategy, BAG-CP, that enhances training by pasting targets only into sky regions.

ABSTRACT

Infrared small target detection presents significant challenges due to the limited intrinsic features of the target and the overwhelming presence of visually similar background distractors. We contend that background semantics are critical for distinguishing between objects that appear visually similar in this context. To address this challenge, we propose a task, clustered infrared small target detection, and introduce DenseSIRST, a benchmark dataset that provides per-pixel semantic annotations for background regions. This dataset facilitates the shift from sparse to dense target detection. This dataset facilitates the shift from sparse to dense target detection. Building on this resource, we propose the Background-Aware Feature Exchange Network (BAFE-Net), a multi-task architecture that jointly tackles target detection and background semantic segmentation. BAFE-Net incorporates a dynamic cross-task feature hard-exchange mechanism, enabling the effective exchange of target and background semantics between the two tasks. Comprehensive experiments demonstrate that BAFE-Net significantly enhances target detection accuracy while mitigating false alarms. The DenseSIRST dataset, along with the code and trained models, is publicly available at https://github.com/GrokCV/BAFE-Net.

Motivation & Objective

• Address the challenge of detecting clustered infrared small targets in complex backgrounds where traditional methods fail due to visual similarity with distractors.
• Overcome the limitations of sparse-target-focused methods by introducing a new detection paradigm tailored for dense, clustered target scenarios.
• Leverage background semantics to improve discrimination between real targets and false alarms, especially in non-uniform backgrounds like buildings.
• Develop a new benchmark dataset, DenseSIRST, with per-pixel semantic annotations for background regions to enable dense target detection research.
• Propose a data augmentation method, BAG-CP, that selectively pastes targets into sky regions to avoid introducing false alarms during training.

Proposed method

• Propose BAFE-Net, a multi-task architecture that jointly performs infrared small target detection and background semantic segmentation using a shared backbone and task-specific heads.
• Implement a cross-task feature hard-exchange mechanism that enables bidirectional, hard-attention-based feature transfer between the detection and segmentation heads to enhance feature discrimination.
• Design a Background-Aware Gaussian Copy-Paste (BAG-CP) augmentation method that selectively pastes small targets into sky regions using Gaussian blending, avoiding false positives in complex backgrounds.
• Use ResNet50 as the optimal backbone after ablation, balancing feature richness and computational efficiency for small target detection.
• Train the model end-to-end with a multi-task loss combining detection and segmentation objectives, optimizing for mAP and recall on the DenseSIRST benchmark.
• Integrate background semantics into the feature learning process by encouraging the segmentation head to suppress non-target, high-activation background regions, thereby reducing false alarms.

Experimental results

Research questions

• RQ1 Can explicit modeling of background semantics improve detection performance in clustered infrared small target detection?
• RQ2 How does cross-task feature exchange between detection and segmentation heads enhance feature discrimination for small targets?
• RQ3 To what extent does selective data augmentation using sky-annotated regions reduce false positives compared to standard Copy-Paste?
• RQ4 Does the proposed multi-task framework outperform single-task state-of-the-art methods on a dense, real-world infrared benchmark?
• RQ5 Can a new dataset with per-pixel background annotations enable more accurate and robust evaluation of clustered infrared small target detection algorithms?

Key findings

• BAFE-Net achieves the highest mAP_{07} and mAP_{12} on the DenseSIRST benchmark, outperforming all state-of-the-art one-stage and two-stage methods.
• The visual analysis shows that BAFE-Net produces stronger activation in target regions and significantly cleaner background activation maps, indicating better suppression of false positives.
• BAG-CP outperforms standard Copy-Paste by reducing false alarms, as it restricts target insertion to sky regions where distractors are less likely to be misclassified.
• ResNet50 was found to be the optimal backbone, balancing feature richness and computational cost, with deeper networks like ResNet101 leading to performance degradation due to feature over-smoothing.
• The multi-task design with cross-task feature hard-exchange leads to improved feature discrimination, as evidenced by higher detection sensitivity and lower background activation.
• The proposed DenseSIRST dataset enables a shift from sparse to dense target detection, providing a realistic benchmark for future research with per-pixel semantic annotations for background regions.