[Paper Review] Infrared Small-Dim Target Detection with Transformer under Complex Backgrounds

Fangcen Liu, Chenqiang Gao+4 more|arXiv (Cornell University)|Sep 29, 2021

Infrared Target Detection MethodologiesEngineering57 references27 citations

TL;DR

The paper proposes a transformer-based framework for infrared small-dim target detection that leverages self-attention for large-range feature interactions, a feature enhancement module for discriminative target features, and a U-Net like decoder to preserve target information, achieving state-of-the-art results on MFIRST and SIRST datasets.

ABSTRACT

The infrared small-dim target detection is one of the key techniques in the infrared search and tracking system. Since the local regions similar to infrared small-dim targets spread over the whole background, exploring the interaction information amongst image features in large-range dependencies to mine the difference between the target and background is crucial for robust detection. However, existing deep learning-based methods are limited by the locality of convolutional neural networks, which impairs the ability to capture large-range dependencies. Additionally, the small-dim appearance of the infrared target makes the detection model highly possible to miss detection. To this end, we propose a robust and general infrared small-dim target detection method with the transformer. We adopt the self-attention mechanism of the transformer to learn the interaction information of image features in a larger range. Moreover, we design a feature enhancement module to learn discriminative features of small-dim targets to avoid miss detection. After that, to avoid the loss of the target information, we adopt a decoder with the U-Net-like skip connection operation to contain more information of small-dim targets. Finally, we get the detection result by a segmentation head. Extensive experiments on two public datasets show the obvious superiority of the proposed method over state-of-the-art methods and the proposed method has stronger cross-scene generalization and anti-noise performance.

Motivation & Objective

• Motivate robust infrared small-dim target detection under complex backgrounds where targets are tiny and easily confused with clutter.
• Address CNN locality limitations by using self-attention to model long-range dependencies among image tokens.
• Enhance discriminative features of small targets through a dedicated feature enhancement module.
• Preserve target information via a U-Net-like decoder to improve segmentation-based detection.
• Demonstrate superior performance and cross-scene generalization compared with state-of-the-art methods.

Proposed method

• Embed an image with a ResNet-50-based feature embedding module to obtain compact feature tokens with position encoding.
• Use a compound encoder consisting of k encoder layers, each with multi-head self-attention (MSA) and a feature enhancement module (FEM) to learn large-range interactions and enhance small-target features.
• Adopt a U-Net-like skip-connection decoder to upsample and fuse features, producing a confidence map for segmentation.
• Apply a sigmoid-based confidence map and adaptive thresholding for final detection. The loss combines IoU objective to address class imbalance and focus on small-target regions.

Experimental results

Research questions

• RQ1 Can a transformer-based architecture capture large-range interactions to differentiate tiny infrared targets from complex backgrounds more effectively than CNNs?
• RQ2 Does a dedicated feature enhancement module improve the discriminative power for small-dim targets within a transformer framework?
• RQ3 Does a U-Net-like decoder help preserve small-target information during upsampling for accurate detection?
• RQ4 How does the proposed method perform in cross-scene scenarios and under additive Gaussian noise compared with existing methods?

Key findings

• The proposed method outperforms state-of-the-art model-driven and deep-learning methods on MFIRST and SIRST datasets in terms of Pd, AUC, and F1 scores across target types.
• On MFIRST, the method achieves Pd 90.08% and AUC 89.34% with F1t 92.59% and F1p 64.59%, while on SIRST it achieves Pd 100.00% and AUC 99.14% with F1t 98.62% and F1p 83.16% (times: 6.41 s per 100 images).
• Ablation studies show that removing the pooling layer degrades performance, while adding the feature enhancement module and a deep encoder improves Pd by about 3.03% and AUC by 4.57% on MFIRST.
• The approach exhibits stronger cross-scene generalization than ACM and MDvsFA, e.g., MFIRST-trained models testing on SIRST outperform by about 0.93% Pd and 2.07% AUC relative to MDvsFA; SIRST-trained models testing on MFIRST outperform ACM by about 20.52% Pd and 23.53% AUC.
• Under Gaussian noise, the proposed method maintains superior robustness, showing slower performance degradation than competing methods as noise variance increases.
• Qualitative results indicate fewer false alarms and better target preservation across extremely challenging backgrounds, including high noise, cluttered scenes, and dim targets.