[Paper Review] CourtNet for Infrared Small-Target Detection

Jingchao Peng, Haitao Zhao+3 more|arXiv (Cornell University)|Sep 28, 2022

Infrared Target Detection Methodologies4 citations

TL;DR

CourtNet is a novel infrared small-target detection method inspired by legal court procedures, featuring a prosecution network for high recall, a defendant network for high precision, and a jury network that adaptively fuses their outputs. It uses a densely connected transformer with fine-grained attention to preserve small targets and an adaptive balance loss to optimize precision-recall trade-offs, achieving state-of-the-art F1-scores of 0.62 on MFIRST and 0.73 on SIRST.

ABSTRACT

Infrared small-target detection (ISTD) is an important computer vision task. ISTD aims at separating small targets from complex background clutter. The infrared radiation decays over distances, making the targets highly dim and prone to confusion with the background clutter, which makes the detector challenging to balance the precision and recall rate. To deal with this difficulty, this paper proposes a neural-network-based ISTD method called CourtNet, which has three sub-networks: the prosecution network is designed for improving the recall rate; the defendant network is devoted to increasing the precision rate; the jury network weights their results to adaptively balance the precision and recall rate. Furthermore, the prosecution network utilizes a densely connected transformer structure, which can prevent small targets from disappearing in the network forward propagation. In addition, a fine-grained attention module is adopted to accurately locate the small targets. Experimental results show that CourtNet achieves the best F1-score on the two ISTD datasets, MFIRST (0.62) and SIRST (0.73).

Motivation & Objective

• To address the challenge of balancing high precision and high recall in infrared small-target detection (ISTD), where small targets are easily lost or confused with background clutter.
• To overcome the limitations of standard Vision Transformers in localizing sub-patch targets due to coarse-grained attention and feature degradation in deep networks.
• To design a detection framework that adaptively balances precision and recall without manual threshold tuning.
• To preserve small target features during deep network forward propagation using dense residual connections and fine-grained attention.

Proposed method

• CourtNet employs a three-branch architecture: the prosecution network maximizes recall by detecting all suspected targets, the defendant network enhances precision by filtering false positives, and the jury network performs weighted fusion of their outputs.
• The prosecution network uses a densely connected transformer to maintain low-level and high-level features, preventing small targets from vanishing during forward pass.
• A fine-grained attention module is applied within each patch to precisely localize small targets inside the patch, complementing the coarse-grained attention of the standard Vision Transformer.
• An adaptive balance loss is introduced, which dynamically scales the precision and recall loss components based on their relative magnitudes to reduce training fluctuations and accelerate convergence.
• The model is trained end-to-end on two benchmark datasets, MFIRST and SIRST, with evaluation using F1-score as the primary metric.

Experimental results

Research questions

• RQ1 Can a court-inspired neural architecture with specialized roles for recall and precision improve the balance between detection recall and precision in ISTD?
• RQ2 Can a densely connected transformer with fine-grained attention effectively preserve and localize sub-patch small targets in infrared images?
• RQ3 Does an adaptive balance loss lead to more stable and faster convergence during training compared to standard loss functions?
• RQ4 Can the proposed method outperform existing SOTA methods on standard ISTD benchmarks in terms of F1-score?

Key findings

• CourtNet achieves the highest F1-score of 0.62 on the MFIRST dataset, outperforming existing methods.
• On the SIRST dataset, CourtNet achieves an F1-score of 0.73, setting a new state-of-the-art performance.
• The fine-grained attention module demonstrates cyclical feature patterns with a period of 9, confirming its ability to locate targets within patches consistently.
• The adaptive balance loss reduces training fluctuations and accelerates convergence, as shown by more stable training curves.
• The prosecution network detects all targets in the constructed test set with 100% detection rate and an F1-score of 80%.