[Paper Review] Local Contrast and Global Contextual Information Make Infrared Small Object Salient Again

Chenyi Wang, Huan Wang+1 more|arXiv (Cornell University)|Jan 28, 2023

Infrared Target Detection Methodologies7 citations

TL;DR

This paper proposes UCFNet, a U-Net-based model for infrared small object segmentation (ISOS) that combines central difference convolution (CDC) to enhance local contrast and fast Fourier convolution (FFC) to capture global context while preserving high-resolution features. The method achieves state-of-the-art performance on public ISOS benchmarks, significantly improving mIoU and detection recall while reducing false alarms.

ABSTRACT

Infrared small object detection (ISOS) aims to segment small objects only covered with several pixels from clutter background in infrared images. It's of great challenge due to: 1) small objects lack of sufficient intensity, shape and texture information; 2) small objects are easily lost in the process where detection models, say deep neural networks, obtain high-level semantic features and image-level receptive fields through successive downsampling. This paper proposes a reliable detection model for ISOS, dubbed UCFNet, which can handle well the two issues. It builds upon central difference convolution (CDC) and fast Fourier convolution (FFC). On one hand, CDC can effectively guide the network to learn the contrast information between small objects and the background, as the contrast information is very essential in human visual system dealing with the ISOS task. On the other hand, FFC can gain image-level receptive fields and extract global information while preventing small objects from being overwhelmed.Experiments on several public datasets demonstrate that our method significantly outperforms the state-of-the-art ISOS models, and can provide useful guidelines for designing better ISOS deep models. Code are available at https://github.com/wcyjerry/BasicISOS.

Motivation & Objective

• To address the challenge of infrared small object segmentation (ISOS) where small objects lack sufficient intensity, shape, and texture cues.
• To mitigate the loss of small objects during feature extraction due to excessive downsampling in deep networks.
• To design a deep learning model that effectively integrates local contrast and global contextual information for improved segmentation accuracy.
• To provide a reliable, generalizable backbone for ISOS that outperforms existing state-of-the-art methods.

Proposed method

• The proposed UCFNet architecture is a U-Net variant enhanced with central difference convolution (CDC), which mimics human visual system behavior by emphasizing contrast between small objects and their background.
• CDC computes the difference between central pixel values and their surrounding neighbors using a learnable parameter θ, enhancing local contrast features critical for detecting dim, small objects.
• Fast Fourier convolution (FFC) is employed to extract global context from high-resolution feature maps by performing convolutions in the frequency domain, enabling image-level receptive fields without downsampling.
• FFC blocks are stacked in residual units to preserve spatial details while aggregating long-range dependencies, preventing small objects from being lost during feature abstraction.
• The model integrates CDC and FFC in encoder and decoder paths to jointly optimize local saliency and global context awareness.
• Ablation studies validate the design choices, including optimal θ=0.7 for CDC and five FFC residual blocks for best performance.

Experimental results

Research questions

• RQ1 How can local contrast information be effectively extracted to enhance small object detection in infrared images with low signal-to-noise ratios?
• RQ2 Can global context be captured at high-resolution feature maps without relying on deep, downsampled layers?
• RQ3 How do CDC and FFC compare to other advanced convolution operators (e.g., deformable, gated) in ISOS tasks?
• RQ4 What is the optimal configuration of CDC and FFC components for maximizing segmentation performance?
• RQ5 Does combining CDC and FFC yield synergistic improvements over using either component alone?

Key findings

• UCFNet with both CDC and FFC achieves the highest mIoU of 80.89% on the SIRST dataset, significantly outperforming the vanilla UCF baseline (74.14%).
• The addition of CDC alone improves mIoU by 1.81 percentage points (to 75.95%) and reduces false alarms (Fa) from 4.83×10⁻⁶ to 3.46×10⁻⁶.
• FFC alone achieves the best mIoU of 79.55% and 100% Pd, with Fa reduced to 1.82×10⁻⁶, demonstrating its effectiveness in global context learning.
• Combining CDC and FFC yields the best overall performance with 100% Pd and Fa of 2.22×10⁻⁶, indicating high detection recall and low false positives.
• Ablation shows that CDC with θ=0.7 achieves optimal performance, while five FFC residual blocks yield the best results, confirming design robustness.
• Compared to alternatives, CDC outperforms gated and deformable convolutions in ISOS, and FFC surpasses dilated convolutions and double attention modules in both mIoU and Fa metrics.