[Paper Review] HazyDet: Open-Source Benchmark for Drone-View Object Detection with Depth-Cues in Hazy Scenes

Changfeng Feng, Zhenyuan Chen+6 more|arXiv (Cornell University)|Sep 30, 2024

Advanced Neural Network Applications4 citations

TL;DR

HazyDet introduces the first large-scale, open-source benchmark for drone-view object detection in hazy scenes, featuring 383,000 real and synthetically hazy instances. It proposes DeCoDet, a depth-conditioned detector with a Multi-scale Depth-aware Detection Head and Dynamic Depth Condition Kernel, achieving significant mAP gains by leveraging depth cues without explicit image restoration.

ABSTRACT

Object detection from aerial platforms under adverse atmospheric conditions, particularly haze, is paramount for robust drone autonomy. Yet, this domain remains largely underexplored, primarily hindered by the absence of specialized benchmarks. To bridge this gap, we present extit{HazyDet}, the first, large-scale benchmark specifically designed for drone-view object detection in hazy conditions. Comprising 383,000 real-world instances derived from both naturally hazy captures and synthetically hazed scenes augmented from clear images, HazyDet provides a challenging and realistic testbed for advancing detection algorithms. To address the severe visual degradation induced by haze, we propose the Depth-Conditioned Detector (DeCoDet), a novel architecture that integrates a Depth-Conditioned Kernel to dynamically modulate feature representations based on depth cues. The practical efficacy and robustness of DeCoDet are further enhanced by its training with a Progressive Domain Fine-Tuning (PDFT) strategy to navigate synthetic-to-real domain shifts, and a Scale-Invariant Refurbishment Loss (SIRLoss) to ensure resilient learning from potentially noisy depth annotations. Comprehensive empirical validation on HazyDet substantiates the superiority of our unified DeCoDet framework, which achieves state-of-the-art performance, surpassing the closest competitor by a notable +1.5\% mAP on challenging real-world hazy test scenarios. Our dataset and toolkit are available at https://github.com/GrokCV/HazyDet.

Motivation & Objective

• Address the lack of large-scale benchmarks for drone-based object detection under adverse weather, particularly haze.
• Overcome challenges in drone detection caused by scale variation, non-uniform object distribution, and image degradation due to haze.
• Leverage depth information as an auxiliary signal to improve detection robustness in hazy conditions without relying on image restoration.
• Develop a flexible, end-to-end framework that integrates depth perception into object detection for improved generalization under foggy conditions.

Proposed method

• Construct HazyDet, a large-scale dataset with 383,000 annotated instances from real hazy scenes and synthetically fogged normal scenes.
• Design the Multi-scale Depth-aware Detection Head (MDDH) to extract and fuse depth-aware features across multiple scales.
• Introduce the Dynamic Depth Condition Kernel (DCK) module to condition detection head predictions on learned depth cues for classification and regression.
• Propose a Scale Invariant Refurbishment Loss to improve the learning of depth cues from pseudo-labeled depth maps.
• Train DeCoDet end-to-end using paired images and pseudo-depth labels, enabling joint optimization of depth estimation and object detection.
• Integrate DeCoDet with various single-stage detectors (e.g., FCOS, VFNet) to evaluate generalization and adaptability.

Experimental results

Research questions

• RQ1 Can depth cues significantly improve object detection performance in hazy drone imagery when used as an auxiliary signal?
• RQ2 How effective is a depth-conditioned detection head in mitigating performance degradation caused by haze and scale variation?
• RQ3 To what extent does the Scale Invariant Refurbishment Loss enhance the learning of depth cues from noisy pseudo-labels?
• RQ4 How does DeCoDet generalize across different detector architectures under adverse weather conditions?
• RQ5 What is the impact of depth map quality on detection performance, and how does it affect model generalization?

Key findings

• DeCoDet achieves a mAP increase of 1.5 points on the HazyDet test set (from 45.9 to 47.4) when integrated with FCOS.
• On the RDDTS benchmark, FCOS-DeCoDet improves mAP from 22.8 to 24.3, demonstrating strong generalization to real-world hazy data.
• The performance gain is significantly reduced for VFNet-DeCoDet due to architectural incompatibility with depth cue integration, indicating detector-specific design constraints.
• High-quality depth maps lead to substantial performance boosts, while poor-quality pseudo-depth labels impair learning and degrade detection accuracy.
• Visualization shows that DeCoDet enhances feature attention on potential target regions, especially in dense and hazy scenes, improving localization precision.
• The proposed Scale Invariant Refurbishment Loss effectively stabilizes training and improves the network’s ability to learn robust depth cues from pseudo-labels.