[Paper Review] Cascaded Zoom-in Detector for High Resolution Aerial Images

Akhil Meethal, Éric Granger+1 more|arXiv (Cornell University)|Mar 15, 2023

Advanced Neural Network Applications4 citations

TL;DR

This paper proposes a Cascaded Zoom-in (CZ) detector that reuses the object detector itself to identify and process density crops—regions with high small-object concentration—in high-resolution aerial images. By adding a new 'density crop' class during training and performing two-stage inference on upsampled crops, the method improves small-object detection accuracy without adding external components, achieving a 3.0+ mAP gain on VisDrone and state-of-the-art performance.

ABSTRACT

Detecting objects in aerial images is challenging because they are typically composed of crowded small objects distributed non-uniformly over high-resolution images. Density cropping is a widely used method to improve this small object detection where the crowded small object regions are extracted and processed in high resolution. However, this is typically accomplished by adding other learnable components, thus complicating the training and inference over a standard detection process. In this paper, we propose an efficient Cascaded Zoom-in (CZ) detector that re-purposes the detector itself for density-guided training and inference. During training, density crops are located, labeled as a new class, and employed to augment the training dataset. During inference, the density crops are first detected along with the base class objects, and then input for a second stage of inference. This approach is easily integrated into any detector, and creates no significant change in the standard detection process, like the uniform cropping approach popular in aerial image detection. Experimental results on the aerial images of the challenging VisDrone and DOTA datasets verify the benefits of the proposed approach. The proposed CZ detector also provides state-of-the-art results over uniform cropping and other density cropping methods on the VisDrone dataset, increasing the detection mAP of small objects by more than 3 points.

Motivation & Objective

• To address the challenge of detecting small, sparsely distributed objects in high-resolution aerial images where standard detectors suffer from feature degradation due to downscaling.
• To eliminate the need for additional learnable components in density-based cropping methods, which complicate training and inference.
• To bridge the gap between research methods and practical deployment by making density cropping as simple as uniform cropping.
• To improve detection mAP, especially for small objects, by leveraging high-resolution crop inference without modifying the standard detection pipeline.

Proposed method

• The method introduces a new 'density crop' class during training, where density crops are automatically labeled using a non-differentiable crop labeling algorithm based on object density and spatial clustering.
• During training, the detector is fine-tuned to detect both base class objects and density crops, with the latter being rescaled to a larger resolution for enhanced feature learning.
• During inference, the detector first detects base objects and density crops on the full-resolution image in a single forward pass.
• The detected density crops are then upsampled to a fixed higher resolution and fed into a second-stage inference pass to refine small-object detection.
• The final detections are merged by combining results from the full-image and high-resolution crop inferences, with a 20-pixel threshold used to merge overlapping boxes.
• The entire pipeline reuses the standard detector architecture, avoiding additional modules or complex training stages.

Experimental results

Research questions

• RQ1 Can a detector be repurposed to detect density crops without adding external learnable components?
• RQ2 Does cascaded inference on upsampled density crops improve small-object detection in high-resolution aerial images compared to uniform cropping?
• RQ3 Can the proposed method achieve state-of-the-art performance while maintaining simplicity comparable to uniform cropping?
• RQ4 How does the performance of the CZ detector vary across object size categories, especially for small objects?

Key findings

• The proposed CZ detector achieves a 3.0+ mAP improvement on small objects (from 29.48 to 33.02) compared to the baseline on the VisDrone dataset.
• The method sets a new state-of-the-art mAP of 33.22 on VisDrone, outperforming all existing methods, including those using external modules like ClusterNet and DensityMap.
• The detector achieves a 2.5-point improvement in mAP for small objects (APs) compared to the next best method, with a 26.06 APs score.
• The method maintains high inference speed (12.69 FPS with FCOS) while significantly improving accuracy, demonstrating practicality.
• The two-stage inference with adaptive merging reduces false positives and improves localization, especially for small objects.
• The approach generalizes well, achieving a 33.67 mAP with FCOS, a modern anchor-free detector, showing robustness across architectures.