[Paper Review] A Normalized Gaussian Wasserstein Distance for Tiny Object Detection

Jinwang Wang, Chang Xu+2 more|arXiv (Cornell University)|Oct 26, 2021

Advanced Neural Network ApplicationsComputer Science38 references256 citations

TL;DR

The paper proposes Normalized Wasserstein Distance (NWD) as a replacement for IoU to measure bounding-box similarity for tiny objects, improving label assignment, NMS, and regression loss in anchor-based detectors, achieving state-of-the-art results on AI-TOD and strong gains on VisDrone2019.

ABSTRACT

Detecting tiny objects is a very challenging problem since a tiny object only contains a few pixels in size. We demonstrate that state-of-the-art detectors do not produce satisfactory results on tiny objects due to the lack of appearance information. Our key observation is that Intersection over Union (IoU) based metrics such as IoU itself and its extensions are very sensitive to the location deviation of the tiny objects, and drastically deteriorate the detection performance when used in anchor-based detectors. To alleviate this, we propose a new evaluation metric using Wasserstein distance for tiny object detection. Specifically, we first model the bounding boxes as 2D Gaussian distributions and then propose a new metric dubbed Normalized Wasserstein Distance (NWD) to compute the similarity between them by their corresponding Gaussian distributions. The proposed NWD metric can be easily embedded into the assignment, non-maximum suppression, and loss function of any anchor-based detector to replace the commonly used IoU metric. We evaluate our metric on a new dataset for tiny object detection (AI-TOD) in which the average object size is much smaller than existing object detection datasets. Extensive experiments show that, when equipped with NWD metric, our approach yields performance that is 6.7 AP points higher than a standard fine-tuning baseline, and 6.0 AP points higher than state-of-the-art competitors. Codes are available at: https://github.com/jwwangchn/NWD.

Motivation & Objective

• Analyze why IoU is highly sensitive to location deviations for tiny objects.
• Develop a distribution-based bounding box representation to better capture tiny object geometry.
• Introduce Normalized Gaussian Wasserstein Distance (NWD) to replace IoU in assignment, NMS, and loss.
• Demonstrate that NWD-based detectors achieve significant performance gains on TOD benchmarks.

Proposed method

• Model bounding boxes as 2D Gaussian distributions with means at the box centers and variances proportional to box width/height.
• Define the Normalized Wasserstein Distance (NWD) as an exponential transform of the 2nd-order Wasserstein distance between the Gaussian models.
• Integrate NWD into anchor-based detectors to replace IoU in three modules: label assignment, NMS, and regression loss (IoU-Loss alternative).
• Perform ablation studies on Faster R-CNN with AI-TOD to evaluate NWD in individual modules and in combinations.
• Evaluate on AI-TOD and VisDrone2019 to compare against IoU-based baselines across AP metrics, including tiny and very tiny object ranges.

Experimental results

Research questions

• RQ1 Can NWD provide a more stable and scale-invariant similarity measure for tiny bounding boxes than IoU?
• RQ2 How does substituting IoU with NWD in label assignment, NMS, and loss affect detection performance for tiny objects?
• RQ3 Is NWD applicable to both single-stage and multi-stage anchor-based detectors for TOD?
• RQ4 What are the empirical gains of NWD on AI-TOD and VisDrone2019 datasets across standard TOD metrics?

Key findings

• NWD-based label assignment yields higher AP and especially AP_t (tiny objects) than IoU-based assignment in Faster R-CNN on AI-TOD.
• NWD improves NMS performance for tiny objects over IoU in the RPN stage.
• NWD-based regression loss provides competitive or better AP than IoU-based losses, particularly for tiny objects.
• Across detectors (RetinaNet, ATSS, Faster R-CNN, Cascade R-CNN, DetectoRS), NWD-based variants consistently improve AP, with DetectoRS* reaching state-of-the-art 20.8 AP on AI-TOD.
• On VisDrone2019, NWD-based variants demonstrate notable gains in AP_t and related metrics, indicating generalization to UAV datasets.