[Paper Review] NOH-NMS: Improving Pedestrian Detection by Nearby Objects Hallucination

Penghao Zhou, Chong Zhou+5 more|arXiv (Cornell University)|Jul 27, 2020

Anomaly Detection Techniques and Applications35 references4 citations

TL;DR

This paper proposes NOH-NMS, a novel non-maximum suppression method for pedestrian detection that uses a Nearby Objects Hallucinator (NOH) to predict Gaussian distributions modeling nearby pedestrians from RoI features, enabling dynamic suppression relaxation in dense regions. It achieves state-of-the-art performance with 89.0% AP, 92.9% recall, and 43.9% MR-2 on CrowdHuman, improving over Greedy-NMS by 3.9% AP, 5.1% recall, and 0.8% MR-2.

ABSTRACT

Greedy-NMS inherently raises a dilemma, where a lower NMS threshold will potentially lead to a lower recall rate and a higher threshold introduces more false positives. This problem is more severe in pedestrian detection because the instance density varies more intensively. However, previous works on NMS don't consider or vaguely consider the factor of the existent of nearby pedestrians. Thus, we propose Nearby Objects Hallucinator (NOH), which pinpoints the objects nearby each proposal with a Gaussian distribution, together with NOH-NMS, which dynamically eases the suppression for the space that might contain other objects with a high likelihood. Compared to Greedy-NMS, our method, as the state-of-the-art, improves by $3.9\%$ AP, $5.1\%$ Recall, and $0.8\%$ $ ext{MR}^{-2}$ on CrowdHuman to $89.0\%$ AP and $92.9\%$ Recall, and $43.9\%$ $ ext{MR}^{-2}$ respectively.

Motivation & Objective

• To address the inherent dilemma in Greedy-NMS where a fixed IoU threshold cannot balance recall and false positives in high-density pedestrian scenes.
• To overcome the limitations of existing NMS variants like Soft-NMS and Adaptive-NMS, which fail to model spatial context of nearby pedestrians.
• To design a method that dynamically adjusts suppression based on the likelihood of nearby objects, improving detection in crowded scenarios.
• To achieve state-of-the-art pedestrian detection performance with minimal computational overhead and no additional annotations.

Proposed method

• The Nearby Objects Hallucinator (NOH) uses RoI features to predict a 4D Gaussian distribution modeling the expected location and size of nearby pedestrians.
• The Gaussian distribution encodes the mean (expected position and shape) and variance (uncertainty) of nearby objects, enabling spatial reasoning.
• NOH-NMS uses the hallucinated Gaussian to dynamically reduce suppression in regions with high likelihood of containing other pedestrians.
• The method integrates seamlessly into both one-stage and two-stage detectors with negligible computational cost.
• The Gaussian parameters are predicted from features, and the standard deviation σ can be trained end-to-end if KL loss is used, though it is set empirically in this work.
• The approach introduces two hyperparameters: density threshold dt and Gaussian standard deviation σ, both shown to be robust across wide ranges.

Experimental results

Research questions

• RQ1 Can modeling the spatial distribution of nearby pedestrians improve NMS performance in dense pedestrian detection?
• RQ2 Does a dynamic suppression strategy based on hallucinated object distributions reduce the trade-off between recall and false positives?
• RQ3 How robust is the proposed method to hyperparameter choices like density threshold dt and Gaussian standard deviation σ?
• RQ4 Can NOH-NMS achieve state-of-the-art performance without requiring additional annotations beyond full-body bounding boxes?

Key findings

• NOH-NMS achieves 89.0% AP, 92.9% recall, and 43.9% MR-2 on the CrowdHuman dataset, outperforming the strong Greedy-NMS baseline by 3.9% AP, 5.1% recall, and 0.8% MR-2.
• On CityPersons, the method achieves 10.8% MR-2, demonstrating strong generalization beyond CrowdHuman.
• The method is robust to hyperparameter choices: performance varies only slightly across a wide range of dt (0.2–0.5) and σ values.
• NOH-NMS is less sensitive to the IoU threshold Nt than Greedy-NMS, maintaining superior performance across all recall levels.
• Qualitative results confirm that NOH successfully hallucinates nearby pedestrians and enables NOH-NMS to recover detections missed by Greedy-NMS and Adaptive-NMS.
• The ablation study shows that NOH-NMS consistently outperforms Soft-NMS and Adaptive-NMS when re-implemented on the same strong baseline.