[Paper Review] NMS by Representative Region: Towards Crowded Pedestrian Detection by Proposal Pairing

Xin Huang, Zheng Ge+2 more|arXiv (Cornell University)|Mar 28, 2020

Video Surveillance and Tracking Methods29 references31 citations

TL;DR

Proposes R2NMS and a Paired-Box Model to exploit visible regions for better NMS in crowded scenes, achieving state-of-the-art results on CrowdHuman and CityPersons.

ABSTRACT

Although significant progress has been made in pedestrian detection recently, pedestrian detection in crowded scenes is still challenging. The heavy occlusion between pedestrians imposes great challenges to the standard Non-Maximum Suppression (NMS). A relative low threshold of intersection over union (IoU) leads to missing highly overlapped pedestrians, while a higher one brings in plenty of false positives. To avoid such a dilemma, this paper proposes a novel Representative Region NMS approach leveraging the less occluded visible parts, effectively removing the redundant boxes without bringing in many false positives. To acquire the visible parts, a novel Paired-Box Model (PBM) is proposed to simultaneously predict the full and visible boxes of a pedestrian. The full and visible boxes constitute a pair serving as the sample unit of the model, thus guaranteeing a strong correspondence between the two boxes throughout the detection pipeline. Moreover, convenient feature integration of the two boxes is allowed for the better performance on both full and visible pedestrian detection tasks. Experiments on the challenging CrowdHuman and CityPersons benchmarks sufficiently validate the effectiveness of the proposed approach on pedestrian detection in the crowded situation.

Motivation & Objective

• Motivate improved pedestrian detection in crowded scenes with heavy occlusion.
• Introduce a new NMS variant that leverages visible regions to reduce false positives and preserve true positives.
• Develop PBM to jointly predict full-body and visible-body boxes with strong correspondence.
• Demonstrate state-of-the-art performance on CrowdHuman and CityPersons benchmarks.

Proposed method

• Introduce NMS by Representative Region (R2NMS) that uses visible-region IoU to decide overlaps between full-body boxes.
• Develop Paired-Box Model (PBM) consisting of Paired RPN, Paired Feature Extractor, and Paired R-CNN to predict full and visible boxes from the same anchor.
• Fuse features from paired full/visible proposals via Paired Proposal Feature Extractor (PPFE) for improved full and visible pedestrian detection.
• Train with a paired-sample unit where ground-truth pairs Q=(F,V) guide anchor assignment and regression.
• Evaluate against CrowdHuman and CityPersons using MR, AP, and Recall metrics; compare with AdaptiveNMS and Repulsion Loss.

Experimental results

Research questions

• RQ1 Can leveraging visible regions via a paired-box representation improve NMS in crowded pedestrian detections?
• RQ2 Does R2NMS reduce redundant detections without inflating false positives in high-occlusion scenarios?
• RQ3 How much performance gain do paired full/visible boxes and PPFE provide over standard Faster R-CNN baselines?
• RQ4 What is the impact of PBM+R2NMS on state-of-the-art benchmarks (CrowdHuman, CityPersons) across full-body and visible-region tasks?

Key findings

• R2NMS improves detection metrics (AP and Recall) over baselines, indicating better handling of crowded scenes.
• PBM with PPFE and R2NMS achieves state-of-the-art results on CrowdHuman with MR dropping from 50.42 to 43.35, AP rising to 89.29, and Recall to 93.33.
• On CityPersons, PBM with mask and R2NMS reduces MR on reasonable and heavy-occlusion subsets (e.g., R from 13.8 to 11.1; HO from 59.0 to 53.3).
• PPFE-based feature integration, especially with attention, yields notable gains over simple concatenation.
• R2NMS enhances AP and Recall across settings, while MR improvements depend on the quality of visible predictions; overall the method shows strong crowd-detection advantages.