[Paper Review] Beta R-CNN: Looking into Pedestrian Detection from Another Perspective

Zixuan Xu, Banghuai Li+2 more|arXiv (Cornell University)|Oct 23, 2022

Advanced Neural Network Applications18 references23 citations

TL;DR

The paper introduces Beta Representation based on a 2D beta distribution to model pedestrians, along with Beta R-CNN (BetaHead and BetaMask) and BetaNMS, to improve detection in occluded and crowded scenes.

ABSTRACT

Recently significant progress has been made in pedestrian detection, but it remains challenging to achieve high performance in occluded and crowded scenes. It could be attributed mostly to the widely used representation of pedestrians, i.e., 2D axis-aligned bounding box, which just describes the approximate location and size of the object. Bounding box models the object as a uniform distribution within the boundary, making pedestrians indistinguishable in occluded and crowded scenes due to much noise. To eliminate the problem, we propose a novel representation based on 2D beta distribution, named Beta Representation. It pictures a pedestrian by explicitly constructing the relationship between full-body and visible boxes, and emphasizes the center of visual mass by assigning different probability values to pixels. As a result, Beta Representation is much better for distinguishing highly-overlapped instances in crowded scenes with a new NMS strategy named BetaNMS. What's more, to fully exploit Beta Representation, a novel pipeline Beta R-CNN equipped with BetaHead and BetaMask is proposed, leading to high detection performance in occluded and crowded scenes.

Motivation & Objective

• Motivate improved pedestrian detection in occluded and crowded scenes beyond traditional 2D bounding boxes.
• Propose a Beta Representation that integrates full-body and visible patterns into a single probabilistic model.
• Develop a detector (Beta R-CNN) with BetaHead and BetaMask to exploit the Beta Representation for better localization and discrimination.
• Introduce a beta-distribution-based NMS (BetaNMS) using KL-divergence to distinguish highly overlapped instances.

Proposed method

• Define a 2D beta distribution as Beta Representation, parameterized by eight values [l,t,r,b,alpha_x,beta_x,alpha_y,beta_y] derived from full-body and visible boxes.
• Compute mean and variance along x and y to obtain regression targets [l,t,r,b, mu_x, mu_y, sigma_x, sigma_y], with normalization as described.
• Introduce BetaHead to regress the eight beta parameters (via four boundary params and four shape params) using a SmoothL1 loss.
• Introduce BetaMask to modulate RoI features with masks sampled from the predicted 2D beta distributions to emphasize visible regions, trained with a KL divergence loss against ground-truth beta masks.
• Adopt KL divergence as the distance metric for a Beta-based NMS (BetaNMS), using a symmetrized KL divergence between beta distributions to suppress overlaps more effectively than IoU-based NMS.

Experimental results

Research questions

• RQ1 Can a beta-distribution-based representation better discriminate occluded and highly overlapped pedestrians than traditional bounding boxes?
• RQ2 Does BetaHead/BetaMask improve localization and recognition when pedestrians are occluded or in crowded scenes?
• RQ3 Is BetaNMS more effective than IoU-based NMS for crowded pedestrian scenarios?

Key findings

• Beta Representation concentrates on the center of visual mass and handles visibility variations better than a uniform box, aiding discrimination in occlusion and crowding.
• BetaHead and BetaMask improve MR^-2 and AP on crowded datasets, demonstrating benefits in occlusion- and crowd-driven scenarios.
• BetaNMS using KL divergence outperforms IoU-based NMS and other alternatives for highly overlapped instances.
• Beta R-CNN with the proposed components achieves state-of-the-art or competitive results on CrowdHuman and CityPersons benchmarks, especially in heavy occlusion/subset scenes.