[Paper Review] Locate, Size and Count: Accurately Resolving People in Dense Crowds via Detection

Deepak Babu Sam, Skand Vishwanath Peri+3 more|arXiv (Cornell University)|Jun 18, 2019

Video Surveillance and Tracking MethodsComputer Science46 references33 citations

TL;DR

The paper presents LSC-CNN, a dense detection framework that localizes, sizes, and counts heads in crowds—surpassing density-regression methods in localization and counting.

ABSTRACT

We introduce a detection framework for dense crowd counting and eliminate the need for the prevalent density regression paradigm. Typical counting models predict crowd density for an image as opposed to detecting every person. These regression methods, in general, fail to localize persons accurate enough for most applications other than counting. Hence, we adopt an architecture that locates every person in the crowd, sizes the spotted heads with bounding box and then counts them. Compared to normal object or face detectors, there exist certain unique challenges in designing such a detection system. Some of them are direct consequences of the huge diversity in dense crowds along with the need to predict boxes contiguously. We solve these issues and develop our LSC-CNN model, which can reliably detect heads of people across sparse to dense crowds. LSC-CNN employs a multi-column architecture with top-down feedback processing to better resolve persons and produce refined predictions at multiple resolutions. Interestingly, the proposed training regime requires only point head annotation, but can estimate approximate size information of heads. We show that LSC-CNN not only has superior localization than existing density regressors, but outperforms in counting as well. The code for our approach is available at https://github.com/val-iisc/lsc-cnn.

Motivation & Objective

• Motivate moving beyond density regression to accurate head localization in dense crowds.
• Develop a single-stage, dense detection framework tailored for extreme crowd density and varied scales.
• Enable head localization with bounding box predictions derived from point head annotations.
• Offer a training regime that estimates bounding box sizes without requiring explicit box annotations.

Proposed method

• Propose LSC-CNN, a dense detection architecture that predicts per-pixel class confidences for predefined head box sizes across multiple scales.
• Use a multi-scale feature extractor based on a modified VGG-16 to produce one-half, one-fourth, one-eighth, and one-sixteenth resolution feature maps.
• Incorporate Top-down Feature Modulators (TFMs) to fuse multi-scale features and provide context for precise localizations.
• Train with per-pixel cross-entropy loss over predefined box classes, using pseudo ground truth derived from point annotations.
• Employ a Grid Winner-Take-All (GWTA) training loss to focus learning on hard regions and mitigate local minima, with scale- and class-balanced weighting.
• Approximate head sizes by nearest-neighbor distances to generate pseudo ground truth box bins across scales.

Experimental results

Research questions

• RQ1 Can dense crowd counting be effectively reframed as a per-pixel head-detection problem rather than density regression?
• RQ2 How can multi-scale features and top-down context improve localization and sizing of heads in highly dense crowds?
• RQ3 Is it feasible to train a head-detection model using only point head annotations and no bounding box annotations?
• RQ4 Does a per-pixel box-classification approach with pseudo ground truth provide accurate counting across density ranges?

Key findings

• LSC-CNN achieves superior localization compared with density regressor approaches.
• The model provides bounding boxes for heads and yields accurate counts across varying crowd densities.
• Top-down feature modulation helps resolve people at multiple scales and reduces false detections in cluttered scenes.
• Training with GWTA loss and point-based pseudo supervision yields effective end-to-end learning without explicit box annotations.
• The approach enables high-resolution detections beyond typical face detectors, suitable for dense crowds.