[Paper Review] Bayesian Loss for Crowd Count Estimation with Point Supervision

Zhiheng Ma, Xing Wei+2 more|arXiv (Cornell University)|Aug 10, 2019

Video Surveillance and Tracking Methods59 references66 citations

TL;DR

The paper introduces Bayesian loss for crowd counting using point annotations, supervising the count expectation rather than pixel-wise density, achieving state-of-the-art results on major benchmarks without external detectors.

ABSTRACT

In crowd counting datasets, each person is annotated by a point, which is usually the center of the head. And the task is to estimate the total count in a crowd scene. Most of the state-of-the-art methods are based on density map estimation, which convert the sparse point annotations into a "ground truth" density map through a Gaussian kernel, and then use it as the learning target to train a density map estimator. However, such a "ground-truth" density map is imperfect due to occlusions, perspective effects, variations in object shapes, etc. On the contrary, we propose \emph{Bayesian loss}, a novel loss function which constructs a density contribution probability model from the point annotations. Instead of constraining the value at every pixel in the density map, the proposed training loss adopts a more reliable supervision on the count expectation at each annotated point. Without bells and whistles, the loss function makes substantial improvements over the baseline loss on all tested datasets. Moreover, our proposed loss function equipped with a standard backbone network, without using any external detectors or multi-scale architectures, plays favourably against the state of the arts. Our method outperforms previous best approaches by a large margin on the latest and largest UCF-QNRF dataset. The source code is available at \url{https://github.com/ZhihengCV/Baysian-Crowd-Counting}.

Motivation & Objective

• Motivate crowd counting with point annotations and limitations of ground-truth density maps.
• Propose a Bayesian loss that supervises count expectations at annotated points.
• Show robustness, generalization, and state-of-the-art performance across benchmarks.

Proposed method

• Construct a density contribution probability model p(xm|yn) from point annotations.
• Compute posterior p(yn|xm) using Bayes’ rule with equal priors.
• Define Bayes loss LBayes by supervising the count expectation at annotated points (sum of p(yn|xm)Dest(xm)).
• Extend to a background label y0 with a dummy background point to model background pixels (LBayes+).
• Visualize label posterior entropy for analysis of localization and boundaries.

Experimental results

Research questions

• RQ1 Can a loss function based on count-centric Bayesian supervision outperform pixel-wise density supervision on standard crowd counting benchmarks?
• RQ2 Does incorporating a background model (Bayesian+ with dummy point) improve robustness to background pixels and annotation noise?
• RQ3 How do the proposed losses perform with different backbones and across datasets?
• RQ4 What is the impact of Gaussian kernel parameters and margin d on performance and robustness?

Key findings

• BAYESIAN+ achieves state-of-the-art accuracy on four benchmark datasets without external detectors or multi-scale architectures.
• BAYESIAN+ consistently improves BAYESIAN by about 3% across all4 datasets.
• Both BAYESIAN and BAYESIAN+ outperform the BASELINE by large margins on all datasets.
• On UCF-QNRF, BAYESIAN+ notably outperforms the previous best method CL-CNN by a sizable margin (as reported).
• The method yields more accurate density localization, especially in dense regions, and better background handling with y0 modeling.