[Paper Review] Reinforcing Local Feature Representation for Weakly-Supervised Dense Crowd Counting

Xiaohong Chen, Hongtao Lu|arXiv (Cornell University)|Feb 22, 2022

Video Surveillance and Tracking Methods4 citations

TL;DR

This paper proposes a self-adaptive feature similarity learning (SFSL) network and global-local consistency (GLC) loss to enhance local feature representation in weakly-supervised dense crowd counting, where only global crowd numbers are available. By learning unbiased feature estimations and enforcing consistency between global and local predictions, the method achieves performance close to fully-supervised methods with significantly reduced annotation effort.

ABSTRACT

Fully-supervised crowd counting is a laborious task due to the large amounts of annotations. Few works focus on weekly-supervised crowd counting, where only the global crowd numbers are available for training. The main challenge of weekly-supervised crowd counting is the lack of local supervision information. To address this problem, we propose a self-adaptive feature similarity learning (SFSL) network and a global-local consistency (GLC) loss to reinforce local feature representation. We introduce a feature vector which represents the unbiased feature estimation of persons. The network updates the feature vector self-adaptively and utilizes the feature similarity for the regression of crowd numbers. Besides, the proposed GLC loss leverages the consistency between the network estimations from global and local areas. The experimental results demonstrate that our proposed method based on different backbones narrows the gap between weakly-supervised and fully-supervised dense crowd counting.

Motivation & Objective

• Address the challenge of limited local supervision in weakly-supervised crowd counting, where only global crowd numbers are available.
• Improve local feature discriminability despite the absence of instance-level annotations.
• Reduce dependency on expensive manual annotations by leveraging global crowd numbers and self-supervised local supervision.
• Enhance model robustness to labeling deviations common in crowd sensing-based annotation approaches.
• Narrow the performance gap between weakly-supervised and fully-supervised crowd counting methods.

Proposed method

• Introduce a self-adaptive feature similarity learning (SFSL) network that maintains an unbiased feature estimation vector as a cluster center for the positive class (persons).
• Update the feature estimation vector via gradient descent during training to adaptively refine the representation of person features.
• Compute pixel-wise feature similarity between the learned feature vector and backbone features to generate a soft binary classification map for density estimation.
• Use the similarity map as input to a linear regression head to predict the final crowd count.
• Propose a global-local consistency (GLC) loss that enforces agreement between global predictions and local predictions from subimages.
• Train local regression in a self-supervised manner using global predictions as supervision, improving local feature learning with minimal annotations.

Experimental results

Research questions

• RQ1 Can local feature representation be effectively reinforced in weakly-supervised crowd counting without instance-level annotations?
• RQ2 How can global crowd numbers be leveraged to supervise local region predictions and improve feature discriminability?
• RQ3 To what extent can the model maintain accuracy under labeling deviations common in automated annotation methods?
• RQ4 How does the proposed method compare to fully-supervised baselines in terms of performance and annotation efficiency?
• RQ5 Can the method generalize across diverse crowd densities, perspectives, and object scales with different backbones?

Key findings

• The proposed method achieves competitive performance on four crowd counting datasets, significantly narrowing the gap between weakly-supervised and fully-supervised methods.
• With a CNN backbone, the method achieves a mean absolute error (MAE) of 12.8 on the ShanghaiTech Part A dataset, approaching fully-supervised baselines.
• The model demonstrates strong robustness to labeling noise: MAE increases by only 3.5% and MSE by 0.9% when standard deviation of Gaussian noise reaches σ=0.1.
• When trained with patch-wise labels (6× more annotations), performance improves substantially, indicating strong potential for further gains with more global labels.
• The method generalizes across different backbones, including both CNN and Vision Transformer (ViT), showing broad applicability.
• The GLC loss effectively improves local feature learning by enforcing consistency between global and local predictions, even without explicit local supervision.