[Paper Review] CrowdMLP: Weakly-Supervised Crowd Counting via Multi-Granularity MLP

Mingjie Wang, Jun Zhou+2 more|arXiv (Cornell University)|Mar 15, 2022

Video Surveillance and Tracking Methods4 citations

TL;DR

CrowdMLP proposes a weakly-supervised crowd counting method that regresses total crowd counts using only count-level annotations, leveraging a multi-granularity MLP regressor to model global dependencies and a self-supervised proxy task (Split-Counting) to improve generalization. It achieves performance on par with state-of-the-art location-level supervised methods while eliminating the need for costly density map annotations.

ABSTRACT

Existing state-of-the-art crowd counting algorithms rely excessively on location-level annotations, which are burdensome to acquire. When only count-level (weak) supervisory signals are available, it is arduous and error-prone to regress total counts due to the lack of explicit spatial constraints. To address this issue, a novel and efficient counter (referred to as CrowdMLP) is presented, which probes into modelling global dependencies of embeddings and regressing total counts by devising a multi-granularity MLP regressor. In specific, a locally-focused pre-trained frontend is cascaded to extract crude feature maps with intrinsic spatial cues, which prevent the model from collapsing into trivial outcomes. The crude embeddings, along with raw crowd scenes, are tokenized at different granularity levels. The multi-granularity MLP then proceeds to mix tokens at the dimensions of cardinality, channel, and spatial for mining global information. An effective proxy task, namely Split-Counting, is also proposed to evade the barrier of limited samples and the shortage of spatial hints in a self-supervised manner. Extensive experiments demonstrate that CrowdMLP significantly outperforms existing weakly-supervised counting algorithms and performs on par with state-of-the-art location-level supervised approaches.

Motivation & Objective

• To reduce the annotation burden in crowd counting by eliminating the need for expensive location-level (density map) annotations.
• To develop a robust regression model that generalizes well under weak supervision with only total count labels.
• To mitigate performance degradation caused by lack of spatial supervision in count-level learning.
• To introduce a self-supervised proxy task that implicitly decouples spatial constraints and improves feature learning.
• To enable scalable, large-scale dataset creation using easily obtainable count-level annotations.

Proposed method

• A locally-focused pre-trained CNN frontend extracts low-level features with intrinsic spatial cues to prevent model collapse.
• Raw input images and feature maps are tokenized at multiple granularity levels (cardinality, channel, spatial) to form hierarchical token sequences.
• A multi-granularity MLP regressor mixes tokens across dimensions to capture long-range global dependencies and model global context.
• A self-supervised proxy task, Split-Counting, is introduced to regularize learning by splitting images into regions and predicting sub-counts.
• The model uses a high-level MLP head to directly predict a single continuous count value from global embeddings.
• T-SNE visualization is used to analyze learned feature representations and assess clustering by scene type and density.

Figure 1: The categories of crowd counter under distinct supervisory signals. (a) Learn from an entire set of location-level density maps. (b) Learn from density maps of partially selected images. (c) Learn from density maps of partial regions in crowd scenes. (d) Our approach learns directly from l

Experimental results

Research questions

• RQ1 Can a purely MLP-based architecture effectively model global context in crowd counting under weak supervision?
• RQ2 How does multi-granularity token mixing improve performance compared to standard MLPs or CNNs?
• RQ3 To what extent does the self-supervised Split-Counting proxy task reduce error and improve generalization without spatial supervision?
• RQ4 Can a model trained only on count-level labels achieve performance comparable to those trained with full density map supervision?
• RQ5 How well does the model handle scale variation, density shifts, and semantic changes across diverse scenes?

Key findings

• CrowdMLP achieves a mean absolute error (MAE) of 57.828 on ShanghaiTech Part A, matching the performance of state-of-the-art location-level supervised models.
• The model outperforms both CNN-based (Crowd-CNN) and Transformer-based (Crowd-Transformer) baselines under the same parameter budget.
• The inclusion of the Split-Counting proxy task improves MAE by 0.55% and MSE by 0.32%, confirming its effectiveness in reducing empirical error.
• T-SNE visualization shows that the model learns discriminative, scene-aware features, with high-density images clustering together and distinct groups forming based on scene semantics.
• Sliding-window inference demonstrates that CrowdMLP can distinguish foreground people from background and adapt to drastic scale and density variations across a single image.
• The model achieves strong performance with only 26.63M parameters, outperforming a larger 36.43M-parameter Transformer-based variant.

Figure 2: The overall architecture of CrowdMLP, which is featured by coarse-to-fine token streams, multi-granularity MLP, and self-supervised proxy task.