[Paper Review] Mixture of Counting CNNs: Adaptive Integration of CNNs Specialized to Specific Appearance for Crowd Counting

Shohei Kumagai, Kazuhiro Hotta+1 more|arXiv (Cornell University)|Mar 28, 2017

Video Surveillance and Tracking Methods17 references61 citations

TL;DR

The paper introduces MoC-CNN, a mixture of expert CNNs with a gating network that adaptively weighs specialized experts to handle appearance and scale changes in crowd counting.

ABSTRACT

This paper proposes a crowd counting method. Crowd counting is difficult because of large appearance changes of a target which caused by density and scale changes. Conventional crowd counting methods generally utilize one predictor (e,g., regression and multi-class classifier). However, such only one predictor can not count targets with large appearance changes well. In this paper, we propose to predict the number of targets using multiple CNNs specialized to a specific appearance, and those CNNs are adaptively selected according to the appearance of a test image. By integrating the selected CNNs, the proposed method has the robustness to large appearance changes. In experiments, we confirm that the proposed method can count crowd with lower counting error than a CNN and integration of CNNs with fixed weights. Moreover, we confirm that each predictor automatically specialized to a specific appearance.

Motivation & Objective

• Address large appearance changes in crowd counting due to density and scale.
• Develop a robust method that uses multiple CNN predictors specialized to different appearances.
• Enable adaptive selection and integration of expert CNNs to improve counting accuracy over single predictors.
• Provide end-to-end training where experts specialize automatically without manual role assignment.
• Demonstrate competitiveness on challenging crowd counting datasets.

Proposed method

• Define K expert CNNs each predicting a count for an image patch.
• Introduce a gating CNN that outputs weights g_k via softmax to blend expert outputs.
• Count y as a weighted sum y = sum_k g_k e_k of expert counts.
• Train experts with L_expert loss to minimize (t_n - sum_k g_nk e_nk)^2.
• Train gating CNN with L_gate loss including a variance regularization term to prevent collapse to a single expert.
• Use end-to-end Adam optimization with separate learning rates for experts and gating network.

Experimental results

Research questions

• RQ1 Can a mixture of specialized CNNs outperform a single predictor for crowd counting under varied appearances?
• RQ2 Does adaptive gating promote specialization of experts and improve counting accuracy across densities and scales?
• RQ3 How does MoC-CNN compare to fixed-weight integration and single CNN baselines on challenging datasets?

Key findings

• MoC-CNN achieves lower MAE and MSD than a single CNN and fixed-weight integration on UCF_CC_50.
• On UCF_CC_50, MoC-CNN attains MAE 361.7 and MSD 493.3, outperforming Ordinary CNN (545.6, 697.5) and Fc-layer Gating (509.6, 670.0).
• MoC-CNN attains competitive results with state-of-the-art methods on UCF_CC_50.
• Gating CNN outputs indicate specialization: different gates respond to sparse, middle, and dense crowds, validating adaptive expert selection.
• The method demonstrates robustness to appearance changes by integrating specialized experts rather than relying on a single predictor.