[Paper Review] Switching Convolutional Neural Network for Crowd Counting

Deepak Babu Sam, Shiv Surya+1 more|ePrints-IISc. (Indian Institute of Science Bangalore)|Aug 1, 2017

Video Surveillance and Tracking Methods20 references99 citations

TL;DR

Switch-CNN relays crowd scene patches to specialized CNN regressors with different receptive fields via a switch classifier, achieving state-of-the-art crowd counting across major datasets.

ABSTRACT

We propose a novel crowd counting model that maps a given crowd scene to its density. Crowd analysis is compounded by myriad of factors like inter-occlusion between people due to extreme crowding, high similarity of appearance between people and background elements, and large variability of camera view-points. Current state-of-the art approaches tackle these factors by using multi-scale CNN architectures, recurrent networks and late fusion of features from multi-column CNN with different receptive fields. We propose switching convolutional neural network that leverages variation of crowd density within an image to improve the accuracy and localization of the predicted crowd count. Patches from a grid within a crowd scene are relayed to independent CNN regressors based on crowd count prediction quality of the CNN established during training. The independent CNN regressors are designed to have different receptive fields and a switch classifier is trained to relay the crowd scene patch to the best CNN regressor. We perform extensive experiments on all major crowd counting datasets and evidence better performance compared to current state-of-the-art methods. We provide interpretable representations of the multichotomy of space of crowd scene patches inferred from the switch. It is observed that the switch relays an image patch to a particular CNN column based on density of crowd.

Motivation & Objective

• Address the challenge of crowd counting under scale, perspective, and occlusion variability.
• Leverage local density variation within an image by routing patches to specialized regressors.
• Develop an end-to-end Switch-CNN framework with differential, coupled, and switch training stages.

Proposed method

• Use three CNN regressors with distinct receptive fields to handle different crowd scales.
• Divide each image into 9 patches and route each patch to the regressor best suited for its density.
• Train a switch classifier (based on a VGG-16 backbone with GAP) to assign patches to regressors.
• Pretrain regressors, apply differential training to maximize per-patch count accuracy, and then perform coupled training to co-adapt switch and regressors.
• Generate ground-truth density maps using geometry-adaptive kernels or fixed spreads depending on dataset characteristics.
• Evaluate using MAE and MSE across standard crowd counting benchmarks.

Experimental results

Research questions

• RQ1 Can patch-level switching among regressors with different receptive fields improve density localization and count accuracy in crowded scenes?
• RQ2 Does a jointly trained switch classifier plus diverse regressors outperform single-model approaches across datasets with varying density and perspective?
• RQ3 How does differential training influence the partitioning of image patches into density-based groups and subsequent counting performance?

Key findings

• Switch-CNN achieves state-of-the-art MAE and MSE on ShanghaiTech Part A and Part B, outperforming MCNN and other methods.
• On ShanghaiTech Part A, Switch-CNN attains MAE 90.4 and MSE 135.0; on Part B, MAE 21.6 and MSE 33.4.
• On UCF_CC_50, Switch-CNN achieves MAE 318.1 and MSE 439.2, with switch accuracy 54.3%.
• On UCSD, Switch-CNN reports MAE 1.62 and MSE 2.10 with switch accuracy 60.9%.
• On WorldExpo’10, Switch-CNN averages MAE 9.4 with perspective maps and 11.2 without perspective maps, outperforming several baselines.
• Differential training creates a multichotomy of patches aligned with density, and coupled training further improves robustness of the switch and regressors.