[论文解读] JHU-CROWD++: Large-Scale Crowd Counting Dataset and A Benchmark Method

Vishwanath A. Sindagi, Rajeev Yasarla+1 more|arXiv (Cornell University)|Apr 7, 2020

Video Surveillance and Tracking Methods被引用 5

一句话总结

本文提出了JHU-CROWD++，一个大规模、多样化的群体计数数据集，包含4,372张图像和151万个标注，涵盖雨天、雪天、雾霾等具有挑战性的环境条件以及干扰物。本文提出了一种置信度引导的深度残差计数网络（CG-DRCN），采用基于不确定性的置信度加权机制进行渐进式残差学习，以优化密度图，从而在ShanghaiTech和UCF-QNRF基准上实现了最先进性能，平均绝对误差（MAE）分别为60.2和95.5。

ABSTRACT

Due to its variety of applications in the real-world, the task of single image-based crowd counting has received a lot of interest in the recent years. Recently, several approaches have been proposed to address various problems encountered in crowd counting. These approaches are essentially based on convolutional neural networks that require large amounts of data to train the network parameters. Considering this, we introduce a new large scale unconstrained crowd counting dataset (JHU-CROWD++) that contains "4,372" images with "1.51 million" annotations. In comparison to existing datasets, the proposed dataset is collected under a variety of diverse scenarios and environmental conditions. Specifically, the dataset includes several images with weather-based degradations and illumination variations, making it a very challenging dataset. Additionally, the dataset consists of a rich set of annotations at both image-level and head-level. Several recent methods are evaluated and compared on this dataset. The dataset can be downloaded from http://www.crowd-counting.com . Furthermore, we propose a novel crowd counting network that progressively generates crowd density maps via residual error estimation. The proposed method uses VGG16 as the backbone network and employs density map generated by the final layer as a coarse prediction to refine and generate finer density maps in a progressive fashion using residual learning. Additionally, the residual learning is guided by an uncertainty-based confidence weighting mechanism that permits the flow of only high-confidence residuals in the refinement path. The proposed Confidence Guided Deep Residual Counting Network (CG-DRCN) is evaluated on recent complex datasets, and it achieves significant improvements in errors.

研究动机与目标

• 解决现有群体计数数据集中存在的多样性不足、偏差以及缺乏环境退化标注等问题。
• 开发一种在恶劣天气和复杂背景条件下具有良好泛化能力的鲁棒群体计数模型。
• 通过利用残差学习对密度图进行渐进式优化，提升计数精度。
• 将图像级标注（如天气、遮挡）融入学习过程，以增强模型鲁棒性。
• 基于大规模、无约束且标注丰富的数据集，建立群体计数的新基准。

提出的方法

• 提出一种新型置信度引导的深度残差计数网络（CG-DRCN），采用VGG16作为主干网络，生成粗粒度密度图。
• 采用渐进式优化策略，每一阶段通过残差学习对前一阶段的密度图进行细化。
• 引入基于不确定性的置信度加权模块，在优化过程中仅筛选并传播高置信度残差。
• 利用图像级标签（如天气、模糊、遮挡）进行类别条件控制，以引导特征学习并提升模型鲁棒性。
• 在JHU-CROWD++数据集上，采用多尺度监督，使用标准损失函数端到端训练网络。
• 采用双分支结构：一个分支用于初始密度预测，另一个分支用于带置信门控的残差优化。

Figure 2: Examples of head-level annotations: (a) Dots (b) Approximate sizes (c) Blur-level.

实验结果

研究问题

• RQ1 大规模、多样化的群体计数数据集（含环境退化与丰富标注）在多大程度上提升模型泛化能力？
• RQ2 基于置信度加权的渐进式残差学习能否有效提升密度图优化效果并减少计数误差？
• RQ3 图像级标注（如天气、遮挡）在恶劣环境下的性能提升作用有多大？
• RQ4 所提出的CG-DRCN在ShanghaiTech和UCF-QNRF等基准数据集上与最先进方法相比表现如何？
• RQ5 基于不确定性的残差选择机制对模型稳定性与准确率有何影响？

主要发现

• 采用ResNet-101主干网络的CG-DRCN在ShanghaiTech Part A基准上实现了60.2的平均绝对误差（MAE），优于表XII中列出的所有先前方法。
• 在UCF-QNRF数据集上，CG-DRCN-Res101模型实现了95.5的MAE，优于大多数近期方法，已接近最先进水平。
• 基于不确定性的置信度加权机制能有效过滤低置信度残差，显著提升优化过程的稳定性和准确性。
• 引入图像级标注（如天气、遮挡）显著提升了模型在恶劣环境下的性能，尤其在雨天、雪天和雾霾场景中表现突出。
• JHU-CROWD++数据集展现出卓越的多样性与真实性，包含4,372张图像和151万个头部标注，均在真实世界条件下采集，涵盖严重天气和复杂背景。
• 消融实验表明，残差优化与置信度加权两个组件对最终性能提升均有显著贡献。

Figure 3: Summary of keywords used to scrape the internet for images.