[论文解读] CNN-based Density Estimation and Crowd Counting: A Survey

Guangshuai Gao, Junyu Gao+3 more|arXiv (Cornell University)|Mar 28, 2020

Video Surveillance and Tracking Methods参考文献 219被引用 141

一句话总结

本论文综述基于卷积神经网络的密度估计与人群计数模型，分析架构、学习范式与评估数据集，提供密度图、测试结果与待解决的问题，以指引未来研究。

ABSTRACT

Accurately estimating the number of objects in a single image is a challenging yet meaningful task and has been applied in many applications such as urban planning and public safety. In the various object counting tasks, crowd counting is particularly prominent due to its specific significance to social security and development. Fortunately, the development of the techniques for crowd counting can be generalized to other related fields such as vehicle counting and environment survey, if without taking their characteristics into account. Therefore, many researchers are devoting to crowd counting, and many excellent works of literature and works have spurted out. In these works, they are must be helpful for the development of crowd counting. However, the question we should consider is why they are effective for this task. Limited by the cost of time and energy, we cannot analyze all the algorithms. In this paper, we have surveyed over 220 works to comprehensively and systematically study the crowd counting models, mainly CNN-based density map estimation methods. Finally, according to the evaluation metrics, we select the top three performers on their crowd counting datasets and analyze their merits and drawbacks. Through our analysis, we expect to make reasonable inference and prediction for the future development of crowd counting, and meanwhile, it can also provide feasible solutions for the problem of object counting in other fields. We provide the density maps and prediction results of some mainstream algorithm in the validation set of NWPU dataset for comparison and testing. Meanwhile, density map generation and evaluation tools are also provided. All the codes and evaluation results are made publicly available at https://github.com/gaoguangshuai/survey-for-crowd-counting.

研究动机与目标

• 提供对基于CNN的密度估计与人群计数模型的全面综述。
• 按网络架构、监督形式、学习范式以及跨域泛化对方法进行分类。
• 分析影响性能的因素，识别表现最优的方法及其优缺点。
• 展示数据集、评估指标和基准结果，以指导未来研究。
• 提出数据集、模型以及对其他领域的泛化等方面的开放问题与未来方向。

提出的方法

• 对基于CNN的人群计数模型进行调查并按基础、多列与单列架构进行分类。
• 检查包括单任务与多任务框架在内的学习范式。
• 讨论作为补丁为基准与整张图像为基础的推理方式。
• 回顾从实例级到弱监督与多任务设置的监督形式。
• 提供密度图生成与评估工具，并公开分享代码与结果。
• 在标准数据集上对代表性模型进行基准测试，并分析其优点与局限。

实验结果

研究问题

• RQ1 哪些是用于人群计数与密度估计的代表性基于CNN的网络架构？
• RQ2 学习范式与监督形式如何影响性能与泛化？
• RQ3 哪些数据集与评估指标最能反映基于CNN的人群计数进展，哪些模型在基准测试中居于前列？
• RQ4 哪些开放挑战仍然存在，哪些方向最有前景适用于未来工作？

主要发现

• 本次综述系统性地评估了基于CNN的密度估计人群计数模型，并提供了分类法与分析。
• 它在代表性数据集上识别出表现最佳的方法并分析其优点与缺点。
• 主流算法的密度图与预测结果在 NWPU 验证集上提供以便比较。
• 作者提供密度图生成与评估工具，并公开发布代码与结果以提高可重复性。
• 该工作讨论了模型设计、数据集收集与领域自适应方面的开放问题、挑战与未来方向。
• 本文将从基础CNN到多列和单列架构及其学习范式与监督水平的发展脉络进行脉络化。