[论文解读] Beyond Counting: Comparisons of Density Maps for Crowd Analysis Tasks - Counting, Detection, and Tracking

Di Kang, Zheng Ma+1 more|arXiv (Cornell University)|May 29, 2017

Video Surveillance and Tracking Methods参考文献 35被引用 24

一句话总结

本文评估了人群分析中密度图估计方法，比较了在计数、检测和跟踪任务中使用低分辨率与全分辨率密度图的性能。尽管低分辨率图在计数任务中表现良好，但通过滑动窗口卷积神经网络（CNN-pixel）和带跳跃连接的全卷积网络（FCNN-skip）生成的全分辨率图在定位任务中显著优于上采样方法，尽管由于计算成本较高和结构更复杂，其计数精度略低。

ABSTRACT

For crowded scenes, the accuracy of object-based computer vision methods declines when the images are low-resolution and objects have severe occlusions. Taking counting methods for example, almost all the recent state-of-the-art counting methods bypass explicit detection and adopt regression-based methods to directly count the objects of interest. Among regression-based methods, density map estimation, where the number of objects inside a subregion is the integral of the density map over that subregion, is especially promising because it preserves spatial information, which makes it useful for both counting and localization (detection and tracking). With the power of deep convolutional neural networks (CNNs) the counting performance has improved steadily. The goal of this paper is to evaluate density maps generated by density estimation methods on a variety of crowd analysis tasks, including counting, detection, and tracking. Most existing CNN methods produce density maps with resolution that is smaller than the original images, due to the downsample strides in the convolution/pooling operations. To produce an original-resolution density map, we also evaluate a classical CNN that uses a sliding window regressor to predict the density for every pixel in the image. We also consider a fully convolutional (FCNN) adaptation, with skip connections from lower convolutional layers to compensate for loss in spatial information during upsampling. In our experiments, we found that the lower-resolution density maps sometimes have better counting performance. In contrast, the original-resolution density maps improved localization tasks, such as detection and tracking, compared to bilinear upsampling the lower-resolution density maps. Finally, we also propose several metrics for measuring the quality of a density map, and relate them to experiment results on counting and localization.

研究动机与目标

• 评估密度图估计方法在多个人群分析任务（计数、检测和跟踪）中的性能。
• 探究全分辨率密度图是否相比上采样的低分辨率图能提升定位精度。
• 识别并量化支持准确计数和有效检测/跟踪的高质量密度图的特征。
• 提出基于空间紧凑性、定位精度和时间一致性的新指标，用于评估密度图质量。
• 理解不同网络架构和训练策略下计算复杂度与性能之间的权衡。

提出的方法

• 提出一种滑动窗口卷积神经网络（CNN-pixel），为输入图像的每个像素预测密度值，从而生成全分辨率密度图。
• 将CNN-pixel改进为带跳跃连接的全卷积网络（FCNN-skip），通过低层特征的跳跃连接在上采样过程中保留空间细节。
• 采用多任务损失函数，结合像素级回归损失和块级计数损失，以平衡结构保真度与全局准确性。
• 使用空洞卷积和替代网络架构（如DenseNet变体）探索不同归纳偏置下的全分辨率预测。
• 引入新评估指标，如空间紧凑性、定位精确度和时间一致性，用于分析密度图质量。
• 在标准数据集（UCSD、ShanghaiTech）上，通过计数（MAE）、检测（IntProg、GMM-weight）和跟踪（MOT指标）比较不同方法。

实验结果

研究问题

• RQ1 在计数、检测和跟踪任务中，低分辨率密度图与全分辨率密度图的性能表现如何比较？
• RQ2 通过密集像素预测（CNN-pixel）生成的全分辨率密度图是否在定位性能上优于上采样的低分辨率图？
• RQ3 损失函数（像素级、计数级或联合损失）在塑造预测密度图的空间结构和准确性方面起到什么作用？
• RQ4 架构选择（如跳跃连接、空洞卷积或网络深度）如何影响密度图在下游任务中的质量与实用性？
• RQ5 哪些指标最能解释计数精度相近但定位性能不同的密度图之间的性能差异？

主要发现

• 由CNN-pixel生成的全分辨率密度图在检测和跟踪任务中表现最佳，显著优于低分辨率图的双线性上采样结果。
• 低分辨率密度图（如MCNN或无跳跃连接的FCNN）的计数精度更高（MAE: 1.26），优于全分辨率方法（如CNN-pixel: MAE: 1.41），表明分辨率与全局计数精度之间存在权衡。
• 同时使用像素级和计数级损失的FCNN-skip模型实现了最佳平衡，MAE为1.26，且定位质量更优；若移除计数损失，误差上升至MAE: 1.41。
• 仅使用计数损失训练的密度图会使其密度值分布更分散，导致定位性能差且MAE升高至1.82，表明像素级监督对空间结构至关重要。
• 使用透视感知真实标签密度图（CNN-pixel-VS）导致所有任务性能下降，MAE升至1.48，原因是预测结果过度平滑。
• 基于空洞卷积的全分辨率预测性能较差（MAE: 1.93），且速度慢于基于上采样的FCNN，表明使用跳跃连接的上采样方法在全分辨率密度图估计中更有效。