[论文解读] Bullying10K: A Large-Scale Neuromorphic Dataset towards Privacy-Preserving Bullying Recognition

Yiting Dong, Yang Li+3 more|arXiv (Cornell University)|Jun 20, 2023

Adversarial Robustness in Machine Learning被引用 9

一句话总结

Bullying10K 引入了用于隐私保护的欺凌/暴力行为识别的大规模事件基数据集，提供行动识别、时序行动定位和姿态估计基准，包含 10,000 个事件片段和 120 亿个事件。

ABSTRACT

The prevalence of violence in daily life poses significant threats to individuals' physical and mental well-being. Using surveillance cameras in public spaces has proven effective in proactively deterring and preventing such incidents. However, concerns regarding privacy invasion have emerged due to their widespread deployment. To address the problem, we leverage Dynamic Vision Sensors (DVS) cameras to detect violent incidents and preserve privacy since it captures pixel brightness variations instead of static imagery. We introduce the Bullying10K dataset, encompassing various actions, complex movements, and occlusions from real-life scenarios. It provides three benchmarks for evaluating different tasks: action recognition, temporal action localization, and pose estimation. With 10,000 event segments, totaling 12 billion events and 255 GB of data, Bullying10K contributes significantly by balancing violence detection and personal privacy persevering. And it also poses a challenge to the neuromorphic dataset. It will serve as a valuable resource for training and developing privacy-protecting video systems. The Bullying10K opens new possibilities for innovative approaches in these domains.

研究动机与目标

• 在公共监控场景中激发隐私保护的暴力检测研究。
• 提供一个使用 Dynamic Vision Sensors (DVS) 捕捉的大规模神经形态数据集。
• 使基于事件的数据在行动识别、时序行动定位和姿态估计方面可评估。
• 提供基准以推动现实场景中隐私保护视频系统的发展。

提出的方法

• 使用两台 Davis346 DVS 摄像机收集暴力和友好动作的多视角事件流。
• 使用与 RGB 对齐的姿态估计工具标注数据集的动作类别、摄像机位置、照明和姿态关键点。
• 将原始事件流转换为帧和 10 ms 的时间单位以作为模型输入。
• 在 DVS 数据上评估多种动作识别、时序定位和姿态估计模型。
• 与 RGB 数据相比，比较隐私保护方法在 DVS 数据上的隐私-性能权衡。

Figure 1: Visualization of the Bullying10K dataset. For each example, the right section illustrates the stream of events captured by a Dynamic Visual Sensor (DVS) camera, showcasing the dynamic changes in brightness at each pixel. The left section demonstrates the related event frame transformed fro

实验结果

研究问题

• RQ1 一个大规模基于事件的数据集能否在保持隐私的前提下捕捉复杂、快速且被遮挡的暴力行为？
• RQ2 在神经形态 DVS 数据上，最先进的动作识别、定位和姿态估计方法在暴力检测任务中的表现如何？
• RQ3 在暴力场景识别中，将隐私保护技术应用于 RGB 与 DVS 模态时，隐私含义与性能权衡如何？

主要发现

• Bullying10K 含有 10,000 个事件片段，总计 12 十亿个事件和 255 GB 的数据。
• 数据集目标为 10 种动作（6 种暴力，4 种友好），并提供三项基准：动作识别、时序行动定位和姿态估计。
• 基于 DVS 的动作识别对光照变化和运动模糊具有鲁棒性，在隐私保护设置下常常优于推导自 RGB 的基线方法。
• Bullying10K 的时序行动定位和姿态估计具有相当挑战性，反映数据集的复杂性以及对专业事件驱动模型的需求。
• 分析包括关键点运动、事件极性分布和 IoU 分布，用于刻画事件动态和遮挡情况。

Figure 2: The flow of the data acquisition process. We employed two DVS cameras, positioned on the left and right sides, respectively. Following the recording, the DVS outputs an event stream for pre-processing. This processed data was then employed for three distinct tasks: action recognition, temp