[论文解读] A Novel Approach for Robust Multi Human Action Detection and Recognition based on 3-Dimentional Convolutional Neural Networks.

Noor Almaadeed, Omar Elharrouss+3 more|arXiv (Cornell University)|Jul 25, 2019

Human Pose and Action Recognition参考文献 57被引用 11

一句话总结

本文提出一种3D卷积神经网络（3D-CNN）架构，通过针对个体的动作序列分析来建模时空特征，实现对监控视频中多个人体动作的鲁棒识别。该方法在自建的多人群体数据集上达到98%的准确率，并在UCF101、Hollywood2、HDMB51和YouTube数据集上无需预处理即超越了当前最先进方法的性能。

ABSTRACT

In recent years, various attempts have been proposed to explore the use of spatial and temporal information for human action recognition using convolutional neural networks (CNNs). However, only a small number of methods are available for the recognition of many human actions performed by more than one person in the same surveillance video. This paper proposes a novel technique for multiple human action recognition using a new architecture based on 3Dimdenisional deep learning with application to video surveillance systems. The first stage of the model uses a new representation of the data by extracting the sequence of each person acting in the scene. An analysis of each sequence to detect the corresponding actions is also proposed. KTH, Weizmann and UCF-ARG datasets were used for training, new datasets were also constructed which include a number of persons having multiple actions were used for testing the proposed algorithm. The results of this work revealed that the proposed method provides more accurate multi human action recognition achieving 98%. Other videos were used for the evaluation including datasets (UCF101, Hollywood2, HDMB51, and YouTube) without any preprocessing and the results obtained suggest that our proposed method clearly improves the performances when compared to state-of-the-art methods.

研究动机与目标

• 解决在多人同时执行动作的拥挤监控视频中识别多个体动作的挑战。
• 克服现有基于CNN的方法主要聚焦于单人或有限多人群体动作识别的局限性。
• 开发一种鲁棒的深度学习框架，有效捕捉复杂场景中多个动作参与者之间的时空特征。
• 在标准基准数据集和新构建的多人群体数据集上评估该方法，以确保其泛化能力与实际应用价值。

提出的方法

• 通过从视频场景中为每个人提取个体动作序列，构建一种新颖的数据表示方式，以隔离多人群体交互行为。
• 应用3D-CNN架构，从提取的个体特定动作序列中学习空间与时间特征。
• 采用两阶段模型：第一阶段为人物动作序列提取；第二阶段通过3D-CNN进行分类识别动作。
• 在KTH、Weizmann和UCF-ARG数据集上进行模型训练，并在新构建的多人群体动作识别数据集上进行验证。
• 在包括UCF101、Hollywood2、HDMB51和YouTube在内的多样化数据集上评估模型，且评估过程中不进行任何数据预处理，以检验其鲁棒性。

实验结果

研究问题

• RQ1 基于3D-CNN的架构是否能有效识别复杂、拥挤的监控视频中的多个人体动作？
• RQ2 与标准视频级建模相比，所提出的基于人物序列的数据表示在多人群体动作识别准确率方面有何提升？
• RQ3 该方法在无需数据预处理的情况下，跨多样化数据集的泛化能力如何？
• RQ4 与现有最先进方法相比，该方法在多人群体动作识别任务中实现了多大的性能提升？

主要发现

• 所提出的方法在新构建的多人群体动作识别数据集上达到98%的准确率，表明其在复杂场景中具有高度鲁棒性。
• 与当前最先进方法相比，该模型在UCF101、Hollywood2、HDMB51和YouTube数据集上的性能显著提升。
• 在YouTube及其他数据集上评估时无需数据预处理，凸显了模型强大的泛化能力与鲁棒性。
• 采用个体特定的动作序列表示可增强特征学习，提升多人群体场景下的识别准确率。