[论文解读] Detecting Violence in Video Based on Deep Features Fusion Technique

Heyam M. Bin Jahlan, Lamiaa A. Elrefaei|arXiv (Cornell University)|Apr 15, 2022

Anomaly Detection Techniques and Applications被引用 7

一句话总结

本文提出了一种基于深度学习的视频暴力检测方法，通过融合两种不同CNN——AlexNet和SqueezeNet——的特征，每种网络后接一个ConvLSTM以提取时空特征。融合后的特征经池化处理后，通过全连接层进行分类，在Hockey Fight、Movie和Violent Flow数据集上分别实现了97%、100%和96%的准确率，优于当前最先进方法。

ABSTRACT

With the rapid growth of surveillance cameras in many public places to mon-itor human activities such as in malls, streets, schools and, prisons, there is a strong demand for such systems to detect violence events automatically. Au-tomatic analysis of video to detect violence is significant for law enforce-ment. Moreover, it helps to avoid any social, economic and environmental damages. Mostly, all systems today require manual human supervisors to de-tect violence scenes in the video which is inefficient and inaccurate. in this work, we interest in physical violence that involved two persons or more. This work proposed a novel method to detect violence using a fusion tech-nique of two significantly different convolutional neural networks (CNNs) which are AlexNet and SqueezeNet networks. Each network followed by separate Convolution Long Short Term memory (ConvLSTM) to extract ro-bust and richer features from a video in the final hidden state. Then, making a fusion of these two obtained states and fed to the max-pooling layer. Final-ly, features were classified using a series of fully connected layers and soft-max classifier. The performance of the proposed method is evaluated using three standard benchmark datasets in terms of detection accuracy: Hockey Fight dataset, Movie dataset and Violent Flow dataset. The results show an accuracy of 97%, 100%, and 96% respectively. A comparison of the results with the state of the art techniques revealed the promising capability of the proposed method in recognizing violent videos.

研究动机与目标

• 为解决监控视频中人工暴力检测效率低、准确率差的问题。
• 开发一种自动化系统，能够检测涉及两人或以上个体的肢体暴力行为。
• 通过融合两种不同卷积神经网络的互补特征，提升检测准确率。
• 在标准基准数据集上评估该方法，以实现对性能的稳健评估。

提出的方法

• 分别使用AlexNet和SqueezeNet作为特征提取器，从视频帧中捕捉多样的视觉模式。
• 在每个CNN之后应用ConvLSTM网络，以建模时间动态并从视频序列中提取鲁棒的时空特征。
• 融合两个ConvLSTM网络的最终隐藏状态，以整合互补的表征。
• 对融合后的特征表示应用最大池化层，以降低维度并增强判别能力。
• 使用一系列全连接层对池化后的特征进行分类，并通过Softmax分类器输出最终的暴力预测结果。

实验结果

研究问题

• RQ1 两种不同CNN架构的融合是否能提升视频序列中暴力检测的准确率？
• RQ2 将ConvLSTM与多个CNN结合，在捕捉暴力行为时空动态方面有多高效？
• RQ3 所提出的特征融合策略是否在标准暴力检测基准上优于现有方法？
• RQ4 该混合深度学习架构在真实世界视频数据集上可达到多高的准确率？

主要发现

• 所提方法在Hockey Fight数据集上实现了97%的检测准确率，表明其在体育类暴力事件中表现强劲。
• 在Movie数据集上，模型达到100%的准确率，表明其在精心筛选的电影暴力片段中具有极佳的泛化能力。
• 系统在Violent Flow数据集上实现了96%的准确率，证实其在多样化视频内容中的鲁棒性。
• AlexNet与SqueezeNet特征结合ConvLSTM的融合策略，在暴力识别任务中优于当前最先进技术。