[论文解读] RGait-NET: An Effective Network for Recovering Missing Information from Occluded Gait Cycles

Dhritimaan Das, Ayush Agarwal+2 more|arXiv (Cornell University)|Dec 14, 2019

Gait Recognition and Analysis参考文献 38被引用 7

一句话总结

该论文提出RGait-NET，一种基于VGG-16主干网络检测行人步态序列中遮挡帧，并利用多目标LSTM网络通过dice损失和交叉熵损失联合训练实现缺失轮廓重建的深度学习框架。该方法在具有挑战性的TUM-IITKGP数据集上实现了94.28%的Rank-1识别准确率，相较于先前方法在严重遮挡条件下显著提升了超过25%的准确率。

ABSTRACT

Gait of a person refers to his/her walking pattern, and according to medical studies gait of every individual is unique. Over the past decade, several computer vision-based gait recognition approaches have been proposed in which walking information corresponding to a complete gait cycle has been used to construct gait features for person identification. These methods compute gait features with the inherent assumption that a complete gait cycle is always available. However, in most public places occlusion is an inevitable occurrence, and due to this, only a fraction of a gait cycle gets captured by the monitoring camera. Unavailability of complete gait cycle information drastically affects the accuracy of the extracted features, and till date, only a few occlusion handling strategies to gait recognition have been proposed. But none of these performs reliably and robustly in the presence of a single cycle with incomplete information, and because of this practical application of gait recognition is quite limited. In this work, we develop deep learning-based algorithm to accurately identify the affected frames as well as predict the missing frames to reconstruct a complete gait cycle. While occlusion detection has been carried out by employing a VGG-16 model, the model for frame reconstruction is based on Long-Short Term Memory network that has been trained to optimize a multi-objective function based on dice coefficient and cross-entropy loss. The effectiveness of the proposed occlusion reconstruction algorithm is evaluated by computing the accuracy of the popular Gait Energy Feature on the reconstructed sequence. Experimental evaluation on public data sets and comparative analysis with other occlusion handling methods verify the effectiveness of our approach.

研究动机与目标

• 解决监控场景中因部分或完全遮挡导致完整步态周期难以获取的严峻挑战，此时完整步态周期极为罕见。
• 克服现有方法依赖不完整数据或假设步态模式具有特定分布的局限性。
• 开发一种鲁棒的端到端深度学习流水线，用于检测和重建单周期步态序列中的缺失帧。
• 实现在频繁发生遮挡的真实世界环境中部署步态识别系统的可行性。
• 通过重建高质量步态周期，提升遮挡序列的识别准确率，且无需依赖外部周期匹配或平均化方法。

提出的方法

• 利用预训练的VGG-16网络，基于轮廓质量与空间一致性，将帧分类为遮挡或非遮挡。
• 训练长短期记忆（LSTM）网络，通过建模步态为时间序列来重建缺失轮廓。
• 采用结合dice系数与交叉熵的多目标损失函数优化LSTM，以平衡分割准确率与边界精度。
• 使用自定义Python脚本在CASIA-B数据集中合成遮挡，生成多样化的检测与重建任务训练数据。
• 从重建序列中提取步态能量图像（GEI）特征，用于在公开基准上评估识别性能。
• 在包含真实世界静态与动态遮挡的TUM-IITKGP数据集上训练并验证模型，以评估其鲁棒性与泛化能力。

实验结果

研究问题

• RQ1 基于深度学习的方法能否有效检测单个步态周期序列中的静态与动态遮挡？
• RQ2 基于LSTM的生成模型能否仅以未遮挡帧为输入，高保真地重建缺失的步态帧？
• RQ3 所提出的多目标损失函数是否相比标准交叉熵或仅dice损失训练，能提升重建质量？
• RQ4 在识别准确率方面，该方法与最先进遮挡处理技术相比表现如何？
• RQ5 该模型对不同的随机权重初始化以及不同遮挡程度是否具有鲁棒性？

主要发现

• RGait-NET在TUM-IITKGP数据集上实现了94.28%的Rank-1识别准确率，相较于先前方法在严重遮挡条件下提升超过25%。
• 该模型表现出高度鲁棒性，四组独立训练模型的准确率标准差仅为3.50%，表明在不同初始化下具有稳定性。
• 在所有排名级别下，识别准确率均保持高水平（Rank-1为94.28%，Rank-5为97.68%），显示出在不同检索层级下的强大泛化能力。
• 结合dice系数与交叉熵的多目标损失函数相比单一损失函数方案，能实现更优的重建质量。
• 该方法能有效处理静态与动态遮挡，包括在高遮挡水平下，传统高斯拟合或平均化方法失效的场景。
• 用于遮挡检测与重建的训练模型、代码及数据已通过所提供链接公开，便于复现与基准测试。