[论文解读] A model-based Gait Recognition Method based on Gait Graph Convolutional Networks and Joints Relationship Pyramid Mapping.

Na Li, Xinbo Zhao+1 more|arXiv (Cornell University)|Apr 27, 2020

Gait Recognition and Analysis参考文献 36被引用 16

一句话总结

该论文提出了一种基于模型的步态识别新方法 JointsGait，该方法利用2D关节坐标通过图卷积网络提取时空特征，并利用关节关系金字塔映射（Joints Relationship Pyramid Mapping）将特征映射到判别性空间。该方法在Kinect Gait Biometry和CASIA-B数据集上均取得了最先进性能，优于基于模型和基于外观的方法，尤其在视角和着装变化下表现更优。

ABSTRACT

Gait, as one of unique biometric features, has the advantage of being recognized from a long distance away, can be widely used in public security. Considering 3D pose estimation is more challenging than 2D pose estimation in practice , we research on using 2D joints to recognize gait in this paper, and a new model-based gait recognition method JointsGait is put forward to extract gait information from 2D human body joints. Appearance-based gait recognition algorithms are prevalent before. However, appearance features suffer from external factors which can cause drastic appearance variations, e.g. clothing. Unlike previous approaches, JointsGait firstly extracted spatio-temporal features from 2D joints using gait graph convolutional networks, which are less interfered by external factors. Secondly, Joints Relationship Pyramid Mapping (JRPM) are proposed to map spatio-temporal gait features into a discriminative feature space with biological advantages according to the relationship of human joints when people are walking at various scales. Finally, we design a fusion loss strategy to help the joints features to be insensitive to cross-view. Our method is evaluated on two large datasets, Kinect Gait Biometry Dataset and CASIA-B. On Kinect Gait Biometry Dataset database, JointsGait only uses corresponding 2D coordinates of joints, but achieves satisfactory recognition accuracy compared with those model-based algorithms using 3D joints. On CASIA-B database, the proposed method greatly outperforms advanced model-based methods in all walking conditions, even performs superior to state-of-art appearance-based methods when clothing seriously affect people's appearance. The experimental results demonstrate that JointsGait achieves the state-of-art performance despite the low dimensional feature (2D body joints) and is less affected by the view variations and clothing variation.

研究动机与目标

• 开发一种基于2D关节坐标而非更难估计的3D姿态的步态识别方法。
• 通过利用人体关节的结构关系，减少外部因素（如着装和视角变化）的干扰。
• 通过将时空步态特征映射到分层的、受生物学启发的特征空间，提升特征的判别能力。
• 通过一种新颖的融合损失策略，最小化关节特征中的视角相关差异，提升跨视角鲁棒性。

提出的方法

• 通过步态图卷积网络（G-CNN）从2D关节坐标中提取时空步态特征，将人体关节建模为随时间演化的图结构。
• 关节关系金字塔映射（JRPM）基于解剖学上的邻近性和行走动力学，构建关节关系的多尺度表示，增强特征的判别能力。
• JRPM模块将步态特征映射到反映生物关节连接性和不同行走尺度下运动模式的分层特征空间。
• 设计了一种融合损失策略，用于对齐不同摄像头视角下的特征，减少域偏移，提升跨视角条件下的泛化能力。
• 采用分类损失与所提出的融合损失相结合的方式，端到端训练模型，以同时优化准确率和视角不变性。

实验结果

研究问题

• RQ1 仅基于2D关节坐标的步态识别模型能否在性能上超越基于3D姿态的方法，达到最先进水平？
• RQ2 所提出的关节关系金字塔映射（JRPM）在不同行走条件下增强特征判别能力的效果如何？
• RQ3 融合损失策略在提升步态识别对跨视角变化的鲁棒性方面，其改善程度如何？
• RQ4 与基于外观的方法相比，JointsGait在严重外观变化（如着装变化）下的表现如何？

主要发现

• 在Kinect Gait Biometry数据集上，JointsGait仅使用2D关节坐标即实现了具有竞争力的识别准确率，优于多种基于3D姿态的模型化方法。
• 在CASIA-B数据集上，JointsGait在所有行走条件下均显著优于先进的模型化步态识别方法。
• 即使在严重的着装变化下，JointsGait在识别准确率方面仍优于最先进的基于外观的步态识别方法。
• 所提出的融合损失策略有效降低了视角差异的影响，使特征更具鲁棒性和视角不变性。
• G-CNN与JRPM的结合使得在仅使用低维2D关节特征的情况下，仍能实现高判别性能。