[论文解读] JointsGait:A model-based Gait Recognition Method based on Gait Graph Convolutional Networks and Joints Relationship Pyramid Mapping

Na Li, Xinbo Zhao+1 more|arXiv (Cornell University)|Apr 27, 2020

Gait Recognition and Analysis被引用 19

一句话总结

JointsGait 提出了一种基于新模型的步态识别方法，利用2D关节坐标和一种步态图卷积网络（G-CNN）来提取时空特征，并结合关节关系金字塔映射（JRPM）以建模多尺度关节关系。该方法在Kinect Gait和CASIA-B数据集上实现了最先进性能，尽管仅使用2D关节坐标，且对视角和衣物变化具有鲁棒性，优于基于模型和基于外观的方法。

ABSTRACT

Gait, as one of unique biometric features, has the advantage of being recognized from a long distance away, can be widely used in public security. Considering 3D pose estimation is more challenging than 2D pose estimation in practice , we research on using 2D joints to recognize gait in this paper, and a new model-based gait recognition method JointsGait is put forward to extract gait information from 2D human body joints. Appearance-based gait recognition algorithms are prevalent before. However, appearance features suffer from external factors which can cause drastic appearance variations, e.g. clothing. Unlike previous approaches, JointsGait firstly extracted spatio-temporal features from 2D joints using gait graph convolutional networks, which are less interfered by external factors. Secondly, Joints Relationship Pyramid Mapping (JRPM) are proposed to map spatio-temporal gait features into a discriminative feature space with biological advantages according to the relationship of human joints when people are walking at various scales. Finally, we design a fusion loss strategy to help the joints features to be insensitive to cross-view. Our method is evaluated on two large datasets, Kinect Gait Biometry Dataset and CASIA-B. On Kinect Gait Biometry Dataset database, JointsGait only uses corresponding 2D coordinates of joints, but achieves satisfactory recognition accuracy compared with those model-based algorithms using 3D joints. On CASIA-B database, the proposed method greatly outperforms advanced model-based methods in all walking conditions, even performs superior to state-of-art appearance-based methods when clothing seriously affect people's appearance. The experimental results demonstrate that JointsGait achieves the state-of-art performance despite the low dimensional feature (2D body joints) and is less affected by the view variations and clothing variation.

研究动机与目标

• 开发一种利用2D关节坐标而非更难估计的3D姿态的步态识别方法。
• 通过基于关节关系而非外观来建模步态，以降低对外部因素（如衣物和视角变化）的敏感性。
• 通过JRPM将时空步态特征映射到分层的、符合生物学原理的表示，以提升特征判别能力。
• 仅使用低维2D关节特征实现高识别准确率，避免依赖复杂的3D姿态估计。

提出的方法

• 使用步态图卷积网络（G-CNN）从2D人体关节序列中提取时空特征。
• 关节关系金字塔映射（JRPM）基于解剖学约束和行走动力学，构建关节关系的多尺度表示。
• JRPM模块将步态特征映射到保留生物学合理性和尺度不变模式的判别性空间。
• 引入融合损失策略以对齐不同视角下的特征，提升跨视角泛化能力。
• 通过分类损失与对比损失的组合，端到端训练模型以提升特征鲁棒性。
• 该方法仅基于2D关节坐标运行，无需3D姿态估计。

实验结果

研究问题

• RQ1 是否可以仅使用2D关节坐标而无需3D姿态估计来有效实现步态识别？
• RQ2 如何建模时空步态特征以使其对视角和衣物变化具有鲁棒性？
• RQ3 基于分层金字塔的关节关系表示能否提升步态识别中的特征判别能力？
• RQ4 基于2D关节的方法在多大程度上能超越最先进的基于外观和基于模型的步态识别方法？
• RQ5 融合损失策略在提升步态特征跨视角泛化能力方面有多有效？

主要发现

• 在Kinect Gait Biometry数据集上，JointsGait仅使用2D关节坐标即实现了具有竞争力的识别准确率，优于依赖3D关节的基于模型的方法。
• 在CASIA-B数据集上，JointsGait在所有行走条件下显著优于先进的基于模型的步态识别方法。
• 当衣物变化严重影响外观特征时，该方法超越了最先进的基于外观的方法。
• 融合损失策略有效降低了对视角变化的敏感性，改善了跨视角特征对齐。
• G-CNN与JRPM的结合在仅使用低维2D关节输入的情况下，仍实现了高判别能力。
• 所提方法对衣物和视角等外部因素表现出强鲁棒性，证实了其在真实世界步态识别中的实际可行性。