[Paper Review] JointsGait:A model-based Gait Recognition Method based on Gait Graph Convolutional Networks and Joints Relationship Pyramid Mapping

Na Li, Xinbo Zhao+1 more|arXiv (Cornell University)|Apr 27, 2020

Gait Recognition and AnalysisEngineering19 citations

TL;DR

JointsGait proposes a novel model-based gait recognition method using 2D joint coordinates and a gait graph convolutional network (G-CNN) to extract spatio-temporal features, combined with Joints Relationship Pyramid Mapping (JRPM) to model multi-scale joint relationships. It achieves state-of-the-art performance on Kinect Gait and CASIA-B datasets, outperforming both model-based and appearance-based methods despite using only 2D joints and being robust to view and clothing variations.

ABSTRACT

Gait, as one of unique biometric features, has the advantage of being recognized from a long distance away, can be widely used in public security. Considering 3D pose estimation is more challenging than 2D pose estimation in practice , we research on using 2D joints to recognize gait in this paper, and a new model-based gait recognition method JointsGait is put forward to extract gait information from 2D human body joints. Appearance-based gait recognition algorithms are prevalent before. However, appearance features suffer from external factors which can cause drastic appearance variations, e.g. clothing. Unlike previous approaches, JointsGait firstly extracted spatio-temporal features from 2D joints using gait graph convolutional networks, which are less interfered by external factors. Secondly, Joints Relationship Pyramid Mapping (JRPM) are proposed to map spatio-temporal gait features into a discriminative feature space with biological advantages according to the relationship of human joints when people are walking at various scales. Finally, we design a fusion loss strategy to help the joints features to be insensitive to cross-view. Our method is evaluated on two large datasets, Kinect Gait Biometry Dataset and CASIA-B. On Kinect Gait Biometry Dataset database, JointsGait only uses corresponding 2D coordinates of joints, but achieves satisfactory recognition accuracy compared with those model-based algorithms using 3D joints. On CASIA-B database, the proposed method greatly outperforms advanced model-based methods in all walking conditions, even performs superior to state-of-art appearance-based methods when clothing seriously affect people's appearance. The experimental results demonstrate that JointsGait achieves the state-of-art performance despite the low dimensional feature (2D body joints) and is less affected by the view variations and clothing variation.

Motivation & Objective

• To develop a gait recognition method that leverages 2D joint coordinates instead of 3D poses, which are harder to estimate in practice.
• To reduce sensitivity to external factors like clothing and viewpoint variation by modeling gait through joint relationships rather than appearance.
• To improve feature discrimination by mapping spatio-temporal gait features into a hierarchical, biologically plausible representation using JRPM.
• To achieve high recognition accuracy with low-dimensional 2D joint features, avoiding reliance on complex 3D pose estimation.

Proposed method

• A gait graph convolutional network (G-CNN) is used to extract spatio-temporal features from 2D human body joint sequences.
• Joints Relationship Pyramid Mapping (JRPM) constructs a multi-scale representation of joint relationships based on anatomical constraints and walking dynamics.
• The JRPM module maps gait features into a discriminative space that preserves biological plausibility and scale-invariant patterns.
• A fusion loss strategy is introduced to align features across different views, enhancing cross-view generalization.
• The model is trained end-to-end using a combination of classification and contrastive loss to improve feature robustness.
• The method operates solely on 2D joint coordinates, avoiding the need for 3D pose estimation.

Experimental results

Research questions

• RQ1 Can gait recognition be effectively performed using only 2D joint coordinates, without requiring 3D pose estimation?
• RQ2 How can spatio-temporal gait features be modeled to be robust against viewpoint and clothing variations?
• RQ3 Can a hierarchical, pyramid-based representation of joint relationships improve feature discriminability in gait recognition?
• RQ4 To what extent can a 2D joint-based method outperform state-of-the-art appearance-based and model-based gait recognition approaches?
• RQ5 How effective is the fusion loss strategy in improving cross-view generalization of gait features?

Key findings

• On the Kinect Gait Biometry Dataset, JointsGait achieves competitive recognition accuracy using only 2D joint coordinates, outperforming model-based methods that rely on 3D joints.
• On the CASIA-B dataset, JointsGait significantly outperforms advanced model-based gait recognition methods across all walking conditions.
• The method surpasses state-of-the-art appearance-based methods when clothing variations severely affect appearance features.
• The fusion loss strategy effectively reduces sensitivity to viewpoint changes, improving cross-view feature alignment.
• The combination of G-CNN and JRPM enables high discriminative power despite the low-dimensional input of 2D joints.
• The proposed method demonstrates strong robustness to external factors such as clothing and viewing angle, confirming its practical viability for real-world gait recognition.