[Paper Review] Gait Recognition via Disentangled Representation Learning

Ziyuan Zhang, Luan Tran+5 more|arXiv (Cornell University)|Apr 9, 2019

Gait Recognition and Analysis42 references19 citations

TL;DR

This paper proposes GaitNet, an autoencoder-based deep learning framework that disentangles pose-based gait features from appearance variations (e.g., clothing, carrying) in RGB videos using novel reconstruction and similarity loss functions. By leveraging multi-layer LSTM to aggregate temporal pose features, GaitNet achieves state-of-the-art performance on CASIA-B, USF, and a new frontal-view gait dataset (FVG), demonstrating superior robustness and computational efficiency over prior methods.

ABSTRACT

Gait, the walking pattern of individuals, is one of the most important biometrics modalities. Most of the existing gait recognition methods take silhouettes or articulated body models as the gait features. These methods suffer from degraded recognition performance when handling confounding variables, such as clothing, carrying and view angle. To remedy this issue, we propose a novel AutoEncoder framework to explicitly disentangle pose and appearance features from RGB imagery and the LSTM-based integration of pose features over time produces the gait feature. In addition, we collect a Frontal-View Gait (FVG) dataset to focus on gait recognition from frontal-view walking, which is a challenging problem since it contains minimal gait cues compared to other views. FVG also includes other important variations, e.g., walking speed, carrying, and clothing. With extensive experiments on CASIA-B, USF and FVG datasets, our method demonstrates superior performance to the state of the arts quantitatively, the ability of feature disentanglement qualitatively, and promising computational efficiency.

Motivation & Objective

• To address the challenge of gait recognition under confounding visual variations such as clothing, carrying, and view angle.
• To overcome limitations of handcrafted features like GEI and skeleton-based methods, which are sensitive to appearance changes or suffer from high computational cost.
• To develop an end-to-end deep learning framework that automatically disentangles gait-relevant pose dynamics from static appearance factors in raw RGB video.
• To collect and release a high-definition, frontal-view gait dataset (FVG) with diverse variations to better evaluate robustness in real-world conditions.
• To demonstrate that disentangled representation learning leads to more invariant and generalizable gait features, improving recognition accuracy and efficiency.

Proposed method

• GaitNet employs a CNN-based autoencoder with two parallel heads: one for pose feature and one for appearance feature, trained via cross-reconstruction loss to ensure disentanglement.
• The cross-reconstruction loss ensures that the appearance of one frame combined with the pose of another can reconstruct the latter frame, enforcing disentanglement.
• A gait similarity loss is introduced to ensure that pose features from the same subject remain similar across different conditions (e.g., clothing, speed), promoting invariance.
• Temporal gait features are learned by feeding the sequence of disentangled pose features into a multi-layer LSTM with an incremental identity loss to enhance discriminative power.
• The final gait representation is obtained via cosine similarity between video-level features, enabling efficient video-to-video matching.
• The method is trained and evaluated on CASIA-B, USF, and a newly collected frontal-view gait dataset (FVG), with ablation studies validating each component.

Experimental results

Research questions

• RQ1 Can an end-to-end deep learning model effectively disentangle gait-related pose dynamics from visual appearance variations in RGB videos without relying on handcrafted features?
• RQ2 How does the proposed disentanglement framework compare to state-of-the-art methods in terms of accuracy and robustness under challenging variations like clothing, carrying, and view angle?
• RQ3 To what extent does the proposed method generalize across different datasets, including a newly collected frontal-view gait dataset with diverse real-world variations?
• RQ4 Does the use of a novel autoencoder with dual loss functions (reconstruction and similarity) lead to more invariant and discriminative gait representations than existing approaches?
• RQ5 What is the computational efficiency of the proposed method compared to pose estimation-based baselines, especially in real-time applications?

Key findings

• On the CASIA-B dataset, GaitNet achieves a TDR@FAR of 91.8% under the CL variation (clothing change), significantly outperforming the next best method (LB network at 61.6%) and demonstrating strong invariance to appearance changes.
• On the USF dataset, GaitNet achieves 99.5±0.2% identification accuracy under probe view variation, surpassing the reported 96.7% of the LB network and 94.7% of multi-task GAN.
• On the newly collected FVG dataset, GaitNet achieves 81.2% TDR@FAR 1% across all variations, outperforming GEI (5.8%), GEINet (13.0%), DCNN (7.9%), and LB (40.7%) under the same protocol.
• GaitNet demonstrates superior computational efficiency, with a total inference time of 1.5 ms per frame on the FVG dataset—faster than pose estimation-based baselines like PE-LSTM (22.5 ms/frame).
• Ablation studies confirm that both the cross-reconstruction and gait similarity losses are essential for disentanglement and performance, with ablation of either leading to significant accuracy drops.
• Qualitative analysis confirms that GaitNet successfully disentangles pose dynamics from appearance, with generated features focusing on gait motion even under extreme variations like carrying or different clothing.