[Paper Review] GaitSet: Regarding Gait as a Set for Cross-View Gait Recognition

Hanqing Chao, Yiwei He+2 more|arXiv (Cornell University)|Nov 15, 2018

Gait Recognition and AnalysisEngineering26 references59 citations

TL;DR

GaitSet treats gait as a set of silhouettes processed by a permutation-invariant Set Pooling and Horizontal Pyramid Mapping to achieve state-of-the-art cross-view gait recognition, robust to view, clothing, and carrying conditions.

ABSTRACT

As a unique biometric feature that can be recognized at a distance, gait has broad applications in crime prevention, forensic identification and social security. To portray a gait, existing gait recognition methods utilize either a gait template, where temporal information is hard to preserve, or a gait sequence, which must keep unnecessary sequential constraints and thus loses the flexibility of gait recognition. In this paper we present a novel perspective, where a gait is regarded as a set consisting of independent frames. We propose a new network named GaitSet to learn identity information from the set. Based on the set perspective, our method is immune to permutation of frames, and can naturally integrate frames from different videos which have been filmed under different scenarios, such as diverse viewing angles, different clothes/carrying conditions. Experiments show that under normal walking conditions, our single-model method achieves an average rank-1 accuracy of 95.0% on the CASIA-B gait dataset and an 87.1% accuracy on the OU-MVLP gait dataset. These results represent new state-of-the-art recognition accuracy. On various complex scenarios, our model exhibits a significant level of robustness. It achieves accuracies of 87.2% and 70.4% on CASIA-B under bag-carrying and coat-wearing walking conditions, respectively. These outperform the existing best methods by a large margin. The method presented can also achieve a satisfactory accuracy with a small number of frames in a test sample, e.g., 82.5% on CASIA-B with only 7 frames. The source code has been released at https://github.com/AbnerHqC/GaitSet.

Motivation & Objective

• Motivate gait recognition that is robust to view and condition variations without relying on sequential constraints or single templates.
• Propose a permutation-invariant set-based framework to learn from sets of silhouettes.
• Develop mechanisms to preserve temporal/spatial information via high-level feature aggregation.
• Demonstrate robustness and scalability across large datasets and varied walking conditions.

Proposed method

• Represent a gait as a set of silhouettes rather than a sequence or a single template.
• Use a CNN to extract frame-level features from each silhouette independently.
• Apply Set Pooling to aggregate frame-level features into a set-level representation in a permutation-invariant way.
• Incorporate attention-enhanced pooling and multiple statistical aggregations (max/mean/median) to form robust set features.
• Employ Horizontal Pyramid Mapping (HPM) with multi-scale strip pooling to map set features into a discriminative space.
• Optionally fuse features from multiple convolutional layers via a Multilayer Global Pipeline (MGP) for multi-level information.

Experimental results

Research questions

• RQ1 Can gait be effectively recognized from unordered sets of silhouettes rather than templates or sequences?
• RQ2 How does permutation-invariant Set Pooling affect recognition accuracy under cross-view and cross-condition scenarios?
• RQ3 What is the impact of multi-scale horizontal pyramid mapping and multi-layer information fusion on discriminability?
• RQ4 How does the approach scale to large datasets and varied viewing conditions?
• RQ5 Can the model maintain high accuracy with limited silhouettes or when combining different views/conditions?

Key findings

• GaitSet achieves high rank-1 accuracy on CASIA-B (95.0% average under normal walking) and OU-MVLP (87.1%) in standard settings, surpassing prior methods.
• Under bag-carrying and coat-wearing conditions on CASIA-B, GaitSet obtains 87.2% and 70.4% respectively, outperforming existing methods.
• GaitSet reaches 82.5% accuracy with only 7 frames on CASIA-B, showing robustness to limited input.
• Ablation shows set-based input substantially outperforms GEI templates, with improvements up to 10%+ on NM and 25%+ on CL subsets.
• Multi-view inputs (two views) generally boost accuracy, demonstrating the model’s ability to fuse information across views.
• The method scales efficiently, e.g., evaluating 133,780 sequences for OU-MVLP in about 7 minutes on 8 GPUs.