[Paper Review] GaitGL: Learning Discriminative Global-Local Feature Representations for Gait Recognition

Beibei Lin, Shunli Zhang+3 more|arXiv (Cornell University)|Aug 2, 2022

Gait Recognition and AnalysisEngineering21 citations

TL;DR

GaitGL presents a global-local gait recognition network with a novel Global and Local Convolutional Layer (GLCL) and mask-based local feature extraction, achieving state-of-the-art results on multiple datasets.

ABSTRACT

Existing gait recognition methods either directly establish Global Feature Representation (GFR) from original gait sequences or generate Local Feature Representation (LFR) from several local parts. However, GFR tends to neglect local details of human postures as the receptive fields become larger in the deeper network layers. Although LFR allows the network to focus on the detailed posture information of each local region, it neglects the relations among different local parts and thus only exploits limited local information of several specific regions. To solve these issues, we propose a global-local based gait recognition network, named GaitGL, to generate more discriminative feature representations. To be specific, a novel Global and Local Convolutional Layer (GLCL) is developed to take full advantage of both global visual information and local region details in each layer. GLCL is a dual-branch structure that consists of a GFR extractor and a mask-based LFR extractor. GFR extractor aims to extract contextual information, e.g., the relationship among various body parts, and the mask-based LFR extractor is presented to exploit the detailed posture changes of local regions. In addition, we introduce a novel mask-based strategy to improve the local feature extraction capability. Specifically, we design pairs of complementary masks to randomly occlude feature maps, and then train our mask-based LFR extractor on various occluded feature maps. In this manner, the LFR extractor will learn to fully exploit local information. Extensive experiments demonstrate that GaitGL achieves better performance than state-of-the-art gait recognition methods. The average rank-1 accuracy on CASIA-B, OU-MVLP, GREW and Gait3D is 93.6%, 98.7%, 68.0% and 63.8%, respectively, significantly outperforming the competing methods. The proposed method has won the first prize in two competitions: HID 2020 and HID 2021.

Motivation & Objective

• Motivate robust gait recognition under diverse conditions where global or local features alone are insufficient.
• Develop a framework that jointly leverages global context and detailed local posture information.
• Introduce a mask-based local feature extractor to enhance local detail exploitation.
• Incorporate temporal information handling to preserve spatial detail via Local Temporal Aggregation (LTA).
• Demonstrate state-of-the-art performance across four popular gait datasets.

Proposed method

• Propose Global and Local Convolutional Layer (GLCL) with two parallel extractors: a Global Feature Representation (GFR) extractor and a mask-based Local Feature Representation (LFR) extractor.
• Introduce mask-based partition strategies (part-level, strip-level, pixel-level) that occlude feature maps using complementary masks to train diverse local representations.
• Replace a conventional spatial pooling layer in early stages with Local Temporal Aggregation (LTA) to reduce temporal redundancy while preserving spatial detail.
• Use a gait recognition head consisting of temporal and spatial feature mapping, with a GeM pooling option for adaptive vertical information aggregation.
• Train with a combined loss of triplet loss and cross-entropy loss, employing a BA (Batch ALL) sampling strategy.

Experimental results

Research questions

• RQ1 How can global contextual relations and local detailed postures be jointly modeled for gait recognition?
• RQ2 Do mask-based local feature extractors improve robustness to intra-class variations and occlusions compared to fixed partition schemes?
• RQ3 Does incorporating Local Temporal Aggregation (LTA) improve performance by managing temporal vs. spatial information trade-offs?
• RQ4 What is the impact of different mask strategies (part/strip/pixel level) on local feature learning?
• RQ5 Can GaitGL achieve state-of-the-art results across multiple standard gait benchmarks?

Key findings

• GaitGL outperforms state-of-the-art gait recognition methods on CASIA-B, OU-MVLP, GREW, and Gait3D datasets (average rank-1: 93.6%, 98.7%, 68.0%, 63.8% respectively).
• The proposed mask-based LFR extractor with complementary masks enhances learning of local posture details.
• Local Temporal Aggregation (LTA) preserves spatial resolution while reducing temporal redundancy, improving recognition accuracy.
• GaitGL achieves leading performance in HID 2020 and HID 2021 competitions.
• The source code for GaitGL is publicly available at the authors’ GitHub repository.