[Paper Review] Progressive Feature Learning for Realistic Cloth-Changing Gait Recognition

Xuqian Ren, Saihui Hou+3 more|arXiv (Cornell University)|Jul 24, 2022

Gait Recognition and Analysis4 citations

TL;DR

This paper introduces a novel Realistic Cloth-Changing Gait Recognition (RCC-GR) task and proposes a Progressive Feature Learning framework to address cross-view and cross-cloth gait recognition with non-overlapping sub-datasets. The method uses Progressive Mapping and Progressive Uncertainty to sequentially learn cross-view and cross-cloth features, improving rank-1 accuracy by up to 48.5% on CASIA-BN-RCC and 55.6% on OUMVLP-RCC under the CL condition.

ABSTRACT

Gait recognition is instrumental in crime prevention and social security, for it can be conducted at a long distance to figure out the identity of persons. However, existing datasets and methods cannot satisfactorily deal with the most challenging cloth-changing problem in practice. Specifically, the practical gait models are usually trained on automatically labeled data, in which the sequences' views and cloth conditions of each person have some restrictions. To be concrete, the cross-view sub-dataset only has normal walking condition without cloth-changing, while the cross-cloth sub-dataset has cloth-changing sequences but only in front views. As a result, the cloth-changing accuracy cannot meet practical requirements. In this work, we formulate the problem as Realistic Cloth-Changing Gait Recognition (abbreviated as RCC-GR) and we construct two benchmarks: CASIA-BN-RCC and OUMVLP-RCC, to simulate the above setting. Furthermore, we propose a new framework called Progressive Feature Learning that can be applied with off-the-shelf backbones to improve their performance in RCC-GR. Specifically, in our framework, we design Progressive Mapping and Progressive Uncertainty to extract cross-view features and then extract cross-cloth features on the basis. In this way, the feature from the cross-view sub-dataset can first dominate the feature space and relieve the uneven distribution caused by the adverse effect from the cross-cloth sub-dataset. The experiments on our benchmarks show that our framework can effectively improve recognition performance, especially in the cloth-changing conditions.

Motivation & Objective

• To address the realistic challenge of gait recognition when individuals wear different clothes across views, a common but underexplored problem in real-world surveillance.
• To construct two new benchmarks, CASIA-BN-RCC and OUMVLP-RCC, that simulate non-overlapping cross-view and cross-cloth data distributions found in practice.
• To develop a framework that enables off-the-shelf gait recognition backbones to generalize effectively under realistic cloth-changing conditions.
• To mitigate feature space distortion caused by uneven intra-class diversity between cross-view and cross-cloth sub-datasets.

Proposed method

• Proposes a two-stage Progressive Mapping module: Cross-View Mapping (CVM) followed by Cross-Cloth Mapping (CCM), which first expands the feature space using cross-view data before learning cross-cloth representations.
• Introduces Progressive Uncertainty, modeling each gait sequence as a Gaussian distribution (mean μ and variance σ²) to stabilize training and improve generalization.
• Employs Uncertainty-aware Triplet Loss (ℒtv, ℒtc) that assigns higher gradient weight to easier cross-view samples and reduces overfitting on hard cross-cloth samples.
• Uses re-sampling from Gaussian distributions to generate additional embeddings, enhancing feature robustness and discriminability.
• Applies the framework with off-the-shelf backbones like GaitSet, enabling plug-and-play improvement without architectural overhaul.
• Uses separate triplet losses ℒnv and ℒnc for cross-view and cross-cloth sub-datasets, respectively, to avoid negative interference during optimization.

Experimental results

Research questions

• RQ1 Can a two-stage feature learning strategy effectively improve gait recognition performance under realistic cloth-changing conditions with non-overlapping data?
• RQ2 How does modeling gait sequences as Gaussian distributions instead of points affect model generalization in cross-cloth scenarios?
• RQ3 To what extent does progressive learning mitigate feature space distortion caused by imbalanced intra-class diversity between cross-view and cross-cloth sub-datasets?
• RQ4 Does the proposed framework outperform standard one-stage feature learning under the same backbone and training protocol?

Key findings

• The Progressive Mapping strategy improved CL accuracy to 48.5% on CASIA-BN-RCC and 55.6% on OUMVLP-RCC, outperforming single-stage baselines.
• Progressive Uncertainty with re-sampling increased CL accuracy to 51.6% on CASIA-BN-RCC and 57.3% on OUMVLP-RCC, demonstrating the benefit of uncertainty-aware training.
• Using only cross-cloth variance (σc) with CCM alone yielded 50.7% CL accuracy, but combining CVM and CCM improved it to 50.3%, showing the importance of full uncertainty modeling.
• The ablation study confirmed that performance gains stem from the two-stage module and triplet design, not increased parameters.
• The framework achieved 90.2% rank-1 accuracy on NM and 83.1% on BG in CASIA-BN-RCC, showing strong generalization across all sub-datasets.
• The proposed benchmarks CASIA-BN-RCC and OUMVLP-RCC effectively simulate real-world cloth-changing gait recognition, enabling fair and realistic evaluation.