[Paper Review] Pose Invariant Person Re-Identification using Robust Pose-transformation GAN

Arnab Karmakar, Deepak Mishra|arXiv (Cornell University)|Apr 11, 2021

Video Surveillance and Tracking Methods4 citations

TL;DR

This paper proposes a pose-invariant person re-identification framework using a robust pose-transformation GAN (pt-GAN) that generates a person’s image in diverse poses, combined with pose clustering and feature fusion to learn viewpoint-invariant features. The method achieves state-of-the-art performance on four benchmark datasets, outperforming existing GAN-based and non-GAN models in accuracy and improvement over baseline.

ABSTRACT

The objective of person re-identification (re-ID) is to retrieve a person's images from an image gallery, given a single instance of the person of interest. Despite several advancements, learning discriminative identity-sensitive and viewpoint invariant features for robust Person Re-identification is a major challenge owing to the large pose variation of humans. This paper proposes a re-ID pipeline that utilizes the image generation capability of Generative Adversarial Networks combined with pose clustering and feature fusion to achieve pose invariant feature learning. The objective is to model a given person under different viewpoints and large pose changes and extract the most discriminative features from all the appearances. The pose transformational GAN (pt-GAN) module is trained to generate a person's image in any given pose. In order to identify the most significant poses for discriminative feature extraction, a Pose Clustering module is proposed. The given instance of the person is modelled in varying poses and these features are effectively combined through the Feature Fusion Network. The final re-ID model consisting of these 3 sub-blocks, alleviates the pose dependence in person re-ID. Also, The proposed model is robust to occlusion, scale, rotation and illumination, providing a framework for viewpoint invariant feature learning. The proposed method outperforms the state-of-the-art GAN based models in 4 benchmark datasets. It also surpasses the state-of-the-art models that report higher re-ID accuracy in terms of improvement over baseline.

Motivation & Objective

• Address the major challenge of large pose variations in person re-identification, which degrade feature discriminability and model accuracy.
• Overcome limitations of deep learning models in small datasets by leveraging GAN-based data augmentation through pose transformation.
• Develop a robust, end-to-end trainable pipeline that learns viewpoint-invariant features from a single query image.
• Improve generalization under real-world conditions such as occlusion, illumination changes, scale variations, and rotation.
• Achieve superior performance compared to state-of-the-art models by combining pose generation, clustering, and feature fusion.

Proposed method

• Train a pose-transformation GAN (pt-GAN) to generate a person’s image in any desired pose from a single input image, using a refined loss function and fine-tuning on re-ID datasets.
• Implement an unsupervised pose clustering module to identify the most discriminative and representative poses for feature extraction, reducing redundancy.
• Extract deep features from the original query image and all generated pose-variant images using a shared backbone network.
• Fuse the extracted features using a learnable Feature Fusion Network (FusionNet) to produce a single, robust, viewpoint-invariant feature vector.
• Use the fused feature vector for similarity-based retrieval in the gallery set, enabling pose-invariant matching.
• Integrate data augmentation and CNN-based feature learning within the GAN framework to enhance robustness to occlusion, scale, and illumination variations.

Experimental results

Research questions

• RQ1 Can GAN-based pose transformation effectively generate diverse, realistic poses from a single person image to improve feature robustness in person re-identification?
• RQ2 How can unsupervised pose clustering identify the most discriminative poses for feature extraction, minimizing redundancy and maximizing discriminability?
• RQ3 To what extent does feature fusion across multiple pose-variant images improve viewpoint invariance and re-ID accuracy?
• RQ4 Can the proposed end-to-end framework outperform state-of-the-art models in terms of both absolute accuracy and improvement over baseline models?
• RQ5 How robust is the model to real-world challenges such as occlusion, illumination changes, scale variations, and detection errors?

Key findings

• The proposed method achieves a Rank-1 accuracy of 73.2% on Market-1501, 74.48% on DukeMTMC-reID, and 83.3% on CUHK03, outperforming state-of-the-art GAN-based models.
• With re-ranking, the model improves to 73.95% on Market-1501, 93.04% on DukeMTMC-reID, and 96.97% on CUHK03, demonstrating strong retrieval performance.
• The model achieves a 9.64% improvement over baseline ResNet50-1 on Market-1501 and a 12.80% improvement on DukeMTMC-reID, exceeding higher-accuracy models in relative improvement.
• The pt-GAN model successfully reconstructs occluded regions and maintains identity consistency across poses, showing strong robustness to occlusion and detection errors.
• Intra-class feature distances are significantly lower than inter-class distances under pose, illumination, and rotation variations, confirming the model's invariance properties.
• Qualitative results show that the proposed model retrieves correct identities more accurately than baseline models, especially in challenging pose and occlusion scenarios.