[Paper Review] Pose-guided Inter- and Intra-part Relational Transformer for Occluded Person Re-Identification

Zhongxing Ma, Yifan Zhao+1 more|arXiv (Cornell University)|Sep 8, 2021

Video Surveillance and Tracking Methods52 references10 citations

TL;DR

This paper proposes a Pose-guided Inter- and Intra-part Relational Transformer (PIRT) for occluded person re-identification, leveraging keypoint-guided feature grouping and dual transformer modules to model both local part relationships and cross-part correlations. The method achieves state-of-the-art performance on the Occluded-DukeMTMC dataset with 50.90 mAP and 60.00% Rank-1 accuracy.

ABSTRACT

Person Re-Identification (Re-Id) in occlusion scenarios is a challenging problem because a pedestrian can be partially occluded. The use of local information for feature extraction and matching is still necessary. Therefore, we propose a Pose-guided inter-and intra-part relational transformer (Pirt) for occluded person Re-Id, which builds part-aware long-term correlations by introducing transformers. In our framework, we firstly develop a pose-guided feature extraction module with regional grouping and mask construction for robust feature representations. The positions of a pedestrian in the image under surveillance scenarios are relatively fixed, hence we propose an intra-part and inter-part relational transformer. The intra-part module creates local relations with mask-guided features, while the inter-part relationship builds correlations with transformers, to develop cross relationships between part nodes. With the collaborative learning inter- and intra-part relationships, experiments reveal that our proposed Pirt model achieves a new state of the art on the public occluded dataset, and further extensions on standard non-occluded person Re-Id datasets also reveal our comparable performances.

Motivation & Objective

• Address the challenge of person re-identification under severe occlusion, where global features are unreliable due to missing or corrupted body parts.
• Overcome limitations in existing methods that rely on weakly localized or misaligned local features due to inaccurate keypoint detection or lack of structural modeling.
• Improve feature representation by fusing pose-guided local features with long-range dependencies through a dual-branch transformer architecture.
• Enhance robustness to occlusion by modeling both intra-part (local) and inter-part (cross-part) relationships using attention mechanisms.
• Achieve state-of-the-art performance on occluded Re-ID benchmarks while generalizing well to standard Re-ID datasets.

Proposed method

• Introduce a pose-guided feature extraction module that uses a pretrained keypoint detector to localize body parts and expand keypoint regions into larger masks for robust feature learning.
• Group keypoint regions into three semantic part groups (e.g., head, torso, legs) to form stable, context-aware local features.
• Design an intra-part relational transformer that learns local correlations within each grouped part using mask-guided features and self-attention.
• Develop an inter-part relational transformer that models cross-part relationships between the grouped parts using cross-attention across part nodes.
• Integrate confidence scores from query and gallery images to refine matching decisions, improving robustness under varying occlusion levels.
• Train the model using a combination of triplet loss and cross-entropy loss to optimize both global and local feature representations.

Experimental results

Research questions

• RQ1 How can pose-guided local features be effectively grouped and enhanced to improve robustness under occlusion?
• RQ2 To what extent can intra-part and inter-part relational modeling via transformers improve feature representation in occluded Re-ID?
• RQ3 What is the optimal number of transformer units for balancing accuracy and computational efficiency in this task?
• RQ4 How do confidence scores from query and gallery images affect matching performance under partial occlusion?
• RQ5 Can the proposed method generalize beyond occluded datasets and achieve competitive performance on standard Re-ID benchmarks?

Key findings

• The proposed PIRT model achieves a new state of the art on the Occluded-DukeMTMC dataset with 50.90% mAP and 60.00% Rank-1 accuracy.
• Ablation studies show that combining both intra-part and inter-part relational transformers yields the highest performance, outperforming models using only one or neither.
• Using confidence scores from both query and gallery images improves matching accuracy by 2.71% in mAP compared to using no confidence scores.
• The model with three transformer units achieves optimal performance, as deeper networks (e.g., four units) show a slight drop in mAP due to training instability.
• The method generalizes well to non-occluded datasets, achieving competitive results comparable to existing SOTA methods on standard Re-ID benchmarks.
• Visualization confirms that the intra-part relational module effectively captures local structural patterns within grouped body parts, enhancing discriminative feature learning.