[Paper Review] Foreground-aware Pyramid Reconstruction for Alignment-free Occluded Person Re-identification

Lingxiao He, Yinggang Wang+5 more|arXiv (Cornell University)|Apr 10, 2019

Video Surveillance and Tracking Methods20 references52 citations

TL;DR

The paper introduces Foreground-aware Pyramid Reconstruction (FPR), an alignment-free occluded person ReID method using FCN-based spatial pyramids and a foreground-guided reconstruction to handle occlusions without external cues.

ABSTRACT

Re-identifying a person across multiple disjoint camera views is important for intelligent video surveillance, smart retailing and many other applications. However, existing person re-identification (ReID) methods are challenged by the ubiquitous occlusion over persons and suffer from performance degradation. This paper proposes a novel occlusion-robust and alignment-free model for occluded person ReID and extends its application to realistic and crowded scenarios. The proposed model first leverages the full convolution network (FCN) and pyramid pooling to extract spatial pyramid features. Then an alignment-free matching approach, namely Foreground-aware Pyramid Reconstruction (FPR), is developed to accurately compute matching scores between occluded persons, despite their different scales and sizes. FPR uses the error from robust reconstruction over spatial pyramid features to measure similarities between two persons. More importantly, we design an occlusion-sensitive foreground probability generator that focuses more on clean human body parts to refine the similarity computation with less contamination from occlusion. The FPR is easily embedded into any end-to-end person ReID models. The effectiveness of the proposed method is clearly demonstrated by the experimental results (Rank-1 accuracy) on three occluded person datasets: Partial REID (78.30\%), Partial iLIDS (68.08\%) and Occluded REID (81.00\%); and three benchmark person datasets: Market1501 (95.42\%), DukeMTMC (88.64\%) and CUHK03 (76.08\%)

Motivation & Objective

• Address occlusion in person re-identification without relying on external cues like masks or poses.
• Develop an alignment-free matching score based on pyramid features and reconstruction error.
• Guide reconstruction with foreground probability maps to reduce occlusion/background contamination.
• Enable end-to-end integration with existing ReID models while improving Rank-1 and mAP under occlusion.

Proposed method

• Use a Fully Convolutional Network (FCN) with Pyramid Pooling to extract multi-scale spatial features from variable-size person images.
• Construct gallery and probe feature matrices from spatial features and perform linear reconstruction of probe features from gallery features.
• Compute reconstruction residuals and define a distance as a weighted reconstruction error guided by a foreground probability vector.
• Train with hard triplet loss together with a foreground probability generator loss to learn discriminative features and accurate foreground maps.
• Foreground Probability Generator uses a 1x1 conv and softmax to produce foreground probability maps that weight features by likelihood of being foreground.
• Integrate FPR as an end-to-end module within ReID models without requiring alignment steps.

Experimental results

Research questions

• RQ1 Can an alignment-free matching score based on spatial pyramid reconstruction achieve robust ReID under heavy occlusion?
• RQ2 Does foreground-aware weighting of spatial features improve matching by minimizing occlusion/background contamination?
• RQ3 How well does the proposed FPR integrate with standard ReID training, and what is its impact on Rank-1/mAP on occluded and non-occluded datasets?

Key findings

• FPR achieves state-of-the-art results on three occluded ReID datasets: Partial REID (Rank-1 81.00%, mAP 76.60%), Partial iLIDS (Rank-1 68.08%, mAP 61.78%), Occluded REID (Rank-1 78.30%, mAP 68.00%).
• On standard benchmarks Market1501, DukeMTMC, and CUHK03, FPR achieves competitive results (Rank-1 around 95.42%/76.08%/88.64% and mAP around 86.58%/72.31%/78.42%).
• FPR outperforms several occluded-ReID methods that rely on external cues or alignment, while remaining alignment-free during inference.
• The foreground probability generator effectively reduces the influence of occlusion and background by weighting foreground spatial features more heavily during reconstruction.
• The approach remains robust to different image sizes and scales due to FCN-based spatial features and pyramid pooling.