[Paper Review] Recognizing Partial Biometric Patterns

Lingxiao He, Zhenan Sun+2 more|arXiv (Cornell University)|Oct 17, 2018

Face recognition and analysisComputer Science42 references84 citations

TL;DR

This paper proposes an alignment-free, end-to-end framework using Spatial Feature Reconstruction (SFR) and batch hard triplet learning to recognize partial biometric patterns across person re-identification and partial face recognition tasks, achieving state-of-the-art results on multiple datasets.

ABSTRACT

Biometric recognition on partial captured targets is challenging, where only several partial observations of objects are available for matching. In this area, deep learning based methods are widely applied to match these partial captured objects caused by occlusions, variations of postures or just partial out of view in person re-identification and partial face recognition. However, most current methods are not able to identify an individual in case that some parts of the object are not obtainable, while the rest are specialized to certain constrained scenarios. To this end, we propose a robust general framework for arbitrary biometric matching scenarios without the limitations of alignment as well as the size of inputs. We introduce a feature post-processing step to handle the feature maps from FCN and a dictionary learning based Spatial Feature Reconstruction (SFR) to match different sized feature maps in this work. Moreover, the batch hard triplet loss function is applied to optimize the model. The applicability and effectiveness of the proposed method are demonstrated by the results from experiments on three person re-identification datasets (Market1501, CUHK03, DukeMTMC-reID), two partial person datasets (Partial REID and Partial iLIDS) and two partial face datasets (CASIA-NIR-Distance and Partial LFW), on which state-of-the-art performance is ensured in comparison with several state-of-the-art approaches. The code is released online and can be found on the website: https://github.com/lingxiao-he/Partial-Person-ReID.

Motivation & Objective

• Address biometric recognition when targets are partially observed due to occlusion, pose variation, or partial view.
• Develop a general framework that does not rely on input size or strict alignment.
• Introduce Spatial Feature Reconstruction (SFR) with dictionary learning to match arbitrary-sized feature maps.
• Embed SFR within a batch hard triplet learning paradigm for end-to-end optimization.
• Demonstrate effectiveness across diverse datasets including full and partial re-id and partial face datasets.

Proposed method

• Use a Fully Convolutional Network (FCN) encoder to generate spatial feature maps from input images.
• Apply a feature post-processing unit consisting of global average pooling (GAP) and pyramid pooling to obtain global and multi-scale spatial features.
• Introduce Spatial Feature Reconstruction (SFR): reconstruct each spatial feature of the probe from a gallery set via a learned linear combination with L2 regularization, enabling alignment-free matching for varying input sizes.
• Compute a spatial feature reconstruction distance Ds(X, Y) based on reconstruction residuals and use it alongside global feature distance in a fused metric.
• Train with a batch hard triplet loss (BH) to maximize distance between different identities and minimize within-identity distances, extended with SFR (BH-SFR) for improved discrimination.
• Provide an alternating optimization routine that updates reconstruction coefficients W then backpropagates through the FCN to optimize θ.

Experimental results

Research questions

• RQ1 Can recognition be robust for arbitrary-size, partially observed biometric targets without alignment?
• RQ2 Does combining global features with spatial feature reconstruction improve partial biometric recognition across re-id and partial face tasks?
• RQ3 How effective is batch hard triplet learning when augmented with spatial feature reconstruction in end-to-end training?
• RQ4 What is the method's performance across standard and partial-biometrics datasets (Market1501, CUHK03, DukeMTMC-reID, Partial-REID, Partial-iLIDS, CASIA-NIR-Distance, Partial LFW)?

Key findings

• SFR achieves competitive performance across standard and partial biometric datasets.
• On Market1501 (single query), SFR reaches 93.04% Rank-1 and 81.02% mAP, outperforming several baselines.
• On Market1501 (multi-query), SFR reaches 94.84% Rank-1 and 85.47% mAP.
• On CUHK03 (Labeled), SFR achieves 67.29% Rank-1 and 61.47% mAP; (Detected) 63.86% Rank-1 and 58.97% mAP.
• The proposed approach demonstrates strong results relative to multiple contemporary methods (part-based, mask-guided, pose-guided, and attention-based) and shows robustness to scale and alignment issues.