[Paper Review] MaskReID: A Mask Based Deep Ranking Neural Network for Person Re-identification

Lei Qi, Jing Huo+3 more|arXiv (Cornell University)|Apr 11, 2018

Video Surveillance and Tracking Methods34 references61 citations

TL;DR

MaskReID introduces masked foreground inputs, a skipped multi-layer feature fusion, and a novel ranking loss to improve person re-identification performance across multiple datasets.

ABSTRACT

Person retrieval faces many challenges including cluttered background, appearance variations (e.g., illumination, pose, occlusion) among different camera views and the similarity among different person's images. To address these issues, we put forward a novel mask based deep ranking neural network with a skipped fusing layer. Firstly, to alleviate the problem of cluttered background, masked images with only the foreground regions are incorporated as input in the proposed neural network. Secondly, to reduce the impact of the appearance variations, the multi-layer fusion scheme is developed to obtain more discriminative fine-grained information. Lastly, considering person retrieval is a special image retrieval task, we propose a novel ranking loss to optimize the whole network. The proposed ranking loss can further mitigate the interference problem of similar negative samples when producing ranking results. The extensive experiments validate the superiority of the proposed method compared with the state-of-the-art methods on many benchmark datasets.

Motivation & Objective

• Mitigate background clutter by using masked foreground inputs for Re-ID.
• Capture fine-grained, cross-view information through multi-layer feature fusion.
• Develop a ranking loss that effectively handles multiple positives and negatives in retrieval.

Proposed method

• Use segmentation to generate masked foreground images and feed both original and masked inputs into the network.
• Employ a skipped feature fusion layer to combine low-, mid-, and high-level features.
• Adopt a novel ranking loss that jointly considers multiple positives and negatives to optimize ranking.
• Base the network on an inception-style architecture and a shared backbone for fused features.
• Train with an N-pair inspired ranking objective extended to multiple positives/negatives per anchor.

Experimental results

Research questions

• RQ1 Can masked foreground inputs reduce background clutter and improve Re-ID accuracy across datasets?
• RQ2 Does combining multi-level features via a skipped fusion layer yield more discriminative representations for person retrieval?
• RQ3 Does a ranking loss that uses multiple positives and negatives per anchor outperform triplet, N-pair, and standard softmax losses in Re-ID tasks?

Key findings

• MaskReID achieves competitive results on small-scale datasets and state-of-the-art performance on large-scale datasets like Market1501 and DukeMTMC-reID.
• The masked input and multi-layer feature fusion contribute to performance gains over the baseline DGD network.
• The proposed ranking loss outperforms softmax, triplet, and N-pair losses, and re-ranking further improves results.
• Ablation shows both masked inputs and fused features are beneficial, with their combination yielding the best performance.
• On Market1501 single-query, MaskReID reaches Rank-1 90.44% and mAP 75.36%; with re-ranking, Rank-1 92.46% and mAP 88.13% (single query).
• On DukeMTMC-reID, MaskReID achieves Rank-1 78.86% and mAP 61.89%; with re-ranking, Rank-1 84.07% and mAP 79.73%.