[Paper Review] Multi-view Information Integration and Propagation for Occluded Person Re-identification

Neng Dong, Shuanglin Yan+3 more|arXiv (Cornell University)|Nov 7, 2023

Video Surveillance and Tracking Methods4 citations

TL;DR

This paper proposes Multi-view Information Integration and Propagation (MVI²P), a novel framework for occluded person re-identification that fuses feature maps from multiple views of the same pedestrian to create a comprehensive, noise-robust representation. By employing CAMs-aware localization and probability-aware quantification for selective, reliable integration, and an information propagation mechanism to distill knowledge to single-image inference, MVI²P achieves state-of-the-art performance with 68.2% Rank-1 and 55.6% mAP on standard benchmarks, outperforming complex models with lower parameter count.

ABSTRACT

Occluded person re-identification (re-ID) presents a challenging task due to occlusion perturbations. Although great efforts have been made to prevent the model from being disturbed by occlusion noise, most current solutions only capture information from a single image, disregarding the rich complementary information available in multiple images depicting the same pedestrian. In this paper, we propose a novel framework called Multi-view Information Integration and Propagation (MVI$^{2}$P). Specifically, realizing the potential of multi-view images in effectively characterizing the occluded target pedestrian, we integrate feature maps of which to create a comprehensive representation. During this process, to avoid introducing occlusion noise, we develop a CAMs-aware Localization module that selectively integrates information contributing to the identification. Additionally, considering the divergence in the discriminative nature of different images, we design a probability-aware Quantification module to emphatically integrate highly reliable information. Moreover, as multiple images with the same identity are not accessible in the testing stage, we devise an Information Propagation (IP) mechanism to distill knowledge from the comprehensive representation to that of a single occluded image. Extensive experiments and analyses have unequivocally demonstrated the effectiveness and superiority of the proposed MVI$^{2}$P. The code will be released at \url{https://github.com/nengdong96/MVIIP}.

Motivation & Objective

• To address the challenge of occluded person re-identification, where single-image models fail due to limited visible body parts and occlusion noise.
• To exploit complementary information from multiple views of the same pedestrian, which are typically ignored in existing single-image re-ID frameworks.
• To develop a method that selectively integrates only reliable, discriminative information from multi-view images while suppressing occlusion noise and irrelevant content.
• To enable single-image inference by distilling knowledge from a comprehensive multi-view representation learned during training.
• To achieve state-of-the-art performance with minimal model complexity, avoiding reliance on external models or data augmentation.

Proposed method

• The framework integrates feature maps from multiple views of the same pedestrian using element-wise addition to form a comprehensive representation.
• A CAMs-aware Localization module uses class activation maps to identify and prioritize spatial regions in feature maps that are discriminative for identity.
• A probability-aware Quantification module assigns dynamic weights to feature maps based on their predicted classification confidence, emphasizing more reliable information.
• An Information Propagation (IP) mechanism transfers knowledge from the comprehensive representation to a single occluded image during inference, enabling single-image reasoning.
• Knowledge distillation loss with a hyper-parameter λ is used to align the single-image feature representation with the comprehensive representation, optimizing for identity accuracy.
• The model uses ResNet-50 as the backbone and introduces only a single additional classifier, resulting in low parameter count (26.3M) and high efficiency.

Experimental results

Research questions

• RQ1 Can multi-view information from multiple images of the same pedestrian be effectively integrated to improve identity representation in occluded re-ID?
• RQ2 How can noisy or irrelevant information (e.g., occluders, non-target people) be filtered out during multi-view feature integration?
• RQ3 What is the optimal number of views to use for integration, and how does it affect performance?
• RQ4 Can knowledge from a comprehensive multi-view representation be effectively transferred to a single-image inference setting?
• RQ5 Does the proposed method achieve state-of-the-art performance with lower model complexity compared to existing SOTA approaches?

Key findings

• MVI²P achieves 68.2% Rank-1 accuracy and 55.6% mAP on the Market-1501 benchmark, outperforming the baseline and existing SOTA methods.
• The optimal hyper-parameter λ for knowledge distillation is 0.007, which maximizes both Rank-1 and mAP performance.
• Integrating information from four views yields the best performance, with a slight drop in accuracy for both two and eight views, indicating an optimal balance between information richness and redundancy.
• The model achieves SOTA performance with only 26.3M parameters, significantly fewer than methods like FED (146.2M) and OPR-DAAO (58.2M), demonstrating superior efficiency.
• Even without data augmentation or external models, MVI²P surpasses methods like UMTS* (51.3M params, 58.6% Rank-1) and IGOAS (32.6M params, 60.1% Rank-1), proving its effectiveness and efficiency.