[Paper Review] Decentralised Person Re-Identification with Selective Knowledge Aggregation

Shitong Sun, Guile Wu+1 more|arXiv (Cornell University)|Oct 21, 2021

Video Surveillance and Tracking Methods38 references4 citations

TL;DR

This paper proposes Selective Knowledge Aggregation (SKA), a novel federated person re-identification method that decouples local model personalization from global model generalization by learning domain-specific attentive and local batch normalization layers. The approach achieves state-of-the-art performance on both seen and unseen domains by selectively aggregating knowledge through a dual local normalization mechanism, significantly improving accuracy over prior decentralized Re-ID methods.

ABSTRACT

Existing person re-identification (Re-ID) methods mostly follow a centralised learning paradigm which shares all training data to a collection for model learning. This paradigm is limited when data from different sources cannot be shared due to privacy concerns. To resolve this problem, two recent works have introduced decentralised (federated) Re-ID learning for constructing a globally generalised model (server)without any direct access to local training data nor shared data across different source domains (clients). However, these methods are poor on how to adapt the generalised model to maximise its performance on individual client domain Re-ID tasks having different Re-ID label spaces, due to a lack of understanding of data heterogeneity across domains. We call this poor 'model personalisation'. In this work, we present a new Selective Knowledge Aggregation approach to decentralised person Re-ID to optimise the trade-off between model personalisation and generalisation. Specifically, we incorporate attentive normalisation into the normalisation layers in a deep ReID model and propose to learn local normalisation layers specific to each domain, which are decoupled from the global model aggregation in federated Re-ID learning. This helps to preserve model personalisation knowledge on each local client domain and learn instance-specific information. Further, we introduce a dual local normalisation mechanism to learn generalised normalisation layers in each local model, which are then transmitted to the global model for central aggregation. This facilitates selective knowledge aggregation on the server to construct a global generalised model for out-of-the-box deployment on unseen novel domains. Extensive experiments on eight person Re-ID datasets show that the proposed approach to decentralised Re-ID significantly outperforms the state-of-the-art decentralised methods.

Motivation & Objective

• Address the limitation of existing decentralized Re-ID methods in achieving strong model personalization on individual client domains due to data heterogeneity and divergent label spaces.
• Improve performance on both seen source domains and unseen novel domains by balancing model personalization and global generalization.
• Develop a mechanism that preserves local domain-specific knowledge without degrading the global model's ability to generalize to new, unseen domains.
• Introduce a dual local normalization strategy to enable selective knowledge aggregation that supports both personalization and generalization in federated Re-ID.

Proposed method

• Incorporate attentive normalization (AN) into batch normalization layers to learn instance-specific, domain-adaptive feature representations per client.
• Introduce local specific normalization (LSN) layers that are trained per client and decoupled from global server aggregation to preserve model personalization.
• Design a dual local normalization mechanism that learns two types of normalization: one for local personalization (LSN) and one for global generalization (G-Norm), both updated via federated learning.
• Transmit only the generalised normalization layers (G-Norm) to the central server for aggregation, while keeping LSN layers local to maintain domain-specific knowledge.
• Use a selective aggregation strategy where the global model is updated using only the G-Norm components, avoiding degradation from direct LSN aggregation.
• Train the model iteratively in a federated setting, where clients update their local models including LSN and G-Norm, and only G-Norm parameters are sent to the server for global aggregation.

Experimental results

Research questions

• RQ1 How can decentralized Re-ID methods improve model personalization on individual client domains with heterogeneous data and label spaces?
• RQ2 What is the impact of learning domain-specific normalization layers on the performance of decentralized Re-ID models on seen source domains?
• RQ3 Can a dual normalization mechanism effectively balance model personalization and generalization in federated Re-ID without degrading performance on unseen domains?
• RQ4 How does selective aggregation of generalised normalization layers compare to naive averaging of all local layers in terms of global model generalization?
• RQ5 To what extent does attentive normalization enhance feature representation in decentralized Re-ID under data heterogeneity?

Key findings

• The proposed SKA method achieves state-of-the-art performance on eight person Re-ID datasets, outperforming existing decentralized methods on both seen and unseen domains.
• On the VIPeR dataset, SKA improves R1 from 48.2% (baseline) to 53.9% on unseen domains, while achieving 66.6% R1 on seen domains, demonstrating strong personalization and generalization.
• The dual local normalization mechanism improves generalization by approximately 2% on VIPeR and iLIDS compared to a baseline with LSN, while maintaining high personalization performance.
• SKA with LSN and AN achieves 78.2% R1 on Market-1501 and 92.7% R1 on DukeMTMC, outperforming the baseline by over 10 percentage points on both seen and unseen domains.
• Ablation studies confirm that the dual normalization mechanism compensates for performance degradation caused by learning local personalization knowledge, enabling better trade-off between personalization and generalization.
• The SKA model with dual normalization achieves 72.7% R1 on iLIDS (unseen) and 49.8% on VIPeR (unseen), significantly outperforming the baseline and variants that average or concatenate local normalization layers.