[Paper Review] Person Re-identification with Deep Similarity-Guided Graph Neural Network

Yantao Shen, Hongsheng Li+3 more|arXiv (Cornell University)|Jul 26, 2018

Video Surveillance and Tracking Methods44 references17 citations

TL;DR

This paper proposes a Similarity-Guided Graph Neural Network (SGGNN) for person re-identification that models inter-pair relationships among probe-gallery image pairs via a graph structure, using gallery-gallery similarity labels to guide edge weights and enable end-to-end message passing for improved similarity estimation. SGGNN achieves state-of-the-art performance, improving mAP by up to 3.6% and top-1 accuracy by 3.0% on public benchmarks.

ABSTRACT

The person re-identification task requires to robustly estimate visual similarities between person images. However, existing person re-identification models mostly estimate the similarities of different image pairs of probe and gallery images independently while ignores the relationship information between different probe-gallery pairs. As a result, the similarity estimation of some hard samples might not be accurate. In this paper, we propose a novel deep learning framework, named Similarity-Guided Graph Neural Network (SGGNN) to overcome such limitations. Given a probe image and several gallery images, SGGNN creates a graph to represent the pairwise relationships between probe-gallery pairs (nodes) and utilizes such relationships to update the probe-gallery relation features in an end-to-end manner. Accurate similarity estimation can be achieved by using such updated probe-gallery relation features for prediction. The input features for nodes on the graph are the relation features of different probe-gallery image pairs. The probe-gallery relation feature updating is then performed by the messages passing in SGGNN, which takes other nodes' information into account for similarity estimation. Different from conventional GNN approaches, SGGNN learns the edge weights with rich labels of gallery instance pairs directly, which provides relation fusion more precise information. The effectiveness of our proposed method is validated on three public person re-identification datasets.

Motivation & Objective

• Address the limitation of existing person re-identification models that treat probe-gallery pairs independently, ignoring inter-pair relationships.
• Improve similarity estimation accuracy for hard positive and negative samples by modeling relational dependencies among multiple probe-gallery pairs.
• Integrate rich gallery-gallery similarity labels into the graph learning process to guide more effective feature fusion.
• Enable end-to-end training of both feature learning and similarity estimation by embedding graph computation into the deep learning pipeline.
• Demonstrate that relational modeling via graph networks enhances not only final ranking performance but also the discriminativeness of learned visual features.

Proposed method

• Construct a graph where each node represents a probe-gallery image pair, with node features initialized as pairwise relation features.
• Define edge weights between nodes based on the similarity scores of the corresponding gallery images, using rich supervision from gallery-gallery pairs.
• Perform deep message passing across the graph to update node representations by aggregating information from neighboring nodes.
• Use a learnable message aggregation function that incorporates both node features and edge weights to refine relation representations.
• Train the entire SGGNN framework end-to-end, jointly optimizing visual feature learning and graph-based similarity refinement.
• Apply a similarity-guided fusion mechanism that dynamically adjusts feature aggregation using gallery-gallery similarity labels as attention weights.

Experimental results

Research questions

• RQ1 Can modeling inter-pair relationships among probe-gallery image pairs improve similarity estimation in person re-identification?
• RQ2 How does incorporating gallery-gallery similarity labels into edge weight learning affect the performance of graph-based re-identification models?
• RQ3 Does end-to-end training of a graph neural network with relational message passing lead to better visual feature learning and ranking accuracy?
• RQ4 To what extent does SGGNN outperform conventional GNNs and post-processing re-ranking methods in terms of mAP and top-1 accuracy?
• RQ5 Can the graph-based message passing mechanism improve the robustness of similarity estimation on hard samples?

Key findings

• SGGNN achieves state-of-the-art performance on three public datasets, improving mAP by 3.6%, 4.1%, and 2.2% on Market-1501, CUHK03, and DukeMTMC, respectively.
• The ablation study shows that removing gallery-gallery supervision (Base Model + SGGNN w/o SG) leads to a 1.6% mAP drop on Market-1501, demonstrating the importance of rich label guidance.
• Visual features learned by SGGNN outperform those from the base model and conventional GNNs, with a 2.1% mAP gain on Market-1501 when using $l_2$ distance for similarity estimation.
• The model is robust to hyperparameter choices: top-100 with $K=4$ achieves the best balance of accuracy and efficiency, while increasing $K$ or top-$K$ yields marginal gains at higher computational cost.
• The ablation study confirms that SGGNN’s message passing mechanism significantly improves top-1 accuracy by 1.7–2.6% compared to models without gallery-gallery supervision.
• The model achieves a 3.0% top-1 accuracy gain on DukeMTMC and a 4.1% mAP gain on CUHK03, highlighting its effectiveness on challenging datasets with complex cross-camera variations.