[Paper Review] SVDNet for Pedestrian Retrieval

Yifan Sun, Liang Zheng+2 more|arXiv (Cornell University)|Mar 16, 2017

Video Surveillance and Tracking Methods24 references123 citations

TL;DR

SVDNet decorrelates the last fully connected layer in CNNs using iterative SVD-based decorrelation (RRI), improving person re-identification accuracy across Market-1501, CUHK03, and DukeMTMC-reID.

ABSTRACT

This paper proposes the SVDNet for retrieval problems, with focus on the application of person re-identification (re-ID). We view each weight vector within a fully connected (FC) layer in a convolutional neuron network (CNN) as a projection basis. It is observed that the weight vectors are usually highly correlated. This problem leads to correlations among entries of the FC descriptor, and compromises the retrieval performance based on the Euclidean distance. To address the problem, this paper proposes to optimize the deep representation learning process with Singular Vector Decomposition (SVD). Specifically, with the restraint and relaxation iteration (RRI) training scheme, we are able to iteratively integrate the orthogonality constraint in CNN training, yielding the so-called SVDNet. We conduct experiments on the Market-1501, CUHK03, and Duke datasets, and show that RRI effectively reduces the correlation among the projection vectors, produces more discriminative FC descriptors, and significantly improves the re-ID accuracy. On the Market-1501 dataset, for instance, rank-1 accuracy is improved from 55.3% to 80.5% for CaffeNet, and from 73.8% to 82.3% for ResNet-50.

Motivation & Objective

• Motivate re-ID by addressing redundancy in FC layer weight vectors after classification training.
• Propose a decorrelation technique to produce orthogonal projection directions in the FC layer.
• Develop a three-step Restraint and Relaxation Iteration (RRI) training to steadily enforce orthogonality while preserving discriminative power.
• Evaluate SVDNet on large-scale re-ID benchmarks and compare with state-of-the-art methods.

Proposed method

• Represent the penultimate FC layer as an Eigenlayer with an orthogonal weight matrix obtained via SVD.
• Replace W with US (from W = USV^T) to decorrelate projection directions while preserving discriminative power in the embedding.
• Apply a three-step training loop: (1) Decorrelation by SVD (W ← US); (2) Restraint by fine-tuning with Eigenlayer fixed; (3) Relaxation by continuing fine-tuning with Eigenlayer unfixed; repeat for multiple RRIs.
• Use an Eigenlayer before the final FC layer (no bias; linear) to enforce orthogonality and produce improved embeddings for Euclidean distance-based retrieval.
• Evaluate with CaffeNet and ResNet-50 backbones, varying Eigenlayer dimension (1024 in experiments) and employing 1,024-dim embeddings for retrieval.

Experimental results

Research questions

• RQ1 Can decorrelating the last but one FC layer via SVD improve Euclidean-distance-based re-ID retrieval without sacrificing discriminability?
• RQ2 Does iterative restraint-relaxation training converge to an orthogonal weight matrix and yield sustained performance gains?
• RQ3 How does the Eigenlayer orthogonality affect both input and output embeddings for re-ID tasks?

Key findings

• SVDNet significantly boosts rank-1 and mAP on Market-1501, CUHK03, and DukeMTMC-reID compared with baselines.
• On Market-1501 with CaffeNet, rank-1 improves from 55.3% to 80.5% and mAP from 30.4% to 55.9% for FC6; with FC7, rank-1 improves from 54.6% to 79.0% and mAP from 30.3% to 54.6%.
• With ResNet-50, SVDNet achieves substantial gains (e.g., DukeMTMC-reID rank-1 76.7%, mAP 56.8%; Market-1501 82.3/62.1) over baselines.
• Replacing W with US preserves discriminative power and improves decorrelation, whereas other decorrelation methods degrade performance.
• SVDNet converges in practice within a limited number of RRIs (e.g., 7 RRIs for ResNet-50, 25 for CaffeNet) and uses a correlation metric S(W) to monitor orthogonality (S(W) grows toward 1/k).
• Output dimension of the Eigenlayer balances performance and redundancy, with a peak performance before degradation as dimension grows beyond an optimum.