[Paper Review] PersonNet: Person Re-identification with Deep Convolutional Neural Networks

Lin Wu, Chunhua Shen+1 more|arXiv (Cornell University)|Jan 27, 2016

Video Surveillance and Tracking MethodsComputer Science2 references210 citations

TL;DR

PersonNet introduces a very deep Siamese CNN with 3x3 filters and a neighborhood difference layer to jointly learn features and a similarity metric for person re-identification, achieving state-of-the-art results on multiple datasets.

ABSTRACT

In this paper, we propose a deep end-to-end neu- ral network to simultaneously learn high-level features and a corresponding similarity metric for person re-identification. The network takes a pair of raw RGB images as input, and outputs a similarity value indicating whether the two input images depict the same person. A layer of computing neighborhood range differences across two input images is employed to capture local relationship between patches. This operation is to seek a robust feature from input images. By increasing the depth to 10 weight layers and using very small (3$ imes$3) convolution filters, our architecture achieves a remarkable improvement on the prior-art configurations. Meanwhile, an adaptive Root- Mean-Square (RMSProp) gradient decent algorithm is integrated into our architecture, which is beneficial to deep nets. Our method consistently outperforms state-of-the-art on two large datasets (CUHK03 and Market-1501), and a medium-sized data set (CUHK01).

Motivation & Objective

• Motivate and develop a deep end-to-end network that jointly learns robust features and a similarity metric for person re-identification.
• Increase network depth with small 3x3 convolutions to improve discriminative power under cross-view variations.
• Incorporate a neighborhood difference layer to model local patch relations and misalignment between camera views.
• Adopt RMSProp for adaptive gradient updates to facilitate training of a deep network.
• Demonstrate state-of-the-art performance on multiple large re-id benchmarks.

Proposed method

• Use a pair of RGB images as input to a Siamese-like network with tied weights across views.
• Stack 3x3 convolutional layers with max-pooling, followed by a neighborhood patch matching layer that computes cross-view differences in local patches.
• Include a patch summary layer and subsequent convolutional/max-pooling layers, ending in three fully connected layers for a softmax similarity decision (same/different).
• Employ a 10-layer deep architecture with 3x3 receptive fields to increase non-linearity and representation capacity.
• Use hyperbolic tangent activations and RMSProp instead of standard SGD for training the deep network.
• Apply online sampling of image pairs and data augmentation (translation and horizontal reflection) to balance positive/negative pairs.

Experimental results

Research questions

• RQ1 Can a deeper CNN with small 3x3 filters improve person re-identification accuracy over prior architectures?
• RQ2 Does incorporating a neighborhood patch difference layer help model local cross-view variations and misalignment?
• RQ3 What is the impact of RMSProp as an optimization method for training deep networks in re-id tasks?
• RQ4 How does the proposed method perform on CUHK03, CUHK01, and Market-1501 compared to state-of-the-art methods?

Key findings

• PersonNet achieves the best reported rank-1 accuracy on CUHK03, CUHK01, and Market-1501 in the reported experiments.
• On CUHK03, 64.80% (Rank-1) and up to 98.20% (Rank-20) are achieved, outperforming prior methods.
• On CUHK01, Rank-1 is 71.14%, with Rank-5 90.07%, Rank-10 95.00%, and Rank-20 98.06%.
• On Market-1501, Rank-1 is 37.21% and mAP is 18.57%.
• The convergence study shows RMSProp provides more stable and faster convergence than SGD for this deep architecture.