[Paper Review] Deep Association Learning for Unsupervised Video Person Re-identification

Yanbei Chen, Xiatian Zhu+1 more|arXiv (Cornell University)|Aug 22, 2018

Video Surveillance and Tracking MethodsComputer Science59 citations

TL;DR

The paper presents Deep Association Learning (DAL), an end-to-end unsupervised video re-id method that jointly optimizes intra-camera and cross-camera association losses to learn discriminative features without identity labels, achieving state-of-the-art results on PRID 2011, iLIDS-VID, and MARS.

ABSTRACT

Deep learning methods have started to dominate the research progress of video-based person re-identification (re-id). However, existing methods mostly consider supervised learning, which requires exhaustive manual efforts for labelling cross-view pairwise data. Therefore, they severely lack scalability and practicality in real-world video surveillance applications. In this work, to address the video person re-id task, we formulate a novel Deep Association Learning (DAL) scheme, the first end-to-end deep learning method using none of the identity labels in model initialisation and training. DAL learns a deep re-id matching model by jointly optimising two margin-based association losses in an end-to-end manner, which effectively constrains the association of each frame to the best-matched intra-camera representation and cross-camera representation. Existing standard CNNs can be readily employed within our DAL scheme. Experiment results demonstrate that our proposed DAL significantly outperforms current state-of-the-art unsupervised video person re-id methods on three benchmarks: PRID 2011, iLIDS-VID and MARS.

Motivation & Objective

• Address the lack of scalable, labelled data for video-based person re-id by developing an unsupervised end-to-end CNN approach.
• Leverage two forms of consistency—local intra-camera space-time consistency and global cross-camera cyclic ranking consistency—to learn robust representations.
• Eliminate manual identity labeling while enabling effective cross-camera tracklet association through self-discovered anchors.

Proposed method

• Introduce two sets of anchors: intra-camera anchors x_{k,i} for each camera and cross-camera anchors a_{k,i} merged from highly associated intra-camera anchors across cameras.
• Define two margin-based top-push association losses: L_I for intra-camera ranking and L_C for cross-camera (cross-camera) ranking, computed against dynamic anchors and tracklet frames.
• Intra-camera learning updates anchors via exponential moving average of frame features and enforces top-push to ensure the source tracklet remains top-ranked within a camera.
• Cross-camera learning discovers cross-camera associations through cyclic ranking of intra-camera anchors and merges paired anchors into cross-camera anchors when cyclic consistency holds.
• Train the model end-to-end by jointly optimizing L_DAL = L_I + lambda L_C with standard SGD/Adam-like optimizers on CNN backbones (ResNet50 or MobileNet) initialized from ImageNet.
• Utilize batch-wise iterative procedures to progressively discover and leverage cross-camera correspondences during training.

Experimental results

Research questions

• RQ1 Can DAL learn effective video re-id representations without any identity labels?
• RQ2 Do intra-camera local consistency and cross-camera cyclic ranking provide complementary supervision to improve unsupervised video re-id?
• RQ3 How well does end-to-end DAL perform across standard CNN backbones on public benchmarks without labelled data?
• RQ4 How does the rate of cross-camera association evolve during training and how does this affect re-id performance?

Key findings

• DAL significantly outperforms state-of-the-art unsupervised video re-id methods on PRID 2011, iLIDS-VID, and MARS.
• Rank-1 accuracy improvements over prior unsupervised methods are 4.4% on PRID 2011, 15.2% on iLIDS-VID, and 12.5% on MARS.
• Using cross-camera association alone already yields competitive results, with further gains when combined with intra-camera learning.
• DAL achieves consistent performance across backbones (ResNet50 and MobileNet), demonstrating versatility with standard CNNs.
• A large portion of tracklets become cross-camera associated during training (90% on PRID 2011, 75% on iLIDS-VID, >50% on MARS), with a high true-match rate among discovered cross-camera pairs.
• Compared to ID-labelled supervised training, DAL can reach comparable performance on smaller datasets and approaches supervised performance on some datasets, highlighting strong unsupervised learning capability.