[Paper Review] Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification

Yixiao Ge, Dapeng Chen+1 more|arXiv (Cornell University)|Jan 6, 2020

Video Surveillance and Tracking Methods50 references394 citations

TL;DR

The paper introduces Mutual Mean-Teaching (MMT), an unsupervised domain adaptation framework for person re-ID that refines noisy pseudo labels via online soft labels and offline hard labels using a two-network mutual teaching setup with a soft softmax-triplet loss.

ABSTRACT

Person re-identification (re-ID) aims at identifying the same persons' images across different cameras. However, domain diversities between different datasets pose an evident challenge for adapting the re-ID model trained on one dataset to another one. State-of-the-art unsupervised domain adaptation methods for person re-ID transferred the learned knowledge from the source domain by optimizing with pseudo labels created by clustering algorithms on the target domain. Although they achieved state-of-the-art performances, the inevitable label noise caused by the clustering procedure was ignored. Such noisy pseudo labels substantially hinders the model's capability on further improving feature representations on the target domain. In order to mitigate the effects of noisy pseudo labels, we propose to softly refine the pseudo labels in the target domain by proposing an unsupervised framework, Mutual Mean-Teaching (MMT), to learn better features from the target domain via off-line refined hard pseudo labels and on-line refined soft pseudo labels in an alternative training manner. In addition, the common practice is to adopt both the classification loss and the triplet loss jointly for achieving optimal performances in person re-ID models. However, conventional triplet loss cannot work with softly refined labels. To solve this problem, a novel soft softmax-triplet loss is proposed to support learning with soft pseudo triplet labels for achieving the optimal domain adaptation performance. The proposed MMT framework achieves considerable improvements of 14.4%, 18.2%, 13.1% and 16.4% mAP on Market-to-Duke, Duke-to-Market, Market-to-MSMT and Duke-to-MSMT unsupervised domain adaptation tasks. Code is available at https://github.com/yxgeee/MMT.

Motivation & Objective

• Tackle the label noise problem in clustering-based unsupervised domain adaptation for person re-ID.
• Develop a pseudo label refinery framework using online soft labels and offline hard labels.
• Enable training with soft triplet labels through a novel soft softmax-triplet loss.
• Show that mutual mean-teaching yields significant gains across standard domain adaptation benchmarks.

Proposed method

• Use two collaboration networks with temporal average models to generate reliable soft pseudo labels for the other network.
• Refine target-domain pseudo labels using off-line hard pseudo labels and on-line soft pseudo labels in an alternating training process.
• Introduce soft softmax-triplet loss to enable learning with soft pseudo triplet labels.
• Combine soft classification loss and soft softmax-triplet loss during joint optimization (with two networks) and select a best-average model for inference.

Experimental results

Research questions

• RQ1 Can refining pseudo labels in the target domain with a mutual teacher-student framework improve unsupervised domain adaptation for person re-ID?
• RQ2 Does a soft softmax-triplet loss enable effective learning with softly refined triplet labels?
• RQ3 What are the performance gains of MMT compared to state-of-the-art clustering-based UDA methods on standard re-ID benchmarks?

Key findings

• MMT achieves substantial improvements over prior clustering-based UDA methods across four domain adaptation tasks: Market-to-Duke, Duke-to-Market, Market-to-MSMT, and Duke-to-MSMT.
• The framework delivers notable mAP gains (consistently reported as 14.4%, 18.2%, 13.4%, and 16.4% in the abstract for the four tasks).
• Using two temporally averaged networks to produce soft pseudo labels helps mitigate label noise and stabilizes training.
• The proposed soft softmax-triplet loss enables effective learning with soft triplet labels, contributing to improved discriminative feature learning.
• MMT approaches fully-supervised-level performance on several tasks without target-domain annotations and without post-processing steps like re-ranking.