[Paper Review] A Simple Baseline for Multi-Object Tracking.

Yifu Zhang, Chunyu Wang+3 more|arXiv (Cornell University)|Apr 4, 2020

Video Surveillance and Tracking Methods25 references76 citations

TL;DR

FairMOT proposes a simple yet effective single-network approach for multi-object tracking by jointly optimizing object detection and re-identification (re-ID) through two homogeneous branches. By ensuring fairness between tasks and replacing ROI-Align with a more robust feature extraction method, FairMOT achieves state-of-the-art performance on multiple benchmarks, significantly reducing identity switches and outperforming prior methods.

ABSTRACT

There has been remarkable progress on object detection and re-identification (re-ID) in recent years which are the key components of multi-object tracking. However, little attention has been focused on jointly accomplishing the two tasks in a single network. Our study shows that the previous attempts ended up with degraded accuracy mainly because the re-ID task is not fairly learned which causes many identity switches. The unfairness lies in two-fold: (1) they treat re-ID as a secondary task whose accuracy heavily depends on the primary detection task. So training is largely biased to the detection task but ignores the re-ID task; (2) they use ROI-Align to extract re-ID features which is directly borrowed from object detection. However, this introduces a lot of ambiguity in characterizing objects because many sampling points may belong to disturbing instances or background. To solve the problems, we present a simple approach \emph{FairMOT} which consists of two homogeneous branches to predict pixel-wise objectness scores and re-ID features. The achieved fairness between the tasks allows \emph{FairMOT} to obtain high levels of detection and tracking accuracy and outperform previous state-of-the-arts by a large margin on several public datasets. The source code and pre-trained models are released at this https URL.

Motivation & Objective

• To address the imbalance in training between detection and re-identification tasks in multi-object tracking.
• To reduce identity switches caused by unfair learning of the re-identification task.
• To eliminate ambiguity in re-ID feature extraction by replacing ROI-Align with a more robust method.
• To design a unified network with homogeneous branches for detection and re-ID to improve task fairness.
• To achieve state-of-the-art tracking accuracy without complex architectures or post-processing.

Proposed method

• FairMOT uses a single network with two homogeneous branches: one for predicting pixel-wise objectness scores and another for predicting re-ID features.
• The re-ID branch uses a feature extraction method that avoids the ambiguity introduced by ROI-Align, which can sample points from background or distractors.
• The network is trained end-to-end with a balanced loss function that fairly weights detection and re-identification objectives.
• Feature maps are used directly for re-ID without relying on region proposals, reducing noise and improving feature consistency.
• The method avoids using additional head networks or complex post-processing, maintaining simplicity and efficiency.
• The training strategy ensures that the re-ID task is not secondary, but equally prioritized during optimization.

Experimental results

Research questions

• RQ1 How does joint learning of detection and re-identification affect tracking performance when both tasks are fairly optimized?
• RQ2 What impact does replacing ROI-Align with a more robust feature extraction method have on re-ID accuracy and identity consistency?
• RQ3 Can a simple, single-branch architecture achieve state-of-the-art performance in multi-object tracking without complex components?
• RQ4 Why do previous methods suffer from high identity switch rates despite strong detection accuracy?
• RQ5 To what extent does task fairness improve re-identification performance in end-to-end multi-object tracking?

Key findings

• FairMOT achieves state-of-the-art performance on multiple public multi-object tracking benchmarks, outperforming previous SOTA methods.
• The method significantly reduces identity switches by ensuring fair learning between detection and re-identification tasks.
• Replacing ROI-Align with a more robust feature extraction strategy improves re-ID feature quality and consistency.
• The use of homogeneous branches for both detection and re-identification leads to better optimization and performance.
• The model achieves high accuracy in both detection and re-identification without requiring complex post-processing or auxiliary networks.
• The source code and pre-trained models are publicly released, enabling reproducibility and further research.