[Paper Review] Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking

Jinlong Peng, Changan Wang+8 more|arXiv (Cornell University)|Jul 29, 2020

Video Surveillance and Tracking Methods40 references32 citations

TL;DR

The paper introduces Chained-Tracker (CTracker), an online end-to-end MOT model that regresses paired bounding boxes for adjacent frames and chains them to form trajectories, achieving state-of-the-art results on MOT16/ MOT17 without extra training data.

ABSTRACT

Existing Multiple-Object Tracking (MOT) methods either follow the tracking-by-detection paradigm to conduct object detection, feature extraction and data association separately, or have two of the three subtasks integrated to form a partially end-to-end solution. Going beyond these sub-optimal frameworks, we propose a simple online model named Chained-Tracker (CTracker), which naturally integrates all the three subtasks into an end-to-end solution (the first as far as we know). It chains paired bounding boxes regression results estimated from overlapping nodes, of which each node covers two adjacent frames. The paired regression is made attentive by object-attention (brought by a detection module) and identity-attention (ensured by an ID verification module). The two major novelties: chained structure and paired attentive regression, make CTracker simple, fast and effective, setting new MOTA records on MOT16 and MOT17 challenge datasets (67.6 and 66.6, respectively), without relying on any extra training data. The source code of CTracker can be found at: github.com/pjl1995/CTracker.

Motivation & Objective

• Motivate and address suboptimality in tracking-by-detection and partially end-to-end MOT approaches.
• Propose a totally end-to-end model that jointly performs detection, feature extraction, and data association.
• Introduce paired attentive regression and a chained-structure to convert cross-frame association into a pair-wise detection problem.
• Demonstrate state-of-the-art MOT performance on MOT16 and MOT17 without extra training data.

Proposed method

• Process adjacent frame pairs (chain nodes) as input and regress paired bounding boxes representing the same target in the two frames.
• Utilize a joint attention module with object-attention and identity-attention to guide the paired-box regression.
• Introduce Chained-Anchors to predict two bounding boxes for adjacent frames within a single regression.
• Chain adjacent nodes via IoU-based matching and the Kuhn-Munkres algorithm to form long trajectories.
• Incorporate a memory sharing mechanism (MSM) to reuse frame features across nodes and speed up inference.
• Train with a multi-task loss that includes regression, classification, and ID verification terms with focal losses.

Experimental results

Research questions

• RQ1 Can an end-to-end MOT model that jointly optimizes detection, feature extraction, and data association outperform traditional tracking-by-detection and partially end-to-end methods?
• RQ2 Does pairing attentions (object and identity) improve regression accuracy and data association in an online MOT setting?
• RQ3 Can a chained, adjacent-frame regression approach convert cross-frame association into a robust pair-wise detection problem?
• RQ4 What are the efficiency and accuracy trade-offs of memory sharing for online MOT inference?

Key findings

• CTracker achieves state-of-the-art MOTA on MOT16 (67.6) and MOT17 (66.6) without extra training data.
• Ablation shows object-attention and joint attention (including ID verification) substantially improve MOTA and IDF1.
• The full Joint-Attention version (CTracker) improves IDF1 notably, reflecting better data association, with a slight drop in MOTP.
• Memory Sharing Mechanism reduces computation, enabling tracking at ~34.4 FPS on 1080p input.
• Compared to online MOT baselines on MOT17, CTracker reaches 66.6 MOTA and 57.4 IDF1 with competitive MOTP.
• The chaining strategy effectively forms long trajectories via IoU-based matching and KM assignment across adjacent frame pairs.