[Paper Review] 3D Multi-Object Tracking: A Baseline and New Evaluation Metrics

Xinshuo Weng, Jianren Wang+2 more|arXiv (Cornell University)|Jul 9, 2019

Video Surveillance and Tracking Methods58 references38 citations

TL;DR

Proposes a simple real-time 3D MOT system using a 3D Kalman filter with Hungarian data association, introduces a 3D MOT evaluation tool and three integral metrics (AMOTA, AMOTP, AMOTA) including sAMOTA, and reports state-of-the-art results on KITTI and nuScenes with high FPS.

ABSTRACT

3D multi-object tracking (MOT) is an essential component for many applications such as autonomous driving and assistive robotics. Recent work on 3D MOT focuses on developing accurate systems giving less attention to practical considerations such as computational cost and system complexity. In contrast, this work proposes a simple real-time 3D MOT system. Our system first obtains 3D detections from a LiDAR point cloud. Then, a straightforward combination of a 3D Kalman filter and the Hungarian algorithm is used for state estimation and data association. Additionally, 3D MOT datasets such as KITTI evaluate MOT methods in the 2D space and standardized 3D MOT evaluation tools are missing for a fair comparison of 3D MOT methods. Therefore, we propose a new 3D MOT evaluation tool along with three new metrics to comprehensively evaluate 3D MOT methods. We show that, although our system employs a combination of classical MOT modules, we achieve state-of-the-art 3D MOT performance on two 3D MOT benchmarks (KITTI and nuScenes). Surprisingly, although our system does not use any 2D data as inputs, we achieve competitive performance on the KITTI 2D MOT leaderboard. Our proposed system runs at a rate of $207.4$ FPS on the KITTI dataset, achieving the fastest speed among all modern MOT systems. To encourage standardized 3D MOT evaluation, our system and evaluation code are made publicly available at https://github.com/xinshuoweng/AB3DMOT.

Motivation & Objective

• Provide a simple, real-time 3D MOT baseline based on classical MOT modules (3D Kalman filter and Hungarian matching).
• Introduce a standardized 3D MOT evaluation tool and three integral metrics to evaluate across multiple operating points.
• Show that the proposed baseline achieves state-of-the-art results on KITTI and nuScenes 3D MOT benchmarks.
• Demonstrate that the system can run at high speed without heavy reliance on appearance features or training.

Proposed method

• Extract 3D detections from LiDAR point clouds using off-the-shelf detectors.
• Extend the Kalman filter state to 3D: (x, y, z, θ, l, w, h, s, vx, vy, vz).
• Predict trajectories with a 3D constant-velocity model.
• Associate detections and predictions with the Hungarian algorithm using 3D IoU or center-distance affinity.
• Update matched states with a Bayes-based update, including an orientation correction when needed.
• Manage birth and death of tracks via a birth/death memory with delay before new track creation and a capped age for tracks.

Experimental results

Research questions

• RQ1 Can a simple 3D MOT pipeline based on a 3D Kalman filter and Hungarian matching achieve strong MOT performance on standard 3D benchmarks?
• RQ2 How should 3D MOT be evaluated to fairly compare systems that operate directly in 3D space?
• RQ3 Do integral metrics over multiple confidence thresholds (AMOTA, AMOTP, sAMOTA) provide a more robust basis for comparing 3D MOT methods than single-threshold metrics?
• RQ4 Does a 3D-based MOT approach offer advantages over 2D-centric approaches in terms of accuracy and speed?

Key findings

• The proposed 3D MOT baseline achieves state-of-the-art performance on KITTI and nuScenes 3D MOT benchmarks.
• The system runs at 207.4 FPS on KITTI, the fastest among modern MOT systems.
• Three new metrics (AMOTA, AMOTP, AMOTA) and two scaled variants (sAMOTA, AMOTA) are proposed to summarize performance across all thresholds.
• The new 3D MOT evaluation tool directly evaluates in 3D space using 3D IoU or center-distance criteria, avoiding projection to the image plane for evaluation.
• Even without 2D inputs, the 3D-based method achieves competitive results on KITTI’s 2D MOT leaderboard, likely due to better depth resolution in 3D tracking.
• Ablation studies show the importance of components such as angular velocity incorporation and orientation handling for performance.