[Paper Review] AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metrics

Xinshuo Weng, Jianren Wang+2 more|arXiv (Cornell University)|Aug 18, 2020

Video Surveillance and Tracking MethodsComputer Science11 references44 citations

TL;DR

A simple, real-time 3D MOT baseline using a 3D Kalman filter and Hungarian data association, plus a new 3D evaluation tool and metrics for KITTI. It achieves strong 3D MOT performance and fastest FPS among modern systems.

ABSTRACT

3D multi-object tracking (MOT) is essential to applications such as autonomous driving. Recent work focuses on developing accurate systems giving less attention to computational cost and system complexity. In contrast, this work proposes a simple real-time 3D MOT system with strong performance. Our system first obtains 3D detections from a LiDAR point cloud. Then, a straightforward combination of a 3D Kalman filter and the Hungarian algorithm is used for state estimation and data association. Additionally, 3D MOT datasets such as KITTI evaluate MOT methods in 2D space and standardized 3D MOT evaluation tools are missing for a fair comparison of 3D MOT methods. We propose a new 3D MOT evaluation tool along with three new metrics to comprehensively evaluate 3D MOT methods. We show that, our proposed method achieves strong 3D MOT performance on KITTI and runs at a rate of $207.4$ FPS on the KITTI dataset, achieving the fastest speed among modern 3D MOT systems. Our code is publicly available at http://www.xinshuoweng.com/projects/AB3DMOT.

Motivation & Objective

• Motivate the need for a simple, real-time 3D MOT baseline that balances accuracy with computational efficiency.
• Propose a straightforward 3D MOT pipeline using 3D detections, a 3D Kalman filter, and the Hungarian algorithm for data association.
• Identify and address gaps in 3D MOT evaluation by introducing a direct 3D evaluation tool and new metrics.
• Evaluate the proposed baseline on KITTI 3D MOT to demonstrate performance and speed advantages.

Proposed method

• Use an off-the-shelf 3D detector to obtain 3D detections from LiDAR point clouds.
• Extend the 3D Kalman filter state to include 3D location, size, velocity, and orientation with a constant-velocity model.
• Apply the Hungarian algorithm for data association in 3D space.
• Develop a 3D MOT evaluation tool by replacing 2D IoU with 3D IoU and matching in 3D space.
• Introduce three integral metrics (sAMOTA, AMOTA, AMOTP) to summarize MOT performance across recall thresholds.

Experimental results

Research questions

• RQ1 Can a simple 3D MOT pipeline using a 3D Kalman filter and Hungarian matching achieve competitive performance on KITTI?
• RQ2 Does evaluating 3D MOT directly in 3D space provide fairer and more informative comparisons than projecting to 2D?
• RQ3 Do integral metrics across recall values better capture robustness to confidence thresholds than traditional MOT metrics?

Key findings

• The proposed 3D MOT system achieves strong performance on KITTI with 3D inputs only and no reliance on 2D data.
• The method runs at 207.4 FPS on the KITTI val set, the fastest among modern 3D MOT systems without GPU.
• A 3D evaluation tool and 3D IoU-based matching enable direct 3D evaluation instead of projecting to 2D.
• Three integral metrics (sAMOTA, AMOTA, AMOTP) summarize MOTA/MOTP across recall values, addressing sensitivity to confidence thresholds.
• Compared to mmMOT and FANTrack, the proposed approach shows higher sAMOTA, AMOTA, and AMOTP on KITTI val, with zero identity switches in the reported run.