[Paper Review] A Baseline for 3D Multi-Object Tracking

Xinshuo Weng, Kris Kitani|arXiv (Cornell University)|Jul 9, 2019

Advanced Neural Network Applications32 references81 citations

TL;DR

This paper proposes a simple yet highly efficient real-time 3D multi-object tracking (MOT) baseline system that combines an off-the-shelf 3D object detector with a 3D Kalman filter and Hungarian algorithm for data association. Despite its simplicity, it achieves state-of-the-art performance on the KITTI 3D MOT benchmark, improving 3D MOTA to 76.47 and running at 214.7 FPS—65× faster than the prior SOTA 2D MOT system.

ABSTRACT

3D multi-object tracking (MOT) is an essential component technology for many real-time applications such as autonomous driving or assistive robotics. However, recent works for 3D MOT tend to focus more on developing accurate systems giving less regard to computational cost and system complexity. In contrast, this work proposes a simple yet accurate real-time baseline 3D MOT system. We use an off-the-shelf 3D object detector to obtain oriented 3D bounding boxes from the LiDAR point cloud. Then, a combination of 3D Kalman filter and Hungarian algorithm is used for state estimation and data association. Although our baseline system is a straightforward combination of standard methods, we obtain the state-of-the-art results. To evaluate our baseline system, we propose a new 3D MOT extension to the official KITTI 2D MOT evaluation along with two new metrics. Our proposed baseline method for 3D MOT establishes new state-of-the-art performance on 3D MOT for KITTI, improving the 3D MOTA from 72.23 of prior art to 76.47. Surprisingly, by projecting our 3D tracking results to the 2D image plane and compare against published 2D MOT methods, our system places 2nd on the official KITTI leaderboard. Also, our proposed 3D MOT method runs at a rate of 214.7 FPS, 65 times faster than the state-of-the-art 2D MOT system. Our code is publicly available at this https URL

Motivation & Objective

• To develop a computationally efficient and accurate 3D MOT system that balances performance and speed.
• To establish a practical real-time baseline for 3D multi-object tracking in autonomous driving and robotics.
• To evaluate 3D MOT performance using an extended KITTI 2D MOT benchmark with new metrics.
• To demonstrate that a simple combination of standard components can outperform complex, specialized systems.

Proposed method

• Uses an off-the-shelf 3D object detector to generate oriented 3D bounding boxes from LiDAR point clouds.
• Employs a 3D Kalman filter for state estimation of tracked objects.
• Applies the Hungarian algorithm for data association between detections and existing tracks.
• Projects 3D tracking results to the 2D image plane for cross-benchmark comparison with 2D MOT methods.
• Introduces a new 3D MOT evaluation protocol extending the official KITTI 2D MOT benchmark.
• Implements two new metrics to assess 3D MOT performance beyond standard MOTA.

Experimental results

Research questions

• RQ1 Can a simple, modular combination of standard 3D detection and tracking components achieve state-of-the-art 3D MOT performance?
• RQ2 How does the performance of a real-time 3D MOT system compare to SOTA 2D MOT systems when projected to 2D?
• RQ3 What is the computational efficiency of a lightweight 3D MOT baseline compared to complex SOTA systems?
• RQ4 To what extent can a simple 3D tracker outperform specialized systems in terms of accuracy and speed?
• RQ5 How does the proposed 3D MOT evaluation protocol improve the assessment of 3D tracking performance?

Key findings

• The proposed 3D MOT baseline achieves a 3D MOTA of 76.47 on the KITTI dataset, surpassing the prior state-of-the-art of 72.23.
• When projected to 2D, the system ranks 2nd on the official KITTI 2D MOT leaderboard.
• The system runs at 214.7 FPS, which is 65 times faster than the state-of-the-art 2D MOT system.
• The method demonstrates that simplicity and efficiency do not compromise accuracy in 3D MOT.
• The new evaluation protocol and metrics enable more comprehensive and fair benchmarking of 3D MOT systems.
• The results show that standard components like Kalman filters and the Hungarian algorithm remain highly effective in modern 3D tracking when properly integrated.