[Paper Review] LEGO: Learning and Graph-Optimized Modular Tracker for Online Multi-Object Tracking with Point Clouds

Zhenrong Zhang, Jianan Liu+4 more|arXiv (Cornell University)|Aug 19, 2023

Video Surveillance and Tracking MethodsComputer Science3 citations

TL;DR

LEGO is a novel online multi-object tracking framework for LiDAR point clouds that integrates graph optimization and self-attention mechanisms to enhance data association accuracy. By fusing geometry, motion, and detection score features through a learnable cost matrix and a Graph-Driven Attention Network (GDAN), LEGO achieves state-of-the-art performance, ranking 1st on the KITTI MOT benchmark for online trackers using only LiDAR input.

ABSTRACT

Online multi-object tracking (MOT) plays a pivotal role in autonomous systems. The state-of-the-art approaches usually employ a tracking-by-detection method, and data association plays a critical role. This paper proposes a learning and graph-optimized (LEGO) modular tracker to improve data association performance in the existing literature. The proposed LEGO tracker integrates graph optimization and self-attention mechanisms, which efficiently formulate the association score map, facilitating the accurate and efficient matching of objects across time frames. To further enhance the state update process, the Kalman filter is added to ensure consistent tracking by incorporating temporal coherence in the object states. Our proposed method utilizing LiDAR alone has shown exceptional performance compared to other online tracking approaches, including LiDAR-based and LiDAR-camera fusion-based methods. LEGO ranked 1st at the time of submitting results to KITTI object tracking evaluation ranking board and remains 2nd at the time of submitting this paper, among all online trackers in the KITTI MOT benchmark for cars1

Motivation & Objective

• To address the challenges of data association, false detection rejection, and tracking continuity in online multi-object tracking using LiDAR point clouds.
• To improve tracking robustness and accuracy by integrating geometric, motion, and detection score features through a unified optimization framework.
• To develop a modular tracker that enhances data association using graph-based attention and Kalman filtering for temporal consistency.
• To achieve state-of-the-art performance on the KITTI online MOT benchmark using only LiDAR input, without camera fusion.

Proposed method

• LEGO employs a two-stage pipeline: object detection followed by tracking using a modular tracker with graph-optimized data association.
• It constructs a cost matrix combining geometry and motion features (matrix A) and detection score features (matrix B), with learnable weights to balance their contributions.
• A Graph-Driven Attention Network (GDAN) is used to compute association scores by modeling topological relationships between detections across frames.
• The tracker integrates a Kalman filter to maintain temporal coherence and refine object state estimates across frames.
• An offset correction module is applied to refine detection bounding boxes, reducing localization errors.
• Track management uses a threshold-based pruning mechanism (N^t) to handle track lifetime and reduce false positives.

Experimental results

Research questions

• RQ1 How can graph optimization and attention mechanisms be effectively combined to improve data association in online 3D multi-object tracking?
• RQ2 What is the optimal balance between geometric, motion, and detection score features in the association cost matrix?
• RQ3 Can a LiDAR-only tracker achieve state-of-the-art performance compared to multi-modal (LiDAR-camera) trackers?
• RQ4 How does the integration of Kalman filtering and track management improve tracking continuity and robustness?

Key findings

• LEGO achieved 85.808% HOTA on the KITTI car validation set, ranking 1st among all online trackers at submission time.
• When using PointRCNN as the detector, LEGO improved HOTA by 2.4 and MOTA by 3.44 over StrongFusionMOT.
• With PointGNN as the detector, LEGO achieved a 3.62 HOTA gain over HIDMOT and a 5.34 HOTA gain over DualTracker.
• LEGO outperformed GNN3DMOT by 1.22 in sAMOTA, 2.51 in AMOTA, and 8.87 in AMOTP, despite using a less accurate detector (PointGNN) than CenterPoint.
• The ablation study showed that setting the weight w_B to 2 and the track threshold N^t to 14 yielded the best HOTA performance (85.808%).
• The offset correction module contributed to improved localization accuracy, enhancing overall tracking robustness.