[Paper Review] Beyond 3D Siamese Tracking: A Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds

Chaoda Zheng, Yan Xu+5 more|arXiv (Cornell University)|Mar 3, 2022

Advanced Neural Network Applications4 citations

TL;DR

This paper proposes M²-Track, a motion-centric 3D single object tracking framework for LiDAR point clouds that bypasses appearance matching by modeling inter-frame target motion. It uses motion transformation in a two-stage pipeline—first localizing the target via motion prediction, then refining the bounding box with motion-assisted shape completion—achieving state-of-the-art performance with 57 FPS and 8–22% precision gains on KITTI, NuScenes, and Waymo Open Dataset.

ABSTRACT

3D single object tracking (3D SOT) in LiDAR point clouds plays a crucial role in autonomous driving. Current approaches all follow the Siamese paradigm based on appearance matching. However, LiDAR point clouds are usually textureless and incomplete, which hinders effective appearance matching. Besides, previous methods greatly overlook the critical motion clues among targets. In this work, beyond 3D Siamese tracking, we introduce a motion-centric paradigm to handle 3D SOT from a new perspective. Following this paradigm, we propose a matching-free two-stage tracker M^2-Track. At the 1^st-stage, M^2-Track localizes the target within successive frames via motion transformation. Then it refines the target box through motion-assisted shape completion at the 2^nd-stage. Extensive experiments confirm that M^2-Track significantly outperforms previous state-of-the-arts on three large-scale datasets while running at 57FPS (~8%, ~17%, and ~22%) precision gains on KITTI, NuScenes, and Waymo Open Dataset respectively). Further analysis verifies each component's effectiveness and shows the motion-centric paradigm's promising potential when combined with appearance matching.

Motivation & Objective

• Address the limitations of Siamese-based 3D SOT in textureless, incomplete, and distractor-rich LiDAR point clouds.
• Overcome the reliance on appearance matching, which fails under fast motion or low frame rates.
• Leverage inherent motion cues in 3D scenes to build a robust, matching-free tracking paradigm.
• Design a two-stage tracker that uses motion modeling and shape completion to improve localization accuracy.
• Demonstrate that motion-centric tracking can be combined with appearance matching to further boost performance.

Proposed method

• Propose a motion-centric paradigm that localizes the target in consecutive frames using relative motion estimation instead of appearance matching.
• Design a two-stage tracker: in Stage 1, predict the inter-frame motion to localize the target via motion transformation.
• In Stage 2, perform motion-assisted shape completion by fusing partial target views using their relative motion to refine the 3D bounding box.
• Use a box-aware feature enhancement module to improve feature representation for motion prediction.
• Integrate a binary motion classification head to distinguish between target motion and background motion.
• Utilize a lightweight PointNet backbone to maintain real-time inference (57 FPS) while achieving high accuracy.

Experimental results

Research questions

• RQ1 Can motion modeling in 3D point clouds serve as a viable alternative to appearance matching in 3D SOT?
• RQ2 How does a motion-centric tracker perform under challenging conditions such as fast motion, occlusion, and dense traffic?
• RQ3 Can motion-based tracking be effectively combined with appearance-matching methods to further improve performance?
• RQ4 What is the contribution of motion modeling versus specific architectural components in the overall tracking accuracy?
• RQ5 How robust is the motion-centric paradigm to variations in frame sampling rates?

Key findings

• M²-Track achieves state-of-the-art performance on KITTI, NuScenes, and Waymo Open Dataset, with 8%, 17%, and 22% relative precision gains respectively.
• The model runs at 57 FPS, making it 1.67× faster than the previous top-performer BAT while using 31.1% less memory.
• Ablation studies confirm that each component—motion prediction, shape completion, and box-aware feature enhancement—contributes significantly to performance.
• The motion-centric paradigm is robust to distractors and appearance changes, especially in dense traffic scenes.
• Integrating M²-Track with appearance-based trackers like BAT or P2B further improves performance, demonstrating compatibility and synergy.
• Motion augmentation improves generalization, particularly for models trained on low-frame-rate data, while harming appearance-based methods due to increased distractor proximity.