[Paper Review] BASE: Probably a Better Approach to Multi-Object Tracking

Martin Vonheim Larsen, Sigmund Rolfsjord+3 more|arXiv (Cornell University)|Sep 21, 2023

Video Surveillance and Tracking MethodsComputer Science3 citations

TL;DR

BASE proposes a minimalist, probabilistic single-hypothesis tracker that achieves state-of-the-art performance on MOT17 and MOT20 by integrating a distance-aware motion model, dynamic clutter estimation, probabilistic association, and calibrated detector confidence—demonstrating that probabilistic methods can outperform ad hoc SOTA trackers when key visual tracking challenges are properly modeled.

ABSTRACT

The field of visual object tracking is dominated by methods that combine simple tracking algorithms and ad hoc schemes. Probabilistic tracking algorithms, which are leading in other fields, are surprisingly absent from the leaderboards. We found that accounting for distance in target kinematics, exploiting detector confidence and modelling non-uniform clutter characteristics is critical for a probabilistic tracker to work in visual tracking. Previous probabilistic methods fail to address most or all these aspects, which we believe is why they fall so far behind current state-of-the-art (SOTA) methods (there are no probabilistic trackers in the MOT17 top 100). To rekindle progress among probabilistic approaches, we propose a set of pragmatic models addressing these challenges, and demonstrate how they can be incorporated into a probabilistic framework. We present BASE (Bayesian Approximation Single-hypothesis Estimator), a simple, performant and easily extendible visual tracker, achieving state-of-the-art (SOTA) on MOT17 and MOT20, without using Re-Id. Code will be made available at https://github.com/ffi-no

Motivation & Objective

• To address the underperformance of probabilistic trackers in visual multi-object tracking (VMOT) compared to ad hoc methods.
• To identify and model critical visual tracking challenges overlooked by prior probabilistic approaches: non-uniform kinematics, clutter distribution, and detector confidence.
• To demonstrate that a principled probabilistic framework can achieve SOTA performance when these factors are properly accounted for.
• To provide a minimal, extensible, and principled foundation for future probabilistic VMOT research.
• To show that probabilistic association and dynamic modeling are essential for performance, especially when combined with calibrated confidence.

Proposed method

• Replaces traditional SHT components with probabilistic counterparts: probabilistic gating, association, and track management.
• Introduces a distance-aware motion model that increases position uncertainty for larger (closer) bounding boxes to reflect higher kinematic uncertainty.
• Models clutter density dynamically using a histogram-based estimation of false positive rates, adapting to scene-specific clutter levels.
• Employs a likelihood-ratio-based probabilistic association that fuses position, size, and confidence into a unified metric.
• Calibrates detector confidence scores using a histogram-based method to improve reliability in association decisions.
• Uses maximum likelihood estimation (MLE) to automatically learn model parameters from training data, ensuring robustness and generalization.

Experimental results

Research questions

• RQ1 Why do probabilistic trackers underperform in visual multi-object tracking despite their theoretical advantages?
• RQ2 How can probabilistic trackers be adapted to handle non-uniform target kinematics and clutter distribution in visual tracking?
• RQ3 What role does detector confidence play in probabilistic association, and how can it be effectively modeled?
• RQ4 Can a minimalist probabilistic tracker achieve SOTA performance without Re-Id or complex architectures?
• RQ5 What components are essential for a probabilistic tracker to match or exceed ad hoc SOTA methods in VMOT?

Key findings

• BASE achieves SOTA on MOT17 and MOT20 benchmarks, outperforming leading ad hoc methods like ByteTrack and BoT-SORT without using Re-Id.
• The ablation study shows that dynamic clutter modeling is essential for the distance-aware motion model to improve performance, as a constant clutter model degrades results.
• Probabilistic association with a naive motion model underperforms IOU-based association, but when combined with the distance-aware model, it surpasses IOU-based methods.
• Calibrated detector confidence improves performance by 0.8 MOTA points over raw confidence and 3.8 MOTA points over ignoring confidence, showing its critical role.
• The full BASE configuration achieves 81.6 MOTA and 70.2 HOTA on MOT17 validation set, outperforming BoT-SORT (78.5 MOTA) and approaching end-to-end models like MOTRv2.
• The method demonstrates that probabilistic trackers can be both simple and high-performing when key visual tracking characteristics—distance effects, clutter, and confidence—are properly modeled.