[Paper Review] Efficient Global Multi-object Tracking Under Minimum-cost Circulation Framework

Congchao Wang, Yizhi Wang+1 more|arXiv (Cornell University)|Nov 2, 2019

Video Surveillance and Tracking Methods40 references4 citations

TL;DR

This paper proposes CINDA, a minimum-cost circulation framework that directly solves global multi-object tracking data association without iterative flow amount search, achieving up to 1,192× speedup over state-of-the-art methods while maintaining optimal solutions. By exploiting unit-capacity vertex structure, it reduces theoretical and practical complexity, enabling efficient, globally optimal tracking even on large-scale datasets with long temporal sequences.

ABSTRACT

We developed a minimum-cost circulation framework for solving the global data association problem, which plays a key role in the tracking-by-detection paradigm of multi-object tracking. The global data association problem was extensively studied under the minimum-cost flow framework, which is theoretically attractive as being flexible and globally solvable. However, the high computational burden has been a long-standing obstacle to its wide adoption in practice. While enjoying the same theoretical advantages and maintaining the same optimal solution as the minimum-cost flow framework, our new framework has a better theoretical complexity bound and leads to orders of practical efficiency improvement. This new framework is motivated by the observation that minimum-cost flow only partially models the data association problem and must be accompanied by an additional and time-consuming searching scheme to determine the optimal object number. By employing a minimum-cost circulation framework, we eliminate the searching step and naturally integrate the number of objects into the optimization problem. By exploring the special property of the associated graph, that is, an overwhelming majority of the vertices are with unit capacity, we designed an implementation of the framework and proved it has the best theoretical complexity so far for the global data association problem. We evaluated our method with 40 experiments on five MOT benchmark datasets. Our method was always the most efficient and averagely 53 to 1,192 times faster than the three state-of-the-art methods. When our method served as a sub-module for global data association methods using higher-order constraints, similar efficiency improvement was attained. We further illustrated through several case studies how the improved computational efficiency enables more sophisticated tracking models and yields better tracking accuracy.

Motivation & Objective

• To address the high computational cost of minimum-cost flow-based global data association in multi-object tracking (MOT), which limits scalability to long sequences.
• To eliminate the need for iterative flow amount search in existing minimum-cost flow methods by integrating object count directly into the optimization.
• To develop a theoretically and practically more efficient framework than minimum-cost flow for global data association in MOT.
• To enable the use of higher-order tracking models and iterative refinement by drastically reducing computational overhead.

Proposed method

• Formulates the MAP-based data association problem as a minimum-cost circulation problem instead of minimum-cost flow, naturally incorporating object count into the optimization.
• Leverages the structural property that most vertices in the association graph have unit capacity to design an efficient algorithm with improved theoretical complexity.
• Implements a specialized algorithm that exploits unit-capacity vertices to reduce computational cost, achieving the best-known theoretical complexity bound for this problem.
• Uses a circulation-based formulation that avoids the need to pre-specify the number of objects, eliminating time-consuming search procedures.
• Employs a network flow model where source and sink nodes connect to detections, and edges represent inter-frame associations with cost based on detection similarity.
• Applies advanced shortest path and dual ascent techniques tailored to the unit-capacity structure for fast convergence.

Experimental results

Research questions

• RQ1 Can a minimum-cost circulation framework outperform minimum-cost flow in both theoretical complexity and practical efficiency for global multi-object tracking?
• RQ2 Does eliminating the iterative flow amount search step significantly improve computational efficiency in data association?
• RQ3 Can the proposed framework scale to large-scale MOT datasets with long sequences and high detection counts?
• RQ4 To what extent does improved efficiency enable more sophisticated tracking models with higher-order constraints?
• RQ5 Can the framework support iterative refinement of affinity scores without prohibitive computational cost?

Key findings

• CINDA achieved 53 to 1,192 times faster runtime than three state-of-the-art methods across 40 experiments on five MOT benchmarks.
• The method consistently outperformed all baselines in every experiment, demonstrating robust and scalable efficiency.
• Theoretical analysis showed that CINDA achieves the best-known computational complexity bound for global data association.
• The framework enabled the use of higher-order tracking models and iterative affinity refinement, which were previously infeasible due to computational cost.
• Case studies showed that the improved efficiency allowed for better tracking accuracy by leveraging longer temporal contexts and more complex models.
• The source code was released with both Python and MATLAB interfaces, facilitating adoption and further research.