[Paper Review] Lifted Disjoint Paths with Application in Multiple Object Tracking

Andrea Horňáková, Roberto Henschel+2 more|arXiv (Cornell University)|Jun 25, 2020

Video Surveillance and Tracking MethodsComputer Science84 citations

TL;DR

This work extends disjoint paths with lifted edges to capture long-range temporal interactions for MOT, develops a tight LP relaxation with cutting planes, and shows strong global optimization performance on MOT benchmarks.

ABSTRACT

We present an extension to the disjoint paths problem in which additional \emph{lifted} edges are introduced to provide path connectivity priors. We call the resulting optimization problem the lifted disjoint paths problem. We show that this problem is NP-hard by reduction from integer multicommodity flow and 3-SAT. To enable practical global optimization, we propose several classes of linear inequalities that produce a high-quality LP-relaxation. Additionally, we propose efficient cutting plane algorithms for separating the proposed linear inequalities. The lifted disjoint path problem is a natural model for multiple object tracking and allows an elegant mathematical formulation for long range temporal interactions. Lifted edges help to prevent id switches and to re-identify persons. Our lifted disjoint paths tracker achieves nearly optimal assignments with respect to input detections. As a consequence, it leads on all three main benchmarks of the MOT challenge, improving significantly over state-of-the-art.

Motivation & Objective

• Extend the disjoint paths framework with lifted edges to encode long-range connectivity priors for MOT.
• Develop a high-quality linear programming relaxation with nontrivial polyhedral inequalities.
• Provide separation routines to efficiently add cutting planes for lifted constraints.
• Apply the approach to multiple object tracking and demonstrate strong performance on MOT benchmarks.
• Offer a tractable two-step graph construction and cost-learning pipeline for practical MOT use.

Proposed method

• Define lifted disjoint paths on a base flow graph and a lifted graph with edges E' encoding v→w connectivity via v→…→w paths in G.
• Derive and tighten linear inequalities (path, path-induced cut, and lifted variants) to obtain a strong LP relaxation.
• Develop efficient separation procedures (Algorithms 1–3) to add violated lifted constraints during solving.
• Solve the ILP with Gurobi using LP-based branch-and-bound on the tightened relaxation.
• Construct graphs over detections and tracklets with a two-step procedure and learn edge costs from visual and motion cues (re-id, DeepMatching, motion, spatio-temporal features).

Experimental results

Research questions

• RQ1 Can lifted edges express long-range temporal connectivity to improve MOT beyond first-order models?
• RQ2 Do new polyhedral inequalities yield a tighter relaxation than naive liftings for the lifted disjoint paths problem?
• RQ3 Is there a practical global optimization approach that leverages long-range interactions to outperform state-of-the-art MOT trackers?
• RQ4 How does the lifted disjoint paths tracker perform on standard MOT benchmarks compared to prior methods?

Key findings

• The lifted disjoint paths problem is NP-hard (even with only negative or only positive lifted edges).
• The proposed lifted path and lifted path-induced cut inequalities yield a strictly tighter LP relaxation than their non-lifted counterparts.
• Separation procedures for the lifted constraints run in linear time with respect to the number of active base edges, enabling practical solving.
• The global optimization approach yields near-optimal assignments and significantly improves MOT performance on major benchmarks.