[Paper Review] Geometric Hypergraph Learning for Visual Tracking

Dawei Du, Honggang Qi+4 more|arXiv (Cornell University)|Mar 18, 2016

Video Surveillance and Tracking Methods40 references4 citations

TL;DR

This paper proposes a geometric hypergraph learning framework for visual tracking that models high-order geometric relations among multiple correspondence hypotheses between target and candidate parts, improving robustness to deformation and occlusion. By formulating tracking as mode-seeking on a confidence-aware hypergraph, the method achieves state-of-the-art performance on VOT2014 and Deform-SOT benchmarks.

ABSTRACT

Graph based representation is widely used in visual tracking field by finding correct correspondences between target parts in consecutive frames. However, most graph based trackers consider pairwise geometric relations between local parts. They do not make full use of the target's intrinsic structure, thereby making the representation easily disturbed by errors in pairwise affinities when large deformation and occlusion occur. In this paper, we propose a geometric hypergraph learning based tracking method, which fully exploits high-order geometric relations among multiple correspondences of parts in consecutive frames. Then visual tracking is formulated as the mode-seeking problem on the hypergraph in which vertices represent correspondence hypotheses and hyperedges describe high-order geometric relations. Besides, a confidence-aware sampling method is developed to select representative vertices and hyperedges to construct the geometric hypergraph for more robustness and scalability. The experiments are carried out on two challenging datasets (VOT2014 and Deform-SOT) to demonstrate that the proposed method performs favorable against other existing trackers.

Motivation & Objective

• Address the limitations of pairwise geometric modeling in visual tracking, which are prone to error propagation under large deformation and occlusion.
• Exploit high-order geometric structures among multiple correspondence hypotheses to improve discriminative power and robustness.
• Develop a scalable and noise-resistant hypergraph construction method through confidence-aware sampling for large-scale tracking.
• Formulate visual tracking as a mode-seeking problem on a geometric hypergraph to identify structurally consistent, high-confidence correspondence groups.
• Demonstrate superior performance on challenging benchmarks with significant deformation, occlusion, and appearance variation.

Proposed method

• Construct a geometric hypergraph where vertices represent correspondence hypotheses between target parts (from previous frame) and candidate parts (in current frame).
• Define hyperedges as high-order geometric relations among multiple correspondence hypotheses, encoding consistent spatial configurations (e.g., triangle angles) across parts.
• Introduce a confidence-aware sampling strategy to select representative vertices and hyperedges, reducing computational cost and improving robustness to noise.
• Formulate tracking as a mode-seeking problem on the hypergraph, using pairwise coordinate update to locate the structural correspondence mode with maximum confidence.
• Use angle invariance in triangle-based similarity measures to handle scale changes and improve robustness to appearance variation.
• Leverage the hypergraph structure to jointly model both inlier geometric structures among target parts and contextual relations with background parts.

Experimental results

Research questions

• RQ1 Can high-order geometric relations among multiple correspondence hypotheses improve tracking robustness under large deformation and occlusion compared to pairwise models?
• RQ2 How can a geometric hypergraph be efficiently constructed and optimized to balance accuracy and scalability in visual tracking?
• RQ3 To what extent does confidence-aware sampling reduce sensitivity to noisy correspondences while preserving key structural information?
• RQ4 Can the mode-seeking process on a hypergraph effectively identify the most consistent and reliable correspondence group for accurate target localization?
• RQ5 How does the proposed method compare to state-of-the-art trackers on benchmarks with severe challenges like deformation, occlusion, and illumination variation?

Key findings

• The proposed Geometric hyperGraph Tracker (GGT) achieves the highest success score and second-highest precision score on the Deform-SOT dataset, outperforming existing methods.
• On the VOT2014 dataset, GGT demonstrates favorable performance, particularly in handling sequences with large deformation and partial occlusion.
• The tracker maintains accurate tracking under abrupt motion, pose variation, and rotation, where methods like SCM and TCP fail due to drift.
• The use of angle-invariant similarity measures in hyperedges enables robustness to scale changes, outperforming DGT, which relies only on pairwise relations.
• The confidence-aware sampling method effectively reduces hypergraph size and computational load while preserving critical structural information, enhancing scalability.
• The method successfully tracks targets in background clutter scenarios (e.g., football sequence) by modeling both structural consistency and contextual relations, avoiding drift to distractors.