[Paper Review] Enforcing Template Representability and Temporal Consistency for Adaptive Sparse Tracking

Xue Yang, Fei Han+2 more|ArXiv.org|Apr 30, 2016

Video Surveillance and Tracking Methods27 references3 citations

TL;DR

This paper proposes TRAC, a novel sparse tracking algorithm that enforces template representability and temporal consistency to improve robustness against appearance variations in visual tracking. By using structured sparsity-inducing norms for temporal regularization and an adaptive template update strategy based on long-term and short-term representability, TRAC significantly reduces drift and outperforms state-of-the-art trackers on 12 benchmark sequences.

ABSTRACT

Sparse representation has been widely studied in visual tracking, which has shown promising tracking performance. Despite a lot of progress, the visual tracking problem is still a challenging task due to appearance variations over time. In this paper, we propose a novel sparse tracking algorithm that well addresses temporal appearance changes, by enforcing template representability and temporal consistency (TRAC). By modeling temporal consistency, our algorithm addresses the issue of drifting away from a tracking target. By exploring the templates' long-term-short-term representability, the proposed method adaptively updates the dictionary using the most descriptive templates, which significantly improves the robustness to target appearance changes. We compare our TRAC algorithm against the state-of-the-art approaches on 12 challenging benchmark image sequences. Both qualitative and quantitative results demonstrate that our algorithm significantly outperforms previous state-of-the-art trackers.

Motivation & Objective

• Address the persistent problem of tracker drift due to temporal appearance variations in visual tracking.
• Improve robustness to appearance changes such as pose variation, occlusion, illumination changes, and background clutter.
• Overcome limitations of traditional sparse trackers that rely solely on importance weights for template updates, which can lead to redundant or non-discriminative templates.
• Develop a method that adaptively selects and updates templates based on their representability across short- and long-term temporal contexts.
• Ensure theoretical convergence and computational efficiency in solving the sparse tracking optimization problem.

Proposed method

• Introduce a structured sparsity-inducing norm as a temporal regularization term to enforce consistency in target appearance across consecutive frames.
• Formulate the tracking problem within a particle filter framework, where each particle is represented using sparse linear combinations of target templates.
• Propose an adaptive template update mechanism that evaluates both long-term and short-term representability of candidate templates, selecting only the most descriptive ones.
• Use a mixed $β_{2,1}$-norm regularization to promote joint sparsity across particles and enhance discriminative power of the dictionary.
• Design a new optimization algorithm with theoretical convergence guarantees to solve the non-convex, structured sparse optimization problem efficiently.
• Integrate the temporal consistency and adaptive representability constraints into a unified objective function for robust tracking.

Experimental results

Research questions

• RQ1 How can temporal consistency in target appearance be effectively modeled to reduce tracker drift in visual tracking?
• RQ2 To what extent does incorporating long-term and short-term representability improve template selection and tracking robustness?
• RQ3 Can an adaptive template update strategy based on representability outperform traditional importance-weight-based update schemes?
• RQ4 How does the proposed structured sparsity regularization compare to standard $β_1$-norm or $β_{2,1}$-norm regularization in handling appearance variations?
• RQ5 Does the proposed optimization framework converge reliably and efficiently in practice, especially under complex visual conditions?

Key findings

• TRAC significantly outperforms 10 state-of-the-art trackers on 12 benchmark sequences, achieving the best performance in both precision and success rate metrics.
• On the pose variation sequences (david2, dudek, trellis), TRAC maintains accurate tracking throughout, while methods like OAB, L1APG, and CT fail due to drift or occlusion.
• In sequences with illumination variation and background clutter, TRAC demonstrates superior robustness, with minimal drift and consistent bounding box overlap.
• The success rate plot shows TRAC achieves the highest average overlap score across all thresholds, particularly excelling at higher overlap thresholds.
• The precision metric reveals that TRAC maintains a high percentage of frames within 20 pixels of the ground truth center location, outperforming all competing methods.
• Ablation studies confirm that both temporal consistency and adaptive template representability contribute significantly to performance, with each component improving tracking accuracy and stability.