[Paper Review] Single Object Tracking: A Survey of Methods, Datasets, and Evaluation Metrics

Zahra Soleimanitaleb, Mohammad Ali Keyvanrad|arXiv (Cornell University)|Jan 31, 2022

Video Surveillance and Tracking Methods12 citations

TL;DR

This survey provides a comprehensive review of single object tracking methods, categorizing them into feature-based, segmentation-based, estimation-based, and learning-based approaches—with a focus on learning-based techniques, including generative, discriminative, and reinforcement learning subcategories. It evaluates state-of-the-art models using standard datasets and metrics, offering a unified reference for researchers in computer vision and tracking systems.

ABSTRACT

Object tracking is one of the foremost assignments in computer vision that has numerous commonsense applications such as traffic monitoring, robotics, autonomous vehicle tracking, and so on. Different researches have been tried later a long time, but since of diverse challenges such as occlusion, illumination variations, fast motion, etc. researches in this area continues. In this paper, different strategies of the following objects are inspected and a comprehensive classification is displayed that classified the following strategies into four fundamental categories of feature-based, segmentation-based, estimation-based, and learning-based methods that each of which has its claim sub-categories. The most center of this paper is on learning-based strategies, which are classified into three categories of generative strategies, discriminative strategies, and reinforcement learning. One of the sub-categories of the discriminative show is deep learning. Since of high-performance, deep learning has as of late been exceptionally much consider. Finally, the different datasets and the evaluation methods that are most commonly used will be introduced.

Motivation & Objective

• To provide a systematic classification of single object tracking methods across four main categories: feature-based, segmentation-based, estimation-based, and learning-based.
• To analyze the evolution and current state of learning-based tracking, particularly deep learning and reinforcement learning approaches.
• To review the most widely used benchmark datasets and evaluation metrics in single object tracking research.
• To serve as a comprehensive reference for researchers by consolidating advances, challenges, and trends in the field since 2019.
• To extend and update a prior conference paper (ICCKE2019) with updated methods, datasets, and evaluation standards.

Proposed method

• Categorizes tracking methods into four fundamental groups: feature-based, segmentation-based, estimation-based, and learning-based, each with sub-categories.
• Classifies learning-based methods into three subgroups: generative, discriminative, and reinforcement learning, emphasizing deep learning's dominance.
• Reviews key deep learning architectures used in tracking, such as Siamese networks and correlation filters, highlighting their design principles.
• Analyzes evaluation protocols using standard benchmarks like OTB, VOT, and TrackingNet, focusing on metrics like AOUC, precision, and success rate.
• Compares performance across methods using standardized metrics and datasets to assess robustness under challenges like occlusion and motion blur.
• Synthesizes insights from recent literature to present a holistic view of methodological trends and limitations in single object tracking.

Experimental results

Research questions

• RQ1 How do different tracking paradigms—feature-based, segmentation-based, estimation-based, and learning-based—compare in terms of accuracy and robustness?
• RQ2 What are the key advancements and limitations within deep learning-based tracking methods, particularly in discriminative and generative frameworks?
• RQ3 Which benchmark datasets and evaluation metrics are most representative of real-world tracking challenges?
• RQ4 How has the integration of reinforcement learning improved tracking performance in dynamic or complex environments?
• RQ5 What are the persistent challenges in single object tracking, such as occlusion, illumination changes, and fast motion, and how are they addressed in current methods?

Key findings

• Deep learning-based methods, especially discriminative models like Siamese networks, have significantly outperformed traditional approaches in accuracy and robustness.
• The VOT and OTB benchmarks remain the most widely used datasets, with success rate and AUC (Area Under Curve) being the dominant evaluation metrics.
• Reinforcement learning-based tracking methods show promise in handling long-term tracking and complex motion patterns, though they remain less mature than deep learning counterparts.
• Occlusion and illumination variation remain the most persistent challenges, with state-of-the-art models still struggling under severe conditions.
• The survey identifies a trend toward hybrid architectures combining appearance modeling with motion estimation to improve tracking stability.
• The paper highlights that while performance has improved, generalization across diverse visual domains remains a key open challenge in the field.