[Paper Review] Video Object Segmentation and Tracking: A Survey

Rui Yao, Guosheng Lin+3 more|arXiv (Cornell University)|Apr 19, 2019

Video Surveillance and Tracking MethodsComputer Science183 references42 citations

TL;DR

This paper provides a comprehensive survey of video object segmentation and tracking (VOST), proposing a hierarchical categorization of methods and summarizing datasets, metrics, and future directions.

ABSTRACT

Object segmentation and object tracking are fundamental research area in the computer vision community. These two topics are diffcult to handle some common challenges, such as occlusion, deformation, motion blur, and scale variation. The former contains heterogeneous object, interacting object, edge ambiguity, and shape complexity. And the latter suffers from difficulties in handling fast motion, out-of-view, and real-time processing. Combining the two problems of video object segmentation and tracking (VOST) can overcome their respective difficulties and improve their performance. VOST can be widely applied to many practical applications such as video summarization, high definition video compression, human computer interaction, and autonomous vehicles. This article aims to provide a comprehensive review of the state-of-the-art tracking methods, and classify these methods into different categories, and identify new trends. First, we provide a hierarchical categorization existing approaches, including unsupervised VOS, semi-supervised VOS, interactive VOS, weakly supervised VOS, and segmentation-based tracking methods. Second, we provide a detailed discussion and overview of the technical characteristics of the different methods. Third, we summarize the characteristics of the related video dataset, and provide a variety of evaluation metrics. Finally, we point out a set of interesting future works and draw our own conclusions.

Motivation & Objective

• Classify existing VOST approaches into a hierarchical taxonomy (unsupervised VOS, semi-supervised VOS, interactive VOS, weakly supervised VOS, segmentation-based tracking).
• Discuss technical characteristics of each category and how they address challenges in VOST (occlusion, deformation, motion blur, scale variation).
• Summarize related video datasets and evaluation metrics used for VOST.
• Identify future research directions and potential applications of VOST methods.

Proposed method

• Propose a five-category hierarchical taxonomy for VOST methods: unsupervised VOS, semi-supervised VOS, interactive VOS, weakly supervised VOS, and segmentation-based tracking.
• Provide detailed discussions and overviews of the technical characteristics for each category.
• Summarize and compare video datasets and evaluation metrics used in VOST research.
• Discuss practical applications and future research directions in VOST.

Experimental results

Research questions

• RQ1 What are the major categories of video object segmentation and tracking methods and how are they organized hierarchically?
• RQ2 What are the key technical characteristics and approaches within each VOST category?
• RQ3 What datasets and evaluation metrics are used to assess VOST methods, and what are their properties?
• RQ4 What future directions and challenges are identified for VOST research?

Key findings

• The survey proposes a hierarchical categorization of VOST methods into five main categories.
• It provides a detailed discussion of the technical characteristics across unsupervised VOS, semi-supervised VOS, interactive VOS, weakly supervised VOS, and segmentation-based tracking.
• It summarizes the characteristics of related video datasets and a variety of evaluation metrics for VOST.
• The paper discusses future work and potential directions to advance VOST research.
• The work clarifies the relationship between VOS and VOT and discusses how segmentation-based tracking integrates both tasks.