[Paper Review] ABCTracker: an easy-to-use, cloud-based application for tracking multiple objects

Lance Rice, Samual Tate+7 more|arXiv (Cornell University)|Jan 27, 2020

Species Distribution and Climate Change4 citations

TL;DR

ABCTracker is a cloud-based, user-friendly multi-object tracking system that enables researchers to track animals in videos through a three-phase workflow: marking, automated tracking, and correction. It requires minimal technical expertise, uses a three-click marking method to auto-tune parameters, and supports both marked and unmarked animals across diverse environments with high accuracy and scalability via server-side processing and automatic video chunking.

ABSTRACT

Visual multi-object tracking has the potential to accelerate many forms of quantitative analyses, especially in research communities investigating the motion, behavior, or social interactions within groups of animals. Despite its potential for increasing analysis throughput, complications related to accessibility, adaptability, accuracy, or scalable application arise with existing tracking systems. Several iterations of prototyping and testing have led us to a multi-object tracking system -- ABCTracker -- that is: accessible in both system as well as technical knowledge requirements, easily adaptable to new videos, and capable of producing accurate tracking data through a mixture of automatic and semi-automatic tracking features.

Motivation & Objective

• To address the accessibility, adaptability, and scalability challenges of existing multi-object tracking systems in behavioral biology.
• To reduce technical barriers for researchers by eliminating the need for GPU, complex installations, or image processing expertise.
• To enable accurate, reproducible tracking of multiple animals in static camera videos, including both marked and unmarked individuals.
• To support efficient processing of long video sequences through automatic chunking and parallel server-side computation.
• To provide intuitive correction tools—both manual and guided—to improve tracking accuracy with minimal user effort.

Proposed method

• Employs a client/server architecture where users mark objects via a three-click procedure (head, middle, tail/foot) on a few frames, which automatically tunes all tracking parameters.
• Uses a modular tracking pipeline that allows substitution of subroutines, enabling future integration of new algorithms and adaptability to diverse tracking challenges.
• Processes long videos by automatically splitting them into chunks, which are tracked in parallel on remote servers to improve scalability and reduce processing time per chunk.
• Applies a default general-purpose multi-object tracking algorithm optimized for static camera recordings, with no assumptions about object confinement or habitat complexity.
• Offers two correction modes: a manual mode with full control over track edits and visualization tools, and a guided mode that identifies common errors and directs users through corrective actions.
• Leverages context-aware help dialogs and low system requirements (only Java) to ensure broad accessibility across diverse computing environments.

Experimental results

Research questions

• RQ1 Can a cloud-based, minimal-technical-knowledge tracking system achieve high accuracy and scalability for multi-object tracking in diverse biological video recordings?
• RQ2 How effective is a three-click marking procedure in automatically tuning tracking parameters across different animal types and video conditions?
• RQ3 To what extent does guided correction mode reduce user effort and improve tracking accuracy compared to manual correction?
• RQ4 How does automatic video chunking and parallel processing enhance the system’s ability to handle long video sequences?
• RQ5 Can the system generalize effectively to both marked and unmarked animals in complex or cluttered backgrounds without prior parameter tuning?

Key findings

• ABCTracker successfully processed over 350 videos of both marked and unmarked animals, including ants, termites, and zebrafish, across simple and complex habitats.
• The three-click marking procedure enables automatic parameter tuning with no direct user input required for image processing or tracking parameters.
• Guided correction mode effectively identifies and directs users through common tracking errors, improving correction efficiency compared to traditional manual methods.
• Automatic video chunking allows processing of long sequences beyond the system’s native limits, with parallel processing on remote servers enabling scalable throughput.
• The system supports batch processing: while tracking runs on the server, users can simultaneously mark, track, or correct other videos, maximizing workflow efficiency.
• Despite limitations in handling complex poses (e.g., snakes), the system demonstrates strong generalization across diverse animal types and recording conditions.