[Paper Review] A Distilled Model for Tracking and Tracker Fusion.

Matteo Dunnhofer, Niki Martinel+1 more|arXiv (Cornell University)|Jul 8, 2020

Video Surveillance and Tracking Methods65 references4 citations

TL;DR

This paper proposes a distilled student model that unifies fast tracking, online adaptation, and tracker fusion by combining knowledge distillation and reinforcement learning. The resulting compact model achieves state-of-the-art performance in real-time tracking across single-shot inference, adaptive tracking, and multi-tracker fusion.

ABSTRACT

Visual object tracking was generally tackled by reasoning independently on fast processing algorithms, accurate online adaptation methods, and fusion of trackers. In this paper, we unify such goals by proposing a novel tracking methodology that takes advantage of other visual trackers, offline and online. A compact student model is trained via the marriage of knowledge distillation and reinforcement learning. The first allows to transfer and compress tracking knowledge of other trackers. The second enables the learning of evaluation measures which are then exploited online. After learning, the student can be ultimately used to build (i) a very fast single-shot tracker, (ii) a tracker with a simple and effective online adaptation mechanism, (iii) a tracker that performs fusion of other trackers. Extensive validation shows that the proposed algorithms compete with state-of-the-art trackers while running in real-time.

Motivation & Objective

• Address the limitations of independent tracker design by unifying fast inference, online adaptation, and tracker fusion into a single framework.
• Reduce computational cost while preserving tracking accuracy by compressing knowledge from multiple trackers into a compact student model.
• Enable real-time performance in tracking applications through a lightweight, trainable model that leverages both offline and online learning.
• Improve tracker robustness and adaptability by learning dynamic evaluation measures through reinforcement learning.

Proposed method

• Train a compact student model using knowledge distillation to transfer tracking knowledge from multiple source trackers.
• Integrate reinforcement learning to learn dynamic evaluation metrics that guide online decision-making during inference.
• Use the distilled student model as a unified framework for single-shot inference, online adaptation, and tracker fusion.
• Leverage offline knowledge transfer to compress complex tracking behaviors into a lightweight architecture.
• Optimize the student model to balance speed and accuracy by learning from diverse tracking strategies.
• Apply the student model at inference time to perform real-time tracking with adaptive behavior and fusion capabilities.

Experimental results

Research questions

• RQ1 Can knowledge distillation and reinforcement learning be effectively combined to create a compact, high-performance tracker?
• RQ2 To what extent can a distilled model achieve real-time performance while maintaining accuracy across diverse tracking scenarios?
• RQ3 How well does the student model generalize to single-shot inference, online adaptation, and tracker fusion?
• RQ4 Can the learned evaluation measures from reinforcement learning improve tracker robustness compared to fixed metrics?

Key findings

• The distilled student model achieves state-of-the-art performance in visual object tracking while running in real-time.
• The model enables fast single-shot inference with minimal latency, suitable for real-time applications.
• The integration of reinforcement learning allows effective online adaptation without requiring retraining.
• Tracker fusion using the student model improves robustness and accuracy compared to individual trackers.
• The knowledge distillation process successfully compresses complex tracking behaviors into a lightweight architecture.
• The model demonstrates strong generalization across diverse tracking sequences, outperforming existing methods in benchmark evaluations.