[Paper Review] Efficient Diverse Ensemble for Discriminative Co-Tracking

Kourosh Meshgi, Shigeyuki Oba+1 more|arXiv (Cornell University)|Nov 16, 2017

Video Surveillance and Tracking Methods56 references3 citations

TL;DR

This paper proposes an Efficient Diverse Ensemble Tracker (DEDT) that enhances discriminative co-tracking by generating artificial training data through a query-by-committee mechanism to maximize diversity among ensemble members. By combining randomized data subsets with actively generated, challenging artificial samples, DEDT reduces model correlation and drift, achieving state-of-the-art performance on OTB50 and OTB100 benchmarks with superior accuracy and robustness to label noise and occlusion.

ABSTRACT

Ensemble discriminative tracking utilizes a committee of classifiers, to label data samples, which are in turn, used for retraining the tracker to localize the target using the collective knowledge of the committee. Committee members could vary in their features, memory update schemes, or training data, however, it is inevitable to have committee members that excessively agree because of large overlaps in their version space. To remove this redundancy and have an effective ensemble learning, it is critical for the committee to include consistent hypotheses that differ from one-another, covering the version space with minimum overlaps. In this study, we propose an online ensemble tracker that directly generates a diverse committee by generating an efficient set of artificial training. The artificial data is sampled from the empirical distribution of the samples taken from both target and background, whereas the process is governed by query-by-committee to shrink the overlap between classifiers. The experimental results demonstrate that the proposed scheme outperforms conventional ensemble trackers on public benchmarks.

Motivation & Objective

• To address model drift and label noise in discriminative tracking caused by self-learning loops and correlated ensemble members.
• To improve ensemble generalization by maximizing diversity among classifiers through controlled data augmentation.
• To develop an efficient, real-time co-tracking framework that maintains long- and short-term memory while enabling active data exchange between primary and auxiliary classifiers.
• To break the self-learning loop using a co-tracking setup with an auxiliary classifier that validates and corrects predictions.
• To enhance tracker robustness under visual challenges such as occlusion, deformation, and background clutter through diversified training.

Proposed method

• The method constructs an ensemble using randomized subsets of tracking data to reduce initial correlation among classifiers.
• Artificial training samples are generated from the empirical distribution of target and background patches, using multivariate Gaussian sampling around the last known target location.
• Labels for artificial samples are selected via a query-by-committee strategy to challenge the ensemble’s current consensus, promoting diversity.
• The ensemble is updated iteratively: if the ensemble’s accuracy improves after incorporating artificial samples, they are accepted; otherwise, new samples are generated.
• An auxiliary classifier in a co-tracking framework provides external supervision, with its long-term memory used to validate and correct the primary tracker’s predictions.
• The framework integrates batch updates for the auxiliary classifier to stabilize learning and reduce drift, while enabling active data sharing through a loopy update mechanism.

Experimental results

Research questions

• RQ1 How can ensemble diversity be systematically increased in online discriminative tracking to reduce model correlation and drift?
• RQ2 What role does artificial data generation play in improving tracker robustness when real positive samples are sparse?
• RQ3 Can a query-by-committee strategy effectively guide the creation of informative artificial samples that enhance ensemble performance?
• RQ4 How does co-tracking with an auxiliary classifier improve accuracy compared to self-learning or isolated ensemble methods?
• RQ5 To what extent does the proposed method outperform state-of-the-art trackers under diverse visual tracking challenges?

Key findings

• DEDT outperformed all compared trackers on the OTB50 benchmark in 10 out of 12 subcategories, with the highest AUC in the ALL and IV categories.
• On OTB100, DEDT achieved top performance in 10 out of 12 subcategories, surpassing even deep-feature-based trackers like CCOT despite using HOG features.
• The ablation study showed that DEDT-first (fixed auxiliary) and DEDT-short (fully updated auxiliary) performed worse than the proposed loopy, active update scheme, confirming the importance of active query selection.
• DEDT-isolated, which fused results only at the end, underperformed the real-time co-tracking variant, demonstrating the benefit of continuous collaboration.
• The method significantly reduced model drift and label noise impact by breaking the self-learning loop and using diversified training data.
• The use of artificial data increased the number of positive samples without introducing outliers, as samples were drawn from a learned data distribution (multivariate Gaussian).