[Paper Review] Large Margin Object Tracking with Circulant Feature Maps

Mengmeng Wang, Yong Liu+1 more|arXiv (Cornell University)|Mar 15, 2017

Video Surveillance and Tracking Methods28 references90 citations

TL;DR

Introduces LMCF, a fast large-margin tracking method that combines structured output SVM with circulant feature maps and correlation filters, plus multimodal detection and high-confidence updates for real-time performance. It also extends to DeepLMCF using CNN features.

ABSTRACT

Structured output support vector machine (SVM) based tracking algorithms have shown favorable performance recently. Nonetheless, the time-consuming candidate sampling and complex optimization limit their real-time applications. In this paper, we propose a novel large margin object tracking method which absorbs the strong discriminative ability from structured output SVM and speeds up by the correlation filter algorithm significantly. Secondly, a multimodal target detection technique is proposed to improve the target localization precision and prevent model drift introduced by similar objects or background noise. Thirdly, we exploit the feedback from high-confidence tracking results to avoid the model corruption problem. We implement two versions of the proposed tracker with the representations from both conventional hand-crafted and deep convolution neural networks (CNNs) based features to validate the strong compatibility of the algorithm. The experimental results demonstrate that the proposed tracker performs superiorly against several state-of-the-art algorithms on the challenging benchmark sequences while runs at speed in excess of 80 frames per second. The source code and experimental results will be made publicly available.

Motivation & Objective

• Advance real-time visual tracking by blending structured output SVM with correlation-filter speedups.
• Improve target localization accuracy and reduce model drift via multimodal detection.
• Prevent model corruption through high-confidence online updates.
• Demonstrate compatibility with both hand-crafted features and deep CNN features (DeepLMCF).
• Validate performance on standard benchmarks against state-of-the-art trackers.

Proposed method

• Formulates tracking-by-detection as a large-margin structured SVM using densely sampled cyclic shifts of the target patch.
• Bridges structured SVM with correlation filters to enable fast online optimization via FFT/DFT operations.
• Proposes a multimodal target detection to handle multiple response peaks and prevent false localization.
• Implements a high-confidence update strategy using F_max and APCE to decide when to update the model.
• Extends to nonlinear kernels via kernel trick, with alpha-based solution for the nonlinear case.
• Provides a DeepLMCF variant that uses CNN features from VGG-19 layers with hierarchical weighting.

Experimental results

Research questions

• RQ1 Can a large-margin structured SVM be efficiently optimized online for dense, circulant samples using correlation-filter techniques?
• RQ2 Does multimodal target detection improve localization accuracy and reduce drift in the presence of distractors?
• RQ3 Can a high-confidence online update strategy prevent model corruption and maintain real-time performance?
• RQ4 How do hand-crafted features compare to deep CNN features in the proposed framework, and can CNN features be integrated effectively?
• RQ5 What are the performance gains on standard benchmarks (OTB-13/OTB-15) relative to state-of-the-art trackers?

Key findings

• LMCF achieves state-of-the-art performance on OTB-13 and OTB-15 in OPE, TRE, and SRE metrics among conventional-feature trackers.
• DeepLMCF with CNN features further improves precision and success while maintaining real-time or near-real-time speeds (DeepLMCF ~8.11 FPS on one metric, faster than several CNN-based trackers).
• The multimodal detection approach reduces false detections from distractors and improves localization accuracy.
• The high-confidence update strategy helps prevent model corruption during occlusion or miss-detections, preserving tracking robustness.
• LMCF runs at over 80 FPS with conventional features, demonstrating real-time applicability; DeepLMCF operates at around 10+ FPS depending on hardware.