[Paper Review] Learning Background-Aware Correlation Filters for Visual Tracking

Hamed Kiani Galoogahi, Ashton Fagg+1 more|arXiv (Cornell University)|Mar 14, 2017

Video Surveillance and Tracking MethodsComputer Science28 references70 citations

TL;DR

This paper introduces Background-Aware Correlation Filters (BACF) that learn from real background patches in addition to the target, enabling real-time, accurate visual tracking on hand-crafted features without heavy deep learning. It uses an ADMM-based optimization in the Fourier domain with online updates and Sherman-Morrison shortcuts for efficiency.

ABSTRACT

Correlation Filters (CFs) have recently demonstrated excellent performance in terms of rapidly tracking objects under challenging photometric and geometric variations. The strength of the approach comes from its ability to efficiently learn - "on the fly" - how the object is changing over time. A fundamental drawback to CFs, however, is that the background of the object is not be modelled over time which can result in suboptimal results. In this paper we propose a Background-Aware CF that can model how both the foreground and background of the object varies over time. Our approach, like conventional CFs, is extremely computationally efficient - and extensive experiments over multiple tracking benchmarks demonstrate the superior accuracy and real-time performance of our method compared to the state-of-the-art trackers including those based on a deep learning paradigm.

Motivation & Objective

• Motivate and address the limitation of traditional correlation filters that ignore background information during learning.
• Propose BACF to utilize real background patches as negatives for robust discrimination.
• Maintain real-time performance with an efficient optimization framework in the Fourier domain.
• Evaluate BACF across standard tracking benchmarks against state-of-the-art CF-based and deep trackers.

Proposed method

• Formulate BACF as a multi-channel ridge regression that samples patches from the full frame via a cropping operator.
• Replace circular foreground-only training with a background-inclusive objective that uses P x[k] to crop from large-support training samples.
• Solve the resulting optimization with an ADMM approach in the Fourier domain, including an efficient g-h update and Sherman–Morrison-based inverse updates.
• Apply online adaptation for robustness to appearance changes, and perform multi-scale search for detection.
• Evaluate on OTB50/OTB100/Temple-Color128/VOT2015 using IoU-based metrics and AUC of success plots.

Experimental results

Research questions

• RQ1 Can learning from real background patches improve CF tracker accuracy without sacrificing real-time performance?
• RQ2 How does BACF perform relative to hand-crafted feature CF trackers, deep-feature CF trackers, and fully deep trackers on standard benchmarks?
• RQ3 Does the proposed ADMM-based BACF framework enable efficient Multi-channel learning in practice on CPU?
• RQ4 What is the impact of online updating and scale/search strategy on tracking robustness across common challenges (occlusion, deformation, background clutter)?

Key findings

• BACF achieves higher AUC-based success than several hand-crafted feature trackers on OTB50, OTB100, and TC128.
• BACF attains real-time CPU performance (≈35 FPS) with competitive accuracy versus deep-feature trackers like CCOT, while being orders of magnitude faster.
• On VOT2015, BACF shows competitive accuracy and robustness relative to top trackers, improving accuracy over several baselines.
• Attribute-based analysis shows BACF gains across clutter, deformation, occlusion, and illumination changes by leveraging background patches.
• Compared to deep-feature CF trackers, BACF delivers substantial speed advantages (≈100–170x faster than CCOT) with competitive accuracy.