[Paper Review] Real-Time Visual Tracking: Promoting the Robustness of Correlation Filter Learning

Yao Sui, Ziming Zhang+3 more|arXiv (Cornell University)|Aug 29, 2016

Video Surveillance and Tracking Methods37 references4 citations

TL;DR

This paper proposes three real-time visual trackers that enhance correlation filter robustness by introducing sparsity-inducing loss functions—ℓ₁, ℓ₁ℓ₂, and ℓ₂,₁—instead of the standard ℓ₂-loss. By exploiting response anisotropy and reducing overfitting to appearance changes, the method improves tracking performance under occlusion, illumination variation, and noise, with peak response sensitivity in successive frames serving as a reliable performance predictor.

ABSTRACT

Correlation filtering based tracking model has received lots of attention and achieved great success in real-time tracking, however, the lost function in current correlation filtering paradigm could not reliably response to the appearance changes caused by occlusion and illumination variations. This study intends to promote the robustness of the correlation filter learning. By exploiting the anisotropy of the filter response, three sparsity related loss functions are proposed to alleviate the overfitting issue of previous methods and improve the overall tracking performance. As a result, three real-time trackers are implemented. Extensive experiments in various challenging situations demonstrate that the robustness of the learned correlation filter has been greatly improved via the designed loss functions. In addition, the study reveals, from an experimental perspective, how different loss functions essentially influence the tracking performance. An important conclusion is that the sensitivity of the peak values of the filter in successive frames is consistent with the tracking performance. This is a useful reference criterion in designing a robust correlation filter for visual tracking.

Motivation & Objective

• Address the overfitting issue in correlation filter learning caused by appearance variations such as occlusion and illumination changes.
• Improve the robustness of real-time visual tracking by replacing the standard ℓ₂-loss with sparsity-promoting loss functions.
• Investigate how different loss functions influence tracking performance through empirical analysis.
• Establish a new criterion—sensitivity of filter peak values across frames—for evaluating and designing robust correlation filters.

Proposed method

• Introduce three sparsity-related loss functions: ℓ₁, ℓ₁ℓ₂, and ℓ₂,₁, to replace the standard ℓ₂-loss in correlation filter learning.
• Leverage the anisotropy of filter responses by assigning different response values to spatially distinct cyclic shift samples, avoiding uniform response enforcement.
• Formulate the learning problem in the frequency domain using FFT to maintain computational efficiency while applying robust loss functions.
• Define a sensitivity metric s = Σ(pᵢ − pₘ)² to quantify peak value fluctuations across frames, linking it to tracking robustness.
• Use the cyclic shift method to generate training samples from the target region, preserving circulant structure for fast computation.
• Optimize the filter using robust loss functions that tolerate large errors during appearance changes, reducing overfitting.

Experimental results

Research questions

• RQ1 How do different loss functions affect the robustness and performance of correlation filter-based trackers?
• RQ2 Can sparsity-inducing loss functions reduce overfitting to appearance variations such as occlusion and illumination changes?
• RQ3 Is the sensitivity of the filter’s peak response across consecutive frames a reliable predictor of tracking performance?
• RQ4 How does the proposed method compare to the KCF tracker in noisy or corrupted video frames?
• RQ5 To what extent do robust loss functions improve tracking stability under challenging visual conditions?

Key findings

• The proposed trackers using ℓ₁, ℓ₁ℓ₂, and ℓ₂,₁ loss functions significantly outperform the KCF tracker on the standard benchmark, especially under occlusion and illumination variation.
• The sensitivity metric s of the filter peak values across frames is strongly correlated with tracking performance: lower s values indicate better robustness and higher accuracy.
• The KCF tracker, relying on ℓ₂-loss, exhibits high sensitivity (large s values), leading to instability under appearance changes, while the proposed methods show lower sensitivity and improved stability.
• In noise-contaminated frames, the proposed trackers maintain performance until ~20% of pixels are corrupted, whereas KCF degrades sharply even with 5% corruption.
• The experimental analysis confirms that robust loss functions reduce overfitting by allowing larger errors in response fitting, especially during significant appearance changes.
• The study establishes that filter peak response sensitivity across frames is a valid and useful criterion for designing robust correlation filters in visual tracking.