[Paper Review] A Structural Correlation Filter Combined with A Multi-task Gaussian Particle Filter for Visual Tracking

Manna Dai, Shuying Cheng+2 more|arXiv (Cornell University)|Mar 3, 2018

Video Surveillance and Tracking Methods5 references3 citations

TL;DR

This paper proposes KCF-GPF, a novel visual tracking framework that fuses multiple structural correlation filters (KCFs) as weak experts with a multi-task Gaussian particle filter (GPF) to enhance robustness. By leveraging spatial-geometric consistency and reliability-weighted fusion, the method achieves state-of-the-art performance on OTB-2013, outperforming 16 SOTA trackers in precision and success metrics across 50 challenging sequences.

ABSTRACT

In this paper, we propose a novel structural correlation filter combined with a multi-task Gaussian particle filter (KCF-GPF) model for robust visual tracking. We first present an assemble structure where several KCF trackers as weak experts provide a preliminary decision for a Gaussian particle filter to make a final decision. The proposed method is designed to exploit and complement the strength of a KCF and a Gaussian particle filter. Compared with the existing tracking methods based on correlation filters or particle filters, the proposed tracker has several advantages. First, it can detect the tracked target in a large-scale search scope via weak KCF trackers and evaluate the reliability of weak trackers q decisions for a Gaussian particle filter to make a strong decision, and hence it can tackle fast motions, appearance variations, occlusions and re-detections. Second, it can effectively handle large-scale variations via a Gaussian particle filter. Third, it can be amenable to fully parallel implementation using importance sampling without resampling, thereby it is convenient for VLSI implementation and can lower the computational costs. Extensive experiments on the OTB-2013 dataset containing 50 challenging sequences demonstrate that the proposed algorithm performs favourably against 16 state-of-the-art trackers.

Motivation & Objective

• Address limitations of correlation filter-based trackers in handling fast motion, occlusion, scale variation, and appearance changes.
• Overcome the instability of particle filters in high-dimensional state spaces and poor initialization by integrating weak experts.
• Improve tracking robustness through reliability evaluation of weak tracker decisions and multi-task particle filtering.
• Enable efficient, parallelizable implementation via importance sampling without resampling for low computational cost.
• Develop a scalable, reliable search mechanism using spatial-geometric relations between target locations in consecutive frames.

Proposed method

• Deploy multiple KCF trackers as weak experts, each using HOG and grayscale features to generate preliminary target location estimates.
• Use spatial-geometric constraints between consecutive frames to define a large-scale, reliable search region for weak tracker decisions.
• Compute confidence maps for each weak tracker to assess decision reliability, which are used as weights in the Gaussian particle filter (GPF).
• Implement a multi-task GPF that simultaneously evaluates HOG and grayscale features across particles, enabling robust state estimation.
• Apply importance sampling without resampling to maintain computational efficiency and enable VLSI-friendly, fully parallel implementation.
• Fuse weak tracker outputs via weighted combination in the GPF, where particle states are updated based on reliability-weighted confidence maps.

Experimental results

Research questions

• RQ1 Can a multi-expert ensemble of structural correlation filters improve tracking robustness under occlusion and appearance variation?
• RQ2 How can spatial-geometric consistency between target locations in consecutive frames enhance search reliability in large-scale tracking?
• RQ3 Can a reliability-weighted fusion strategy in a Gaussian particle filter outperform standard particle filters in handling fast motion and scale changes?
• RQ4 To what extent does multi-task particle filtering with HOG and grayscale features improve tracking accuracy compared to single-feature approaches?
• RQ5 Can the proposed framework achieve high performance with low computational cost through resampling-free, importance-sampling-based GPF?

Key findings

• KCF-GPF achieves the highest average AUC score of 0.624 in success rate and 0.572 in precision on the OTB-2013 benchmark, outperforming 16 state-of-the-art trackers.
• The method ranks in the top three across all 11 challenging attributes (e.g., occlusion, deformation, fast motion), with the best performance in out-of-plane rotation and deformation.
• Qualitative results show KCF-GPF maintains tracking accuracy under severe occlusion, illumination changes, and fast motion, while baseline trackers like KCF and CFNet frequently drift.
• The reliability-weighted fusion mechanism significantly improves decision robustness, especially during partial occlusions and scale variations.
• The GPF-based multi-task tracking with importance sampling enables efficient, parallel computation and reduces computational cost compared to resampling-based methods.
• The large-scale search strategy based on spatial-geometric consistency enables effective re-detection after occlusion, improving recovery performance.