[Paper Review] Faster and Simpler Siamese Network for Single Object Tracking

Shaokui Jiang, Baile Xu+2 more|arXiv (Cornell University)|May 7, 2021

Video Surveillance and Tracking Methods28 references4 citations

TL;DR

This paper proposes a faster and simpler siamese network for single object tracking that achieves real-time performance on low-end hardware while maintaining high accuracy. By integrating Squeeze-and-Excitation blocks for feature refinement, using a lightweight correlation layer with reduced kernel size, and employing a 1×1 convolution and fully connected layer for regression, the method accelerates training and inference without sacrificing precision, achieving state-of-the-art speed and competitive accuracy on VOT and OTB benchmarks with only 5 training epochs and minimal data.

ABSTRACT

Single object tracking (SOT) is currently one of the most important tasks in computer vision. With the development of the deep network and the release for a series of large scale datasets for single object tracking, siamese networks have been proposed and perform better than most of the traditional methods. However, recent siamese networks get deeper and slower to obtain better performance. Most of these methods could only meet the needs of real-time object tracking in ideal environments. In order to achieve a better balance between efficiency and accuracy, we propose a simpler siamese network for single object tracking, which runs fast in poor hardware configurations while remaining an excellent accuracy. We use a more efficient regression method to compute the location of the tracked object in a shorter time without losing much precision. For improving the accuracy and speeding up the training progress, we introduce the Squeeze-and-excitation (SE) network into the feature extractor. In this paper, we compare the proposed method with some state-of-the-art trackers and analysis their performances. Using our method, a siamese network could be trained with shorter time and less data. The fast processing speed enables combining object tracking with object detection or other tasks in real time.

Motivation & Objective

• To address the trade-off between speed and accuracy in siamese network-based single object trackers, especially under limited hardware resources.
• To reduce model complexity and training time while preserving high tracking accuracy.
• To enable real-time deployment of object trackers on low-end devices, including mobile platforms.
• To simplify the network architecture without compromising performance, using fewer parameters and faster inference.
• To train effectively with minimal data—only 15,000 video clips from ImageNet-VID and GOT—within just 5 epochs.

Proposed method

• Integrates Squeeze-and-Excitation (SE) blocks into the feature extractor to enhance channel-wise feature representation and accelerate training.
• Uses a correlation layer with smaller kernel size to reduce the number of parameters and speed up feature matching between template and search regions.
• Employs a 1×1 convolutional layer followed by a fully connected layer for efficient regression to predict the target bounding box coordinates.
• Applies end-to-end training on ImageNet-VID and GOT datasets, using only 15,000 video clips and 5 training epochs.
• Optimizes for inference speed by minimizing computational complexity in both feature extraction and regression stages.
• Deploys the model on low-end hardware, including a GeForce 840M GPU and i7-4510U CPU, to validate real-time performance.

Experimental results

Research questions

• RQ1 Can a siamese network be simplified to achieve real-time inference without sacrificing tracking accuracy?
• RQ2 How does integrating SE blocks improve training speed and feature representation in a lightweight siamese tracker?
• RQ3 To what extent can training time and data requirements be reduced while maintaining competitive performance?
• RQ4 How does the proposed method compare in speed and accuracy to state-of-the-art trackers on standard benchmarks like VOT and OTB100?
• RQ5 Can a lightweight siamese tracker achieve high performance on low-end hardware without reinitialization in one-pass evaluation?

Key findings

• The proposed tracker achieves 53.05 fps on a low-end device (i7-4510U + GeForce 840M), significantly outperforming other trackers in speed.
• On VOT2016, the tracker achieves an EAO score of 0.19 and robustness of 0.53, outperforming several top trackers in robustness despite lower accuracy.
• On VOT2017, the tracker maintains an EAO of 0.13 and robustness of 0.81, demonstrating strong performance even on more challenging sequences.
• In the one-pass evaluation on OTB100, the tracker achieves competitive success and precision rates, outperforming many traditional and deep learning-based trackers.
• The model size is smaller than most siamese networks, including SiamFC, making it suitable for deployment on mobile and embedded devices.
• The network is trained effectively in only 5 epochs using just 15,000 video clips from ImageNet-VID and GOT, indicating high data efficiency.