[Paper Review] COMET: Context-Aware IoU-Guided Network for Small Object Tracking

Seyed Mojtaba Marvasti-Zadeh, Javad Khaghani+3 more|arXiv (Cornell University)|Jun 4, 2020

Video Surveillance and Tracking Methods58 references4 citations

TL;DR

COMET proposes a context-aware, IoU-guided two-stream tracker for small object tracking in aerial videos from medium to high altitudes. It uses an offline proposal generation strategy to enhance context learning and a multitask network with multi-scale features and attention modules to improve robustness to occlusion and viewpoint changes, achieving a 6.2% average precision and 7% success score gain over ATOM on UAVDT, VisDrnde-2019, and Small-90 benchmarks.

ABSTRACT

We consider the problem of tracking an unknown small target from aerial videos of medium to high altitudes. This is a challenging problem, which is even more pronounced in unavoidable scenarios of drastic camera motion and high density. To address this problem, we introduce a context-aware IoU-guided tracker (COMET) that exploits a multitask two-stream network and an offline reference proposal generation strategy. The proposed network fully exploits target-related information by multi-scale feature learning and attention modules. The proposed strategy introduces an efficient sampling strategy to generalize the network on the target and its parts without imposing extra computational complexity during online tracking. These strategies contribute considerably in handling significant occlusions and viewpoint changes. Empirically, COMET outperforms the state-of-the-arts in a range of aerial view datasets that focusing on tracking small objects. Specifically, COMET outperforms the celebrated ATOM tracker by an average margin of 6.2% (and 7%) in precision (and success) score on challenging benchmarks of UAVDT, VisDrone-2019, and Small-90.

Motivation & Objective

• Address the challenge of tracking small objects in high-altitude aerial videos where targets occupy few pixels and are prone to occlusion and viewpoint changes.
• Overcome limitations of existing trackers that fail to handle small object tracking due to insufficient target representation and lack of context modeling.
• Improve robustness to large occlusions, viewpoint changes, and background clutter in dense, wide-view aerial scenes.
• Develop an efficient online inference mechanism by offloading complex context learning to offline training without increasing online computation.
• Enhance feature representation through multi-scale feature aggregation and attention mechanisms focused on salient target parts.

Proposed method

• Introduces an offline reference proposal generation strategy that extracts high-quality proposals from the reference frame to enrich context and target part representations during training.
• Employs a two-stream multitask network that processes appearance and motion features separately before fusing them for improved localization.
• Applies multi-scale feature aggregation to capture spatial and semantic representations across different receptive fields.
• Integrates lightweight spatial and channel attention modules to emphasize relevant features and suppress background distractors.
• Optimizes a multitask loss function combining intersection-over-union (IoU) and normalized center location error (CLE) to improve bounding box regression accuracy and robustness.
• Uses the predicted IoU and CLE to refine bounding box predictions in real time, enabling dynamic adaptation to motion and appearance changes.

Experimental results

Research questions

• RQ1 Can offline proposal generation improve small object tracking performance without increasing online inference cost?
• RQ2 How effective is a multitask two-stream network with attention and multi-scale features in handling occlusion and viewpoint changes in aerial tracking?
• RQ3 To what extent does IoU-guided regression improve localization accuracy for tiny targets in high-altitude videos?
• RQ4 How does COMET perform across diverse attributes such as occlusion, aspect ratio change, and illumination variation compared to state-of-the-art trackers?
• RQ5 Can context-aware learning from target parts and surrounding regions significantly enhance generalization in complex aerial tracking scenarios?

Key findings

• COMET achieves a 6.2% average precision gain and 7% success score gain over the ATOM tracker on the UAVDT, VisDrone-2019, and Small-90 benchmarks.
• On the VisDrone-2019 dataset, COMET achieves an AUC score of 64.5, outperforming ATOM (57.1) and DiMP-50 (60.8) across all attributes.
• COMET improves by up to 9.5% on the small object attribute and 4.4% on the viewpoint change attribute compared to DiMP-50.
• For occlusion-related attributes (FOC, POC, LO), COMET achieves a 12.1% improvement over SiamMask, demonstrating strong robustness to partial and full occlusions.
• In the long-term tracking attribute, COMET outperforms ATOM by 4.3%, indicating better re-identification and tracking persistence after long out-of-view events.
• Qualitative results show COMET maintains tracking accuracy during significant aspect ratio changes and complex background clutter, outperforming SiamRPN++ and SiamMask in challenging sequences.