[Paper Review] SoccerNet 2023 Tracking Challenge -- 3rd place MOT4MOT Team Technical Report

Gal Shitrit, Ishay Be’ery+1 more|arXiv (Cornell University)|Aug 31, 2023

Video Analysis and SummarizationComputer Science3 citations

TL;DR

This paper presents the MOT4MOT team's approach to the SoccerNet 2023 Tracking Challenge, achieving 3rd place with a HOTA score of 66.27. The method separates player and ball tracking: player tracking uses a fine-tuned YOLOv8 detector with DeepOC-SORT++ tracker, enhanced by post-processing interpolation (GSI), appearance-free track merging (AFLink), and appearance-based track merging; ball tracking uses a low-confidence YOLOv8l detector with temporal smoothing and filtering, followed by linear interpolation for continuity.

ABSTRACT

The SoccerNet 2023 tracking challenge requires the detection and tracking of soccer players and the ball. In this work, we present our approach to tackle these tasks separately. We employ a state-of-the-art online multi-object tracker and a contemporary object detector for player tracking. To overcome the limitations of our online approach, we incorporate a post-processing stage using interpolation and appearance-free track merging. Additionally, an appearance-based track merging technique is used to handle the termination and creation of tracks far from the image boundaries. Ball tracking is formulated as single object detection, and a fine-tuned YOLOv8l detector with proprietary filtering improves the detection precision. Our method achieves 3rd place on the SoccerNet 2023 tracking challenge with a HOTA score of 66.27.

Motivation & Objective

• To address the challenge of multi-object tracking in fast-paced, dynamic soccer video sequences with high accuracy.
• To improve tracking robustness for both players and the ball, particularly under occlusion, small object size, and appearance variation.
• To optimize detection and tracking performance through domain-specific fine-tuning and post-processing techniques.
• To achieve state-of-the-art performance in the SoccerNet 2023 Tracking Challenge using a hybrid online-offline approach.

Proposed method

• Player tracking is performed using a two-stage approach: first detecting players with a fine-tuned YOLOv8l detector, then tracking with DeepOC-SORT++.
• Post-processing includes Gaussian Process Interpolation (GSI) to recover missing detections and AFLink to merge tracks without appearance cues.
• Appearance-based track merging is applied to reconnect tracks terminated far from image boundaries if a new track appears shortly after with high appearance similarity.
• Ball tracking is treated as single-object detection using a fine-tuned YOLOv8l detector with a low confidence threshold (0.05) to maximize recall.
• Temporal filtering includes 3rd-order polynomial smoothing of the most confident ball detection per frame, followed by selection of detections within 100 pixels of the smoothed trajectory.
• Missing ball detections are recovered via linear interpolation to ensure continuous tracking box sequences.

Experimental results

Research questions

• RQ1 How can online multi-object tracking be improved in soccer videos with high motion, occlusion, and small object size?
• RQ2 What role does post-processing interpolation and track merging play in enhancing tracker robustness when future frames are unavailable?
• RQ3 To what extent does domain-specific fine-tuning of detectors and appearance models improve tracking performance on SoccerNet data?
• RQ4 How does separating ball and player tracking improve overall HOTA compared to joint tracking approaches?

Key findings

• The final tracker achieved a HOTA score of 66.27 on the SoccerNet 2023 test set, securing 3rd place in the challenge.
• Fine-tuning the player detector on SoccerNet data improved its AP@0.5 from 0.954 (on COCO) to 0.990, indicating strong domain adaptation.
• The ablation study showed that the fine-tuned detector contributed most to HOTA improvement, yielding a 5.9-point gain when combined with other components.
• Appearance-based track merging improved HOTA by 0.38% compared to using only AFLink and GSI in post-processing.
• Using ground-truth (GT) boxes instead of detections increased HOTA from 66.00 to 87.85 for DeepOC-SORT++, indicating detector quality is a major bottleneck.
• The appearance model with crop augmentation achieved Rank-1 accuracy of 0.95 and mAP of 0.80, showing strong performance despite limited identities (199) and high similarity among teammates.