[论文解读] Design and Implementation of A Soccer Ball Detection System with Multiple Cameras

Lei Li, Tianfang Zhang+2 more|arXiv (Cornell University)|Jan 31, 2023

Video Analysis and Summarization被引用 4

一句话总结

本文提出了一种使用36台摄像机和深度学习的实时3D足球检测系统，利用基于YOLO的目标检测、多摄像机融合以及束调整技术实现精确的3D定位。尽管面临遮挡、运动和光照变化等挑战，系统仍实现了高精度和实时性能，并成功部署于现场体育转播应用。

ABSTRACT

The detection of small and medium-sized objects in three dimensions has always been a frontier exploration problem. This technology has a very wide application in sports analysis, games, virtual reality, human animation and other fields. The traditional three-dimensional small target detection technology has the disadvantages of high cost, low precision and inconvenience, so it is difficult to apply in practice. With the development of machine learning and deep learning, the technology of computer vision algorithms is becoming more mature. Creating an immersive media experience is considered to be a very important research work in sports. The main work is to explore and solve the problem of football detection under the multiple cameras, aiming at the research and implementation of the live broadcast system of football matches. Using multi cameras detects a target ball and determines its position in three dimension with the occlusion, motion, low illumination of the target object. This paper designed and implemented football detection system under multiple cameras for the detection and capture of targets in real-time matches. The main work mainly consists of three parts, football detector, single camera detection, and multi-cameras detection. The system used bundle adjustment to obtain the three-dimensional position of the target, and the GPU to accelerates data pre-processing and achieve accurate real-time capture of the target. By testing the system, it shows that the system can accurately detect and capture the moving targets in 3D. In addition, the solution in this paper is reusable for large-scale competitions, like basketball and soccer. The system framework can be well transplanted into other similar engineering project systems. It has been put into the market.

研究动机与目标

• 解决在复杂体育环境中对小而快速移动的物体（如足球）进行实时3D检测的挑战。
• 克服传统3D检测系统存在的高成本、低精度和实时性能差等局限性。
• 为足球和篮球等体育项目的现场转播应用开发一种可扩展、可重用的系统。
• 确保在真实世界条件下具备鲁棒性，包括遮挡、运动模糊和光照不均。
• 整合多摄像机视觉与深度学习，利用GPU加速处理实现实时、高精度的3D球体追踪。

提出的方法

• 在单摄像机视角中使用基于YOLO的深度卷积神经网络（YOLOv3）实现实时2D球体检测。
• 应用模型压缩和优化技术，在保持高精度的同时实现实时推理。
• 使用束调整技术对多摄像机的2D检测结果进行三角测量，计算球体的精确3D位置。
• 实现一种自适应视频检测框架，以处理来自36台摄像机的高容量视频流。
• 利用GPU加速实现数据预处理和实时推理，满足低延迟要求。
• 设计模块化系统架构，包含专门处理数据预处理、单摄像机检测和多摄像机融合的模块。

Figure 1: Convolutional neural network extracted features

实验结果

研究问题

• RQ1 如何利用多台摄像机和深度学习设计一种实时、高精度的3D足球检测系统？
• RQ2 哪些技术能有效应对动态体育场景中遮挡、运动模糊和光照变化的问题？
• RQ3 如何将多摄像机的2D检测结果融合，实现实时、高精度的小目标（球体）3D定位？
• RQ4 在大规模摄像机部署下，为实现实时推理并保持精度，需要多大程度的模型压缩与优化？
• RQ5 所提出的系统在其他体育项目或追踪应用中的可重用性和可扩展性达到何种程度？

主要发现

• 系统在遮挡和运动等挑战性条件下，成功实现了对足球的高精度3D检测与追踪。
• 模型压缩与优化技术使系统在保持检测精度的同时实现实时推理，这对现场转播应用至关重要。
• 束调整技术有效融合了多摄像机的2D检测结果，实现了即使目标尺寸极小（约图像面积的~2×10⁻⁵）的精确3D球体定位。
• 系统在各种光照和运动条件下表现出稳健性能，验证了其在真实世界中的适用性。
• 由于其模块化和可扩展的设计，该框架可重用并适用于其他体育项目，如篮球。
• 该系统已实现商业化部署，证实了其在真实现场体育环境中的实用性与实时性能。

Figure 2: Convolutional neural network diagram.