[Paper Review] A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision

Zhonghan Zhao, Wenhao Chai+7 more|arXiv (Cornell University)|Jul 7, 2023

Sports Analytics and Performance8 citations

TL;DR

This paper provides a comprehensive survey of deep learning in sports, organized into perception, comprehension, and decision, and covers datasets, virtual environments, challenges, and future trends.

ABSTRACT

Deep learning has the potential to revolutionize sports performance, with applications ranging from perception and comprehension to decision. This paper presents a comprehensive survey of deep learning in sports performance, focusing on three main aspects: algorithms, datasets and virtual environments, and challenges. Firstly, we discuss the hierarchical structure of deep learning algorithms in sports performance which includes perception, comprehension and decision while comparing their strengths and weaknesses. Secondly, we list widely used existing datasets in sports and highlight their characteristics and limitations. Finally, we summarize current challenges and point out future trends of deep learning in sports. Our survey provides valuable reference material for researchers interested in deep learning in sports applications.

Motivation & Objective

• Propose a hierarchical framework to organize deep learning tasks in sports into Perception, Comprehension, and Decision.
• Summarize widely used sports datasets and virtual environments, highlighting their characteristics and limitations.
• Identify current challenges in deep learning for sports and outline future research directions.

Proposed method

• Present a hierarchical taxonomy of deep learning tasks in sports: Perception, Comprehension, and Decision.
• Survey perception tasks including player/ball localization, tracking, re-identification, pose estimation, and camera calibration.
• Survey comprehension tasks such as individual/group action recognition, action quality assessment, video summarization, and captioning.
• Survey decision tasks covering match evaluation, play forecasting, game simulation, and motion synthesis in sports.

Experimental results

Research questions

• RQ1 What is the proposed hierarchical structure for organizing deep learning tasks in sports (Perception, Comprehension, Decision)?
• RQ2 What are the major datasets and virtual environments used for sports deep learning, and what are their limitations?
• RQ3 What are the key challenges and future trends in applying deep learning to sports performance analysis?
• RQ4 How do perception outputs feed into higher-level comprehension and decision tasks in sports?

Key findings

• The authors introduce a three-level hierarchy (Perception, Comprehension, Decision) for sports deep learning tasks.
• They summarize a wide range of datasets and virtual environments across multiple sports, noting characteristics and limitations.
• They identify current challenges and outline feasible future research directions in motion, datasets, and methods.
• They discuss leading methods and benchmarks used for perception, comprehension, and decision tasks in sports.