[Paper Review] Indoor Activity Detection and Recognition for Sport Games Analysis

Georg Waltner, Thomas Mauthner+1 more|arXiv (Cornell University)|Apr 25, 2014

Human Pose and Action Recognition14 references3 citations

TL;DR

This paper proposes a context-aware approach for single-player activity recognition in indoor sports, specifically volleyball, by integrating spatio-temporal context via an Activity Context (AC) descriptor that aggregates player activity probabilities over time and space. The method improves average player-centered classification accuracy from 77.56% to 90.19% on volleyball-specific actions by incorporating contextual information from neighboring players over a 1.6-second window.

ABSTRACT

Activity recognition in sport is an attractive field for computer vision research. Game, player and team analysis are of great interest and research topics within this field emerge with the goal of automated analysis. The very specific underlying rules of sports can be used as prior knowledge for the recognition task and present a constrained environment for evaluation. This paper describes recognition of single player activities in sport with special emphasis on volleyball. Starting from a per-frame player-centered activity recognition, we incorporate geometry and contextual information via an activity context descriptor that collects information about all player's activities over a certain timespan relative to the investigated player. The benefit of this context information on single player activity recognition is evaluated on our new real-life dataset presenting a total amount of almost 36k annotated frames containing 7 activity classes within 6 videos of professional volleyball games. Our incorporation of the contextual information improves the average player-centered classification performance of 77.56% by up to 18.35% on specific classes, proving that spatio-temporal context is an important clue for activity recognition.

Motivation & Objective

• To improve single-player activity recognition in indoor sports using contextual cues from teammates.
• To develop a novel Activity Context (AC) descriptor that models spatio-temporal relationships among players.
• To evaluate the impact of contextual information on activity recognition performance in professional volleyball videos.
• To provide a new real-life dataset of 36k annotated frames across 6 professional volleyball games.
• To demonstrate that team-level context enhances recognition of sport-specific actions like 'Block' and 'Service'.

Proposed method

• The method starts with per-frame feature extraction using HOG and HOF for appearance and motion, combined with Real World Player Coordinates (RWPC) and Spatial Context (SC) descriptors.
• Player activities are classified using a Support Vector Machine (SVM) trained on these features.
• An Activity Context (AC) descriptor is introduced to aggregate activity probabilities from all players over a temporal window of k frames.
• The AC descriptor uses spatial binning on the court plane to model positional and activity context relative to the target player.
• The method evaluates over 750 descriptor and SVM parameter combinations to optimize performance.
• The AC descriptor is applied over varying time spans (k = 10 to 100 frames) to determine optimal context length.

Experimental results

Research questions

• RQ1 How does incorporating spatio-temporal context from teammates improve single-player activity recognition in volleyball?
• RQ2 What is the optimal temporal window length for context aggregation in activity recognition?
• RQ3 To what extent does the AC descriptor reduce confusion between similar classes like 'Reception' and 'Defense/Move'?
• RQ4 How do different feature combinations (HOG, HOF, RWPC, SC) affect recognition accuracy?
• RQ5 Can context-based modeling improve recognition of sport-specific actions such as 'Block' and 'Service'?

Key findings

• The integration of the Activity Context (AC) descriptor improves average classification accuracy on volleyball-specific actions by 7.20% compared to baseline single-player recognition.
• The highest overall accuracy of 90.19% is achieved with a 40-frame (1.6-second) context window, outperforming the baseline of 77.56%.
• The 'Block' class achieves the highest recognition rate of 97.60% with the AC descriptor, while 'Service' reaches 97.13%.
• The 'Reception' class sees the largest improvement of 18.35%, rising from 73.27% to 91.62% with context, resolving confusion with 'Defense/Move'.
• The two general classes 'Stand' and 'Defense/Move' show reduced accuracy with AC, as they occur randomly and lack consistent temporal context.
• The RBF and polynomial SVM kernels outperform linear and sigmoidal kernels, with polynomial showing the best overall performance.