[Paper Review] Action recognition by learning pose representations

Alessia Saggese, Nicola Strisciuglio+2 more|arXiv (Cornell University)|Aug 2, 2017

Human Pose and Action Recognition24 references3 citations

TL;DR

This paper proposes a trainable pose detector that learns spatial joint arrangements from prototype skeletons to recognize human actions using skeleton data. By automatically configuring detectors on sample poses and measuring similarity via joint position scores, the method achieves a 64% recognition rate on the MSRDA dataset, demonstrating effectiveness comparable to state-of-the-art approaches.

ABSTRACT

Pose detection is one of the fundamental steps for the recognition of human actions. In this paper we propose a novel trainable detector for recognizing human poses based on the analysis of the skeleton. The main idea is that a skeleton pose can be described by the spatial arrangements of its joints. Starting from this consideration, we propose a trainable pose detector, that can be configured on a prototype skeleton in an automatic configuration process. The result of the configuration is a model of the position of the joints in the concerned skeleton. In the application phase, the joint positions contained in the model are compared with the ones of their homologous joints in the skeleton under test. The similarity of two skeletons is computed as a combination of the position scores achieved by homologous joints. In this paper we describe an action classification method based on the use of the proposed trainable detectors to extract features from the skeletons. We performed experiments on the publicly available MSDRA data set and the achieved results confirm the effectiveness of the proposed approach.

Motivation & Objective

• To develop a trainable pose detector that learns from prototype skeleton poses without manual feature engineering.
• To improve action recognition by modeling the spatial arrangement of joints as a learned representation.
• To enable automatic configuration of pose detectors through an optimization process on training samples.
• To reduce reliance on hand-crafted features by learning discriminative representations directly from data.
• To provide a robust, trainable alternative to fixed feature detectors in skeleton-based action recognition.

Proposed method

• The method uses a trainable detector that learns a spatial configuration of joints from a single prototype skeleton pose.
• The detector computes similarity between a test skeleton and the prototype by evaluating position scores of homologous joints with tolerance for spatial deviation.
• The overall similarity between two skeletons is computed as a weighted combination of individual joint position scores.
• The detector configuration is performed via an automatic optimization process that tunes the detector parameters on a set of prototype poses.
• The approach treats the detector as a trainable feature, similar to COSFIRE filters, but adapted for skeleton-based pose representation.
• The method is integrated into a classification pipeline where pose features are extracted per frame and used for action recognition.

Experimental results

Research questions

• RQ1 Can a trainable detector learn effective spatial representations of human poses from a single prototype sample?
• RQ2 How does the performance of a learned pose detector compare to hand-crafted or fixed-feature methods in action recognition?
• RQ3 To what extent does the automatic configuration of detectors reduce the need for expert-designed features?
• RQ4 How robust is the method to noise and distortions in joint positions during pose matching?
• RQ5 Can the learned pose representations generalize well across different actions when combined with temporal modeling?

Key findings

• The proposed method achieved a recognition rate of 64% on the MSRDA dataset, which is comparable to state-of-the-art methods based on skeleton analysis.
• The method outperforms several existing approaches, including TSD, 3DTSD, and Eigen Joints, which achieved 58% or less.
• The highest-performing baseline method, Joint Position, achieved 68%, indicating the proposed approach is highly competitive.
• The trainable nature of the detector eliminates the need for manual feature engineering, reducing dependency on domain expertise.
• The method shows robustness to noise and distortion due to the tolerance in joint position comparisons.
• The approach is scalable and can be extended with temporal modeling and feature selection to improve efficiency and accuracy.