[Paper Review] Human Motion Prediction using Semi-adaptable Neural Networks

Yujiao Cheng, Weiye Zhao+2 more|arXiv (Cornell University)|Oct 1, 2018

Human Pose and Action Recognition16 references4 citations

TL;DR

This paper proposes a semi-adaptable neural network for human motion prediction that combines offline training with online recursive least squares parameter adaptation (RLS-PAA) to enable real-time model adaptation to time-varying human behaviors and uncertainty estimation. The method achieves significantly higher prediction accuracy and computational efficiency than state-of-the-art approaches, with mean squared error reductions of up to 75% on benchmark datasets.

ABSTRACT

Human motion prediction is an important component to facilitate human robot interaction. Robots need to accurately predict human's future movement in order to safely plan its own motion trajectories and efficiently collaborate with humans. Many recent approaches predict human's movement using deep learning methods, such as recurrent neural networks. However, existing methods lack the ability to adapt to time-varying human behaviors, and many of them do not quantify uncertainties in the prediction. This paper proposes an approach that uses a semi-adaptable neural network for human motion prediction, and provides uncertainty bounds of the predictions in real time. In particular, a neural network is trained offline to represent the human motion transition model, and then recursive least square parameter adaptation algorithm (RLS-PAA) is adopted for online parameter adaptation of the neural network and for uncertainty estimation. Experiments on several human motion datasets verify that the proposed method significantly outperforms the state-of-the-art approach in terms of prediction accuracy and computation efficiency.

Motivation & Objective

• To address the limitations of existing deep learning methods in human motion prediction, particularly their inability to adapt to time-varying behaviors and quantify prediction uncertainty.
• To develop a computationally efficient method that maintains high accuracy across diverse motion patterns and individual differences.
• To enable real-time online adaptation of neural network parameters for improved prediction fidelity in dynamic human-robot interaction scenarios.
• To provide reliable uncertainty bounds for safe robot motion planning in human-robot collaboration.

Proposed method

• A feedforward neural network is trained offline to learn the global human motion transition model from historical joint trajectory data.
• Online parameter adaptation is performed only on the last layer of the network using the Recursive Least Square Parameter Adaptation Algorithm (RLS-PAA).
• RLS-PAA enables fast, recursive updates of the final layer weights based on real-time prediction errors, allowing rapid adaptation to changing motion dynamics.
• The algorithm incorporates a forgetting factor to control the influence of past data, enabling dynamic adjustment to recent motion patterns.
• Uncertainty in predictions is estimated directly from the RLS-PAA covariance matrix, providing real-time confidence bounds.
• The method is computationally efficient compared to full online fine-tuning, as only the last layer is adapted, avoiding backpropagation through the entire network.

Experimental results

Research questions

• RQ1 Can a semi-adaptable neural network architecture improve prediction accuracy for human motion compared to fixed offline models?
• RQ2 Does online adaptation of only the last layer using RLS-PAA enable faster and more accurate response to sudden changes in human motion compared to full-network online learning?
• RQ3 Can the proposed method reliably estimate uncertainty in human motion predictions for safe robot motion planning?
• RQ4 How does the performance of the RLS-PAA-based method compare to the identifier-based algorithm across diverse motion datasets and motion types?

Key findings

• The proposed method achieved a mean squared estimation error (MSEE) of 5.5728 cm² on the CMU dataset, representing a 74.6% reduction compared to the identifier-based method (21.9248 cm²).
• On the Kinect dataset, the MSEE was reduced from 27.3493 cm² (identifier-based) to 12.0419 cm² (RLS-PAA), a 55.9% improvement.
• In time-variant (TV) and time-invariant (TI) artificial datasets, the RLS-PAA method achieved MSEE values of 2.2261 cm² and 1.3180 cm² respectively, outperforming both identifier-based and non-adaptive baselines.
• The method demonstrated robust performance across multiple motion categories and datasets, with consistent improvements in accuracy and stability.
• The computational efficiency of the method was significantly higher than the identifier-based approach, as it avoids full backpropagation during online adaptation.
• The uncertainty estimates provided by RLS-PAA were shown to be reliable and responsive to changes in motion dynamics, supporting safe robot trajectory planning.