[Paper Review] A Probabilistic Representation for Dynamic Movement Primitives

Franziska Meier, Stefan Schaal|arXiv (Cornell University)|Dec 18, 2016

Robot Manipulation and Learning15 references19 citations

TL;DR

This paper proposes a probabilistic formulation of Dynamic Movement Primitives (DMPs) as a linear dynamical system with control inputs, enabling Kalman filtering and smoothing for online inference. The approach naturally integrates sensor feedback through uncertainty-based gains, enabling real-time trajectory modulation and failure detection via likelihood monitoring, with evaluation showing 100% failure detection accuracy on a 220-test-case dataset.

ABSTRACT

Dynamic Movement Primitives have successfully been used to realize imitation learning, trial-and-error learning, reinforce- ment learning, movement recognition and segmentation and control. Because of this they have become a popular represen- tation for motor primitives. In this work, we showcase how DMPs can be reformulated as a probabilistic linear dynamical system with control inputs. Through this probabilistic repre- sentation of DMPs, algorithms such as Kalman filtering and smoothing are directly applicable to perform inference on pro- prioceptive sensor measurements during execution. We show that inference in this probabilistic model automatically leads to a feedback term to online modulate the execution of a DMP. Furthermore, we show how inference allows us to measure the likelihood that we are successfully executing a given motion primitive. In this context, we show initial results of using the probabilistic model to detect execution failures on a simulated movement primitive dataset.

Motivation & Objective

• To develop a probabilistic representation of DMPs that supports uncertainty propagation and online inference.
• To enable online feedback modulation during DMP execution using sensor measurements and Kalman filtering.
• To provide a principled method for detecting execution failures by monitoring the likelihood of observed sensor data under the learned DMP model.
• To integrate sensory memory into DMPs by associating expected sensor trajectories with motor primitives.
• To demonstrate the utility of the probabilistic DMP framework in robust motion execution and failure detection.

Proposed method

• Reformulate DMPs as a probabilistic linear dynamical system with phase-dependent variability in the forcing term.
• Model the DMP dynamics using a state-space representation where position, velocity, and acceleration are hidden states, and the forcing term is a probabilistic function of phase.
• Apply Kalman filtering and smoothing to infer the most likely trajectory given noisy sensor observations, automatically generating a feedback term.
• Use the Kalman gain matrix to scale feedback based on uncertainty, ensuring adaptive modulation of the desired trajectory.
• Represent the DMP's nonlinearity through a weighted combination of basis functions with learned, phase-dependent covariance.
• Perform failure detection by computing the log-likelihood of observed sensor data under the model; classify failure if likelihood drops below twice the minimum training likelihood.

Experimental results

Research questions

• RQ1 Can DMPs be reformulated as a probabilistic linear dynamical system to enable uncertainty-aware inference?
• RQ2 How can Kalman filtering be used to enable online feedback modulation in DMPs without ad hoc coupling terms?
• RQ3 Can the likelihood of observed sensor data under the probabilistic DMP model be used to detect execution failures?
• RQ4 How does the probabilistic DMP framework compare to traditional DMPs in handling disturbances and deviations?
• RQ5 Can the model detect artificial execution blockages with high accuracy using likelihood-based monitoring?

Key findings

• The probabilistic DMP formulation enables online trajectory modulation via Kalman filtering, with feedback naturally scaled by uncertainty.
• The feedback term in the probabilistic DMP is derived from the Kalman innovation, eliminating the need for task-specific gain tuning.
• Failure detection via likelihood monitoring successfully identified all 220 perturbed test cases as failed, with only 2 false positives in non-perturbed executions.
• The minimum log-likelihood of training demonstrations was used as a baseline, and any drop below twice this value triggered failure classification.
• The method achieved 100% detection rate for artificial blockages in a 2D letter-writing dataset with 22 motion primitives and 10 demonstrations per primitive.
• The probabilistic model preserves the original DMP dynamics while adding robustness through principled uncertainty handling and sensory feedback integration.