[Paper Review] Learning Periodic Tasks from Human Demonstrations

Jingyun Yang, Ruoyu Zhang+4 more|arXiv (Cornell University)|Sep 28, 2021

Robot Manipulation and Learning4 citations

TL;DR

This paper proposes ViPTL, a method that learns periodic robot manipulation skills from single human video demonstrations without markers or annotations. By leveraging rhythmic DMPs, unsupervised keypoint tracking, and Bayesian optimization, ViPTL achieves high imitation performance on tasks involving deformable and granular objects—such as wiping, winding, and stirring—within 50 robot trials.

ABSTRACT

We develop a method for learning periodic tasks from visual demonstrations. The core idea is to leverage periodicity in the policy structure to model periodic aspects of the tasks. We use active learning to optimize parameters of rhythmic dynamic movement primitives (rDMPs) and propose an objective to maximize the similarity between the motion of objects manipulated by the robot and the desired motion in human video demonstrations. We consider tasks with deformable objects and granular matter whose states are challenging to represent and track: wiping surfaces with a cloth, winding cables/wires, and stirring granular matter with a spoon. Our method does not require tracking markers or manual annotations. The initial training data consists of 10-minute videos of random unpaired interactions with objects by the robot and human. We use these for unsupervised learning of a keypoint model to get task-agnostic visual correspondences. Then, we use Bayesian optimization to optimize rDMPs from a single human video demonstration within few robot trials. We present simulation and hardware experiments to validate our approach.

Motivation & Objective

• To enable sample-efficient learning of periodic manipulation tasks from unstructured human video demonstrations.
• To address the challenge of representing and tracking deformable and granular object states without manual annotations or markers.
• To exploit periodicity in human demonstrations to reduce the complexity of learning long-horizon, repetitive tasks.
• To develop a perception and optimization framework that generalizes across diverse periodic tasks involving non-rigid materials.
• To achieve high imitation performance with minimal human intervention and few robot trials

Proposed method

• Uses rhythmic Dynamic Movement Primitives (rDMPs) to model periodic motion with shift parameters for temporal adaptation.
• Employs unsupervised keypoint detection from task-agnostic robot play data to establish visual correspondences across human and robot motions.
• Applies periodicity estimation to segment human demonstrations into single-period components for rDMP training.
• Uses Bayesian optimization (BO) with a similarity objective based on keypoint distances to optimize rDMP parameters.
• Generates 'imagined' trajectories from robot play data to provide warm-start candidates for BO, improving sample efficiency.
• Computes motion similarity using a keypoint-based distance metric between robot executions and human demo frames

Figure 1 : Examples of everyday periodic tasks. Our robot learns to imitate wiping, stirring and winding from human video demonstrations (top row).

Experimental results

Research questions

• RQ1 Can a robot learn complex periodic manipulation tasks involving deformable and granular materials from a single unstructured human video demonstration?
• RQ2 How effective is an unsupervised keypoint model in enabling motion similarity computation without ground-truth annotations?
• RQ3 To what extent does leveraging periodicity in the policy structure improve sample efficiency in imitation learning?
• RQ4 Can Bayesian optimization with keypoint-based scoring outperform model-free or dynamics-based baselines in imitation of periodic tasks?
• RQ5 How well does the method generalize across diverse tasks such as wiping, winding, and stirring under real-world stochastic dynamics?

Key findings

• ViPTL successfully learns table wiping, rope winding, and food stirring tasks from a single human video demonstration using only 50 robot trials.
• The method outperforms direct imitation and model-based imitation learning baselines in all three tasks, especially in handling deformable and granular materials.
• The keypoint-based cost function significantly improves performance compared to a classifier-based metric, which showed limited accuracy in all tasks.
• Bayesian optimization with imagined trajectory warm-starts enables rapid convergence to high-scoring rDMP parameters, achieving consistent improvement over trials.
• In real-world experiments, ViPTL maintained strong performance despite increased stochasticity, while the Direct Imitation baseline underperformed due to sensitivity to real-world dynamics.
• Ablation studies confirm that both the perception (keypoint model) and optimization (BO with warm-start) modules are essential for high performance

Figure 2 : Overview of the proposed approach. Our method is composed of two modules: the unsupervised visual representation module and the parameter optimization module. The unsupervised module learns a model for keypoint correspondences between the motions of the objects in the human demonstration