[Paper Review] A Framework for Robot Programming in Cobotic Environments: First user experiments

Ying Liang, Damien Pellier+2 more|arXiv (Cornell University)|Oct 19, 2018

Robot Manipulation and Learning17 references3 citations

TL;DR

This paper presents a hybrid framework combining Programming by Demonstration (PbD) and automated planning to enable non-expert users to teach robots atomic actions with preconditions and effects, allowing autonomous generation of goal-directed action sequences. User experiments with a Baxter robot show that novice users intuitively understand action models and successfully teach the robot to solve complex tasks like object permutation.

ABSTRACT

The increasing presence of robots in industries has not gone unnoticed. Large industrial players have incorporated them into their production lines, but smaller companies hesitate due to high initial costs and the lack of programming expertise. In this work we introduce a framework that combines two disciplines, Programming by Demonstration and Automated Planning, to allow users without any programming knowledge to program a robot. The user teaches the robot atomic actions together with their semantic meaning and represents them in terms of preconditions and effects. Using these atomic actions the robot can generate action sequences autonomously to reach any goal given by the user. We evaluated the usability of our framework in terms of user experiments with a Baxter Research Robot and showed that it is well-adapted to users without any programming experience.

Motivation & Objective

• Address the barrier of high programming expertise required for robot deployment in small industrial settings.
• Enable non-expert users to teach robots new skills without writing code or understanding low-level programming.
• Develop a system that learns atomic action models from demonstrations and uses automated planning to generate sequences for arbitrary goals.
• Evaluate the usability and intuitive understanding of preconditions and effects among users with diverse educational backgrounds.

Proposed method

• Users demonstrate atomic actions (e.g., moveObject) to a Baxter robot, specifying start and end states.
• The system extracts action models defined by preconditions (required world state) and effects (resulting world state) from demonstrations.
• These models are encoded in a logical representation compatible with automated planning techniques.
• An automated planner uses the action models to generate sequences that achieve user-specified goals from any initial state.
• The framework supports reusability of atomic actions across different goals and contexts.
• User experiments involve teaching actions, testing generalization in failure scenarios, and evaluating perception of robot intelligence.

Experimental results

Research questions

• RQ1 Can non-expert users intuitively understand and correctly specify preconditions and effects for robot actions using PbD?
• RQ2 Can users successfully teach the robot atomic actions that are later used by a planner to achieve new, unseen goals?
• RQ3 How do users perceive the robot’s ability to autonomously generate action sequences from learned action models?
• RQ4 What difficulties do users encounter when formulating complete action models without failure scenarios?

Key findings

• All participants successfully understood and validated the action models, including preconditions and effects, despite no prior programming experience.
• Users found the PbD process highly intuitive and reported it as easy to learn and use.
• No user encountered difficulties during the experiment, and all completed all phases successfully.
• While users initially missed missing preconditions (e.g., 'zone A is empty'), they recognized them quickly when confronted with failure scenarios.
• Users struggled to formulate precise preconditions (e.g., 'zone A is empty') and instead proposed equivalent but less formal conditions.
• All users perceived the robot’s autonomous solution of the permutation problem as intelligent, even though they had no background in automated planning.