[Paper Review] Framing Human-Robot Task Communication as a POMDP

Mark P. Woodward, Robert J. Wood|arXiv (Cornell University)|Apr 1, 2012

Robot Manipulation and Learning22 references3 citations

TL;DR

This paper proposes modeling human-robot task communication as a Partially Observable Markov Decision Process (POMDP), where the robot infers unobservable task details and human intentions from unstructured signals, using actions that reduce uncertainty. The approach enables robust, intelligent task learning even with teacher errors, as validated in a user experiment with a virtual robot and binary approval feedback.

ABSTRACT

As general purpose robots become more capable, pre-programming of all tasks at the factory will become less practical. We would like for non-technical human owners to be able to communicate, through interaction with their robot, the details of a new task; we call this interaction "task communication". During task communication the robot must infer the details of the task from unstructured human signals and it must choose actions that facilitate this inference. In this paper we propose the use of a partially observable Markov decision process (POMDP) for representing the task communication problem; with the unobservable task details and unobservable intentions of the human teacher captured in the state, with all signals from the human represented as observations, and with the cost function chosen to penalize uncertainty. We work through an example representation of task communication as a POMDP, and present results from a user experiment on an interactive virtual robot, compared with a human controlled virtual robot, for a task involving a single object movement and binary approval input from the teacher. The results suggest that the proposed POMDP representation produces robots that are robust to teacher error, that can accurately infer task details, and that are perceived to be intelligent.

Motivation & Objective

• To develop a robust, general-purpose framework for human-robot task communication that does not rely on pre-programmed task execution.
• To enable non-technical users to teach robots new tasks through natural, unstructured interactions.
• To model task communication as a POMDP, treating task details and human intentions as hidden states and human signals as observations.
• To design robot actions that actively reduce uncertainty in task inference, rather than passively awaiting input.
• To evaluate the approach empirically using a user study comparing POMDP-based and human-controlled virtual robots.

Proposed method

• Represent task communication as a POMDP with hidden random variables for unobservable task details and human intentions.
• Model human signals (e.g., spacebar presses) as observations that update the robot’s belief over task states.
• Define a cost function that penalizes uncertainty in task inference, guiding action selection toward information gain.
• Use belief updates via Bayesian inference to maintain a probability distribution over possible tasks over time.
• Employ a POMDP solver to select actions that minimize expected future uncertainty, based on current belief and observation models.
• Integrate models of human behavior to anticipate teacher responses and optimize communication efficiency.

Experimental results

Research questions

• RQ1 Can a POMDP framework effectively model the uncertainty in human-robot task communication where task details and human intentions are hidden?
• RQ2 How do POMDP-based robots compare to human-controlled robots in terms of task inference accuracy and robustness to teacher error?
• RQ3 Can robot actions that reduce uncertainty lead to more efficient and perceived-intelligent task communication?
• RQ4 How does the POMDP representation support transitions between task communication and execution phases?
• RQ5 To what extent can the POMDP model be extended to handle complex tasks and richer human signals?

Key findings

• The POMDP-based robot demonstrated robustness to teacher errors, maintaining accurate task inference even with inconsistent feedback.
• The robot achieved high accuracy in inferring task details, outperforming a human-controlled baseline in the user experiment.
• Participants perceived the POMDP-based robot as more intelligent and capable than the human-controlled alternative.
• The robot’s action selection strategy effectively reduced uncertainty by choosing informative actions, such as testing specific task parameters.
• The framework naturally supports future extensions to complex tasks, richer signals (e.g., gaze, gestures), and multi-agent modeling via IPOMDP.
• The results suggest that active, uncertainty-reducing communication is more effective than passive task execution during teaching.