[Paper Review] Human-robot co-manipulation of extended objects: Data-driven models and control from analysis of human-human dyads

Erich Mielke, Eric Townsend+2 more|arXiv (Cornell University)|Jan 3, 2020

Robot Manipulation and Learning4 citations

TL;DR

This paper proposes a data-driven human-robot co-manipulation framework using motion and force data from human-human dyads to predict human intent and enable intuitive, adaptive control. It introduces a recurrent neural network (RNN)-based controller (NNPC) that outperforms a force-based controller (EVIC) in task completion time and accuracy, though users preferred EVIC for its perceived safety and predictability, highlighting a trade-off between performance and human trust.

ABSTRACT

Human teams are able to easily perform collaborative manipulation tasks. However, for a robot and human to simultaneously manipulate an extended object is a difficult task using existing methods from the literature. Our approach in this paper is to use data from human-human dyad experiments to determine motion intent which we use for a physical human-robot co-manipulation task. We first present and analyze data from human-human dyads performing co-manipulation tasks. We show that our human-human dyad data has interesting trends including that interaction forces are non-negligible compared to the force required to accelerate an object and that the beginning of a lateral movement is characterized by distinct torque triggers from the leader of the dyad. We also examine different metrics to quantify performance of different dyads. We also develop a deep neural network based on motion data from human-human trials to predict human intent based on past motion. We then show how force and motion data can be used as a basis for robot control in a human-robot dyad. Finally, we compare the performance of two controllers for human-robot co-manipulation to human-human dyad performance.

Motivation & Objective

• To address the challenge of human-robot co-manipulation of extended, rigid objects where intent ambiguity exists between translation and rotation.
• To develop a control framework that learns from human-human interaction patterns to enable intuitive, adaptive physical human-robot interaction.
• To compare the performance and user perception of a neural network-based controller (NNPC) against a force-based controller (EVIC) and blindfolded human-human dyads.
• To investigate whether motion and force data from human dyads can reliably predict human intent in co-manipulation tasks.
• To evaluate the feasibility of using deep learning on human motion data to enable real-time, responsive robot control in collaborative manipulation.

Proposed method

• Collected motion and force data from 21 human-human dyads performing planar co-manipulation tasks using a long board as a rigid object.
• Analyzed human-human data to identify trends: non-negligible interaction forces, torque triggers at movement onset, and minimum-jerk-like trajectories.
• Trained a recurrent neural network (RNN) on past motion data (position, velocity) to predict future human intent in terms of velocity.
• Implemented a neural network-based controller (NNPC) on a Baxter robot with a mobile base, using the RNN to estimate intent and command robot motion.
• Designed and evaluated Extended Variable Impedance Control (EVIC), a novel force-based controller for 3-DOF planar co-manipulation, improving on standard and bi-manual impedance control.
• Compared NNPC and EVIC against blindfolded human-human dyads in a table-carrying task using completion time, trajectory smoothness, and user preference.

Experimental results

Research questions

• RQ1 Can motion and force data from human-human dyads reliably predict human intent in co-manipulation tasks involving extended objects?
• RQ2 How does a deep learning-based intent predictor (RNN) compare to a force-based controller (EVIC) in performance and user perception?
• RQ3 To what extent do human-human co-manipulation patterns, such as minimum-jerk trajectories and torque triggers, inform effective human-robot control strategies?
• RQ4 Can a data-driven, motion-based controller (NNPC) achieve performance comparable to human-human dyads in co-manipulation tasks?
• RQ5 Why do users prefer a less accurate, force-based controller (EVIC) over a higher-performing, learning-based controller (NNPC), despite the latter's superior metrics?

Key findings

• The RNN-based controller (NNPC) significantly outperformed the EVIC controller in task completion time and trajectory accuracy, demonstrating the effectiveness of learning from human motion data.
• NNPC achieved performance comparable to blindfolded human-human dyads in completion time, indicating that motion-based intent prediction can match human-level collaboration.
• Despite superior performance, users preferred the EVIC controller due to perceived safety, predictability, and reduced confusion, highlighting a trust-performance trade-off.
• Interaction forces between human partners were non-negligible and played a key role in communication, contradicting assumptions that forces should be minimized in co-manipulation.
• The beginning of lateral movements was consistently initiated by torque triggers from the leader, indicating a distinct biomechanical signature of intent.
• Deviation from minimum-jerk trajectories served as a valid metric for evaluating co-manipulation performance, with smoother trajectories indicating better coordination.