[Paper Review] Balancing Shared Autonomy with Human-Robot Communication

Rosario Scalise, Yonatan Bisk+4 more|arXiv (Cornell University)|May 20, 2018

Social Robot Interaction and HRI19 references3 citations

TL;DR

This paper investigates how human-robot communication in shared autonomy balances cognitive load and task efficiency during table-clearing tasks. By analyzing natural language instructions on Amazon Mechanical Turk, it shows that users adapt their language from simple to strategic based on robot capabilities and task complexity, with high-level strategies improving plan efficiency but increasing user effort—highlighting a trade-off between helpfulness and usability in human-robot collaboration.

ABSTRACT

Robotic agents that share autonomy with a human should leverage human domain knowledge and account for their preferences when completing a task. This extra knowledge can dramatically improve plan efficiency and user-satisfaction, but these gains are lost if communicating with a robot is taxing and unnatural. In this paper, we show how viewing humanrobot language through the lens of shared autonomy explains the efficiency versus cognitive load trade-offs humans make when deciding how cooperative and explicit to make their instructions.

Motivation & Objective

• To understand how human language in human-robot interaction reflects expectations of robot capabilities and influences task efficiency.
• To investigate the trade-off between user cognitive load and plan efficiency when communicating with a robot in shared autonomy.
• To analyze how task complexity and robot capabilities shape the form and content of human-provided language for robotic task planning.
• To identify and categorize high-level language strategies (e.g., heuristics, constraints) that users naturally produce to assist robots.
• To inform the design of interactive task-planning systems that leverage human insights while minimizing user burden.

Proposed method

• Conducted user studies on Amazon Mechanical Turk with non-expert participants to simulate human-robot interaction in a table-clearing task.
• Used the MAGI task and motion planning library to generate geometrically feasible plans from high-level language inputs.
• Collected and analyzed natural language instructions, categorizing them by form (goal, ordering, constraints, strategies) and complexity.
• Varied task configurations to assess the impact of object count, color uniqueness, and spatial layout on language use and user satisfaction.
• Measured plan efficiency and user effort via time-to-plan and post-task satisfaction surveys.
• Applied linguistic principles (Grice’s Maxims) to interpret user language patterns and assess informativeness and clarity.

Experimental results

Research questions

• RQ1 How do users' language strategies shift in response to robot capabilities and task complexity in shared autonomy?
• RQ2 What is the trade-off between plan efficiency and user cognitive load when users provide high-level language constraints or strategies?
• RQ3 To what extent do users naturally produce effective, non-trivial planning heuristics or constraints that improve robot performance?
• RQ4 How do preferences for referring expressions (e.g., color, position) change with increasing task complexity?
• RQ5 What language forms are most effective in reducing robot planning time while remaining cognitively manageable for users?

Key findings

• Users shift from simple referring expressions to complex, strategy-based language as task complexity increases, indicating adaptation to robot limitations.
• Providing high-level constraints or heuristics significantly improves plan efficiency, but increases user effort and decreases satisfaction when the task is complex.
• Despite increased effort, users consistently produced helpful strategies (e.g., partial orderings, spatial heuristics), suggesting strong willingness to assist.
• Preference for unique colors as referring expressions decreased in complex scenes, with no strong consensus on the easiest configuration, indicating context-dependent usability.
• There was little difference in plan time between the most common plan and alternative user-generated plans, suggesting that users did not typically propose infeasible actions.
• The study reveals a clear tension: users are effective planners but become increasingly frustrated when asked to provide detailed plans, highlighting a usability bottleneck in shared autonomy.