[Paper Review] MAD-TN: A Tool for Measuring Fluency in Human-Robot Collaboration

Seth Isaacson, Gretchen Rice+1 more|arXiv (Cornell University)|Sep 14, 2019

Constraint Satisfaction and Optimization7 references4 citations

TL;DR

This paper introduces MAD-TN, a Multi-Agent Daisy Temporal Network that models human-robot collaboration using temporal constraint networks to measure fluency. It formalizes actions and petals to represent task subcomponents and agent responsibilities, enabling precise measurement of fluency metrics like functional delay, resource delay, and concurrent inactivity, with strong hypothesized correlations to human perception of fluency.

ABSTRACT

Fluency is an important metric in Human-Robot Interaction (HRI) that describes the coordination with which humans and robots collaborate on a task. Fluency is inherently linked to the timing of the task, making temporal constraint networks a promising way to model and measure fluency. We show that the Multi-Agent Daisy Temporal Network (MAD-TN) formulation, which expands on an existing concept of daisy-structured networks, is both an effective model of human-robot collaboration and a natural way to measure a number of existing fluency metrics. The MAD-TN model highlights new metrics that we hypothesize will strongly correlate with human teammates' perception of fluency.

Motivation & Objective

• To develop a formal, scalable framework for modeling and measuring fluency in human-robot collaboration.
• To extend the daisy-structured temporal network model into a more precise, generalizable Multi-Agent Daisy Temporal Network (MAD-TN) using temporal network formalism.
• To identify and formalize new fluency metrics—resource delay and concurrent inactivity—that may better predict human perception of team fluency.
• To validate existing fluency metrics (e.g., functional delay, idle time) within a structured temporal modeling framework.
• To provide a foundation for future empirical validation of fluency metrics through physical human-robot experiments.

Proposed method

• The MAD-TN models collaborative tasks as a network of petals, where each petal represents a sequence of actions assigned to a single agent.
• Each action is defined by a start and end timepoint with a makespan constraint (e.g., [0.5, 3] seconds) to ensure non-negative duration.
• Constraints between actions within a petal enforce temporal order, while inter-petal constraints model dependencies across agents.
• The model uses disjunctive temporal constraints to represent alternative execution paths, enhancing flexibility in scheduling.
• Functional delay is computed as the time between the end of a preceding action and the start of a subsequent one when the agent arrives late.
• Resource delay is defined as the time between when a resource becomes available and when the agent begins acting on it, capturing staleness effects.

Experimental results

Research questions

• RQ1 How can a temporal network model be extended to better represent multi-agent human-robot collaboration with precise timing constraints?
• RQ2 To what extent do resource delay and concurrent inactivity correlate with human perception of fluency in human-robot teams?
• RQ3 How do existing fluency metrics—such as functional delay and idle time—behave within a formalized temporal network framework?
• RQ4 Can the MAD-TN model support the measurement of both existing and novel fluency metrics in a unified, scalable way?
• RQ5 How might the structure of petals and disjunctive constraints improve scheduling flexibility and fluency in complex tasks?

Key findings

• The MAD-TN model successfully formalizes daisy-structured collaboration in the language of temporal networks, enabling precise modeling of agent-specific actions and inter-agent dependencies.
• Functional delay was found to correlate strongly with human perception of fluency in prior studies, and the MAD-TN provides a formal mechanism to measure it.
• Resource delay is introduced as a new metric that captures the impact of stale resources on perceived fluency, particularly when agents arrive late to a shared task.
• Concurrent inactivity is proposed as a novel metric reflecting periods when both agents are idle simultaneously, potentially affecting team fluency perception.
• The model supports disjunctive constraints, allowing for flexible scheduling under uncertainty, though solution finding remains NP-complete.
• The authors hypothesize that significant resource delay will negatively impact human teammates’ perception of fluency, motivating future empirical validation.