[Paper Review] Robot Inner Attention Modeling for Task-Adaptive Teaming of Heterogeneous Multi Robots

Chao Huang, Rui Liu|arXiv (Cornell University)|Jun 28, 2020

Reinforcement Learning in Robotics35 references4 citations

TL;DR

This paper proposes innerATT, a novel multi-agent reinforcement learning framework that integrates an attention mechanism into a multi-agent actor-critic architecture to enable flexible, task-adaptive teaming of heterogeneous robots. By dynamically assigning attention weights to teammates based on task demands and robot capabilities, innerATT achieves superior team composition accuracy and adaptability across single, double, and mixed tasks, outperforming baseline methods in dynamic environments with robot failures and changing task requirements.

ABSTRACT

Attracted by team scale and function diversity, a heterogeneous multi-robot system (HMRS), where multiple robots with different functions and numbers are coordinated to perform tasks, has been widely used for complex and large-scale scenarios, including disaster search and rescue, site surveillance, and social security. However, due to the variety of the task requirements, it is challenging to accurately compose a robot team with appropriate sizes and functions to dynamically satisfy task needs while limiting the robot resource cost to a low level. To solve this problem, in this paper, a novel adaptive cooperation method, inner attention (innerATT), is developed to flexibly team heterogeneous robots to execute tasks as task types and environment change. innerATT is designed by integrating a novel attention mechanism into a multi-agent actor-critic reinforcement learning architecture. With an attention mechanism, robot capability will be analyzed to flexibly form teams to meet task requirements. Scenarios with different task variety ("Single Task", "Double Task", and "Mixed Task") were designed. The effectiveness of the innerATT was validated by its accuracy in flexible cooperation.

Motivation & Objective

• To address the challenge of dynamically composing optimal robot teams in heterogeneous multi-robot systems (HMRS) under varying task demands and real-world constraints.
• To overcome limitations of pre-defined teaming strategies and non-adaptive reinforcement learning methods that fail under robot faults or changing task complexity.
• To enable robots to autonomously select teammates based on real-time cooperation potential using an attention mechanism.
• To improve teaming adaptability and robustness in complex, large-scale scenarios such as disaster response and traffic control.

Proposed method

• Integrates a multi-head self-attention mechanism into a multi-agent actor-critic reinforcement learning framework to model inter-robot cooperation dynamics.
• Uses robot observations, status, actions, and teammate communication data as input to compute attention weights for each potential teammate.
• Employs a Q-value network that uses learned attention weights to select optimal cooperation strategies based on task-specific needs.
• Trains the model end-to-end using deep reinforcement learning, allowing attention weights to be automatically learned during deployment.
• Supports multiple attention heads to evaluate different aspects of robot cooperability, enhancing flexibility in team formation.
• Designed to be robust to robot failures by selectively focusing on reliable teammates, reducing reliance on faulty information.

Experimental results

Research questions

• RQ1 How can heterogeneous robots dynamically form optimal teams in response to changing task types and environmental conditions?
• RQ2 To what extent does an attention mechanism improve teaming adaptability compared to non-selective or fixed-weight strategies?
• RQ3 Can the proposed method maintain high performance under robot failures such as sensor degradation or partial functionality loss?
• RQ4 How does the attention mechanism support behavioral adaptation during mission transitions between different tasks?
• RQ5 What is the impact of attention-based selection on teaming accuracy and resource efficiency in mixed-task scenarios?

Key findings

• innerATT significantly outperformed baseline methods in teaming accuracy across all task types, including single, double, and mixed tasks.
• The attention mechanism enabled robots to dynamically shift cooperation focus—from medical assistance robots to navigation robots—during mission transitions, as evidenced by attention weight shifts in Figure 5.
• Entropy of attention weights decreased to approximately 1.02 over training, indicating stable and selective attention learning, confirming the model’s ability to identify key teammates.
• In UAV participation rate comparisons, innerATT showed statistically non-significant differences (χ² < 3.84) under varying tasks, indicating consistent and adaptive teaming performance.
• The method demonstrated robustness to robot failures by minimizing reliance on faulty teammates, as shown by selective attention to reliable collaborators.
• PPO-innerATT and TD-innerATT variants achieved balanced UAV participation rates across tasks, unlike baseline methods that showed strong biases, indicating improved fairness and adaptability.