[Paper Review] Decentralized Task and Path Planning for Multi-Robot Systems

Yuxiao Chen, Ugo Rosolia+1 more|arXiv (Cornell University)|Nov 19, 2020

Reinforcement Learning in Robotics28 references4 citations

TL;DR

This paper proposes a fully decentralized task and path planning (DTPP) framework for multi-robot systems that jointly optimizes task allocation and collision-free path planning. Using a factor graph representation of Markov Decision Processes (MDPs) or Partially Observed MDPs (MOMDPs), the max-sum algorithm enables decentralized task allocation to maximize expected pure reward (reward minus cost), while a localized forward dynamic programming scheme resolves agent conflicts in real time, demonstrated via ROS simulations and real-world experiments with Segway and Turtlebot platforms.

ABSTRACT

We consider a multi-robot system with a team of collaborative robots and multiple tasks that emerges over time. We propose a fully decentralized task and path planning (DTPP) framework consisting of a task allocation module and a localized path planning module. Each task is modeled as a Markov Decision Process (MDP) or a Mixed Observed Markov Decision Process (MOMDP) depending on whether full states or partial states are observable. The task allocation module then aims at maximizing the expected pure reward (reward minus cost) of the robotic team. We fuse the Markov model into a factor graph formulation so that the task allocation can be decentrally solved using the max-sum algorithm. Each robot agent follows the optimal policy synthesized for the Markov model and we propose a localized forward dynamic programming scheme that resolves conflicts between agents and avoids collisions. The proposed framework is demonstrated with high fidelity ROS simulations and experiments with multiple ground robots.

Motivation & Objective

• To address the challenge of decentralized task and path planning in multi-robot systems where tasks emerge over time and require coordination.
• To maximize the expected pure reward (reward minus cost) of a robotic team in a decentralized manner.
• To resolve inter-agent conflicts during path execution without centralized coordination or global state knowledge.
• To enable real-time, scalable coordination for multi-robot systems using local, distributed computation.

Proposed method

• Each robot is modeled as an MDP or MOMDP, representing its stochastic dynamics and partial observability.
• The total expected pure reward is formulated as a factor graph, where nodes represent tasks and edges represent inter-agent dependencies.
• The max-sum algorithm is applied to the factor graph to solve for optimal task allocation in a fully decentralized manner.
• A localized forward dynamic programming scheme resolves potential collisions between agents by re-planning local trajectories in real time.
• High-level commands are sent to low-level controllers (e.g., MPC or PID), which generate real-time control inputs based on waypoints and constraints.
• The system integrates task allocation, conflict resolution, and low-level control in a hierarchical, distributed architecture.

Experimental results

Research questions

• RQ1 How can task allocation and path planning be jointly optimized in a fully decentralized multi-robot system?
• RQ2 Can the max-sum algorithm effectively solve for optimal task allocation in a decentralized MDP/MOMDP framework with partial observability?
• RQ3 How can inter-agent conflicts be resolved locally without global coordination or prior knowledge of future actions?
• RQ4 What is the performance of the DTPP framework in terms of expected pure reward and collision avoidance in dynamic, real-world scenarios?

Key findings

• The DTPP framework successfully achieves decentralized task allocation using the max-sum algorithm, maximizing the expected pure reward across multiple robots.
• The localized forward dynamic programming scheme effectively prevents collisions between agents in real time, even when trajectories conflict.
• In ROS simulations with three Segway robots, the system dynamically reassigns tasks when existing tasks expire, and resolves conflicts to ensure safe navigation.
• In experiments with two Turtlebots on a 5×5 grid, the max-sum algorithm correctly assigned robots to different tasks when beneficial, and allowed one robot to remain idle when it reduced cost.
• The expected pure reward curve (Fig. 8) demonstrated dynamic adaptation to new tasks, with values increasing when high-reward tasks emerged and were efficiently assigned.
• The system maintained real-time performance using MPC and PID controllers, enabling stable execution on physical robots.