[Paper Review] Cooperative Task and Motion Planning for Multi-Arm Assembly Systems

Jingkai Chen, Jiaoyang Li+8 more|arXiv (Cornell University)|Mar 4, 2022

Modular Robots and Swarm Intelligence20 citations

TL;DR

The paper presents a hierarchical task and motion planning framework that combines MILP-based high-level task allocation with collision-aware low-level multi-agent path finding to safely and efficiently assemble complex multi-robot structures, demonstrated on various domains with up to 23 objects.

ABSTRACT

Multi-robot assembly systems are becoming increasingly appealing in manufacturing due to their ability to automatically, flexibly, and quickly construct desired structural designs. However, effectively planning for these systems in a manner that ensures each robot is simultaneously productive, and not idle, is challenging due to (1) the close proximity that the robots must operate in to manipulate the structure and (2) the inherent structural partial orderings on when each part can be installed. In this paper, we present a task and motion planning framework that jointly plans safe, low-makespan plans for a team of robots to assemble complex spatial structures. Our framework takes a hierarchical approach that, at the high level, uses Mixed-integer Linear Programs to compute an abstract plan comprised of an allocation of robots to tasks subject to precedence constraints and, at the low level, builds on a state-of-the-art algorithm for Multi-Agent Path Finding to plan collision-free robot motions that realize this abstract plan. Critical to our approach is the inclusion of certain collision constraints and movement durations during high-level planning, which better informs the search for abstract plans that are likely to be both feasible and low-makespan while keeping the search tractable. We demonstrate our planning system on several challenging assembly domains with several (sometimes heterogeneous) robots with grippers or suction plates for assembling structures with up to 23 objects involving Lego bricks, bars, plates, or irregularly shaped blocks.

Motivation & Objective

• Motivate flexible automation for robotic assembly and reduce manual reprogramming needs.
• Develop a framework that jointly optimizes task assignment and collision-free motions for multiple robots.
• Incorporate critical collision and duration information into high-level planning to improve feasibility and efficiency.
• Demonstrate applicability to heterogeneous robot teams assembling complex structures within close-proximity operations.

Proposed method

• Use mode graphs to describe robot manipulation procedures and precedence-constrained tasks.
• Generate multi-modal roadmaps for each robot to capture feasible movements and interactions with objects.
• Solve a relaxed MILP to assign tasks to robots while incorporating key collisions and precedence constraints.
• Annotate potential collisions across roadmaps and use a priority-based, decoupled MAPF-like search (PBS-AT) to produce collision-free motions.
• Extract subplan sequences from MILP solutions and apply a prioritized, iterative path planning approach with rSIPP to handle moving obstacles and precedence.

Experimental results

Research questions

• RQ1 How can task allocation and motion planning be integrated for multi-arm assembly to minimize makespan?
• RQ2 What high-level constraints (e.g., collisions, precedence) are essential to ensure feasible, collision-free low-level plans?
• RQ3 Can a hierarchical approach with MILP-based task assignment and MAPF-inspired planning scale to complex assemblies with multiple robots and objects?
• RQ4 How does incorporating collision-aware, task-relevant information at the high level influence plan feasibility and performance?

Key findings

• The framework yields safe, low-makespan plans for teams of robots assembling structures with up to 23 objects across diverse domains.
• Collision constraints and movement durations included during high-level planning improve feasibility and guide the search toward low-makespan solutions.
• The MILP-based task assignment effectively maps tasks to robots while respecting precedence and roadmap connectivity.
• The PBS-AT algorithm extends priority-based search to handle assigned tasks and moving obstacles, producing collision-free, precedence-compliant plans.
• Roadmap generation and annotation steps enable scalable planning by reusing collision information and decomposing the problem into tractable subproblems.