[Paper Review] Monte-Carlo Tree Search for Efficient Visually Guided Rearrangement Planning

Yann Labbé, Sergey Zagoruyko+5 more|arXiv (Cornell University)|Apr 23, 2019

Robot Manipulation and Learning42 references73 citations

TL;DR

The paper presents a complete pipeline for visually guided rearrangement planning using Monte-Carlo Tree Search (MCTS) to plan actions for moving many objects from an initial to a target arrangement, with a calibration-free deep visual state estimator trained on synthetic data. It achieves fast planning (60 ms for 25 objects) and robust real-robot performance on a UR-5 setup.

ABSTRACT

We address the problem of visually guided rearrangement planning with many movable objects, i.e., finding a sequence of actions to move a set of objects from an initial arrangement to a desired one, while relying on visual inputs coming from an RGB camera. To do so, we introduce a complete pipeline relying on two key contributions. First, we introduce an efficient and scalable rearrangement planning method, based on a Monte-Carlo Tree Search exploration strategy. We demonstrate that because of its good trade-off between exploration and exploitation our method (i) scales well with the number of objects while (ii) finding solutions which require a smaller number of moves compared to the other state-of-the-art approaches. Note that on the contrary to many approaches, we do not require any buffer space to be available. Second, to precisely localize movable objects in the scene, we develop an integrated approach for robust multi-object workspace state estimation from a single uncalibrated RGB camera using a deep neural network trained only with synthetic data. We validate our multi-object visually guided manipulation pipeline with several experiments on a real UR-5 robotic arm by solving various rearrangement planning instances, requiring only 60 ms to compute the plan to rearrange 25 objects. In addition, we show that our system is insensitive to camera movements and can successfully recover from external perturbations. Supplementary video, source code and pre-trained models are available at https://ylabbe.github.io/rearrangement-planning.

Motivation & Objective

• Motivate autonomous visually guided rearrangement of many movable objects in uncalibrated settings.
• Develop a scalable planning method that handles non-monotone, multi-object rearrangements without buffer space.
• Create a robust visual state prediction system trained purely on synthetic data via domain randomization.
• Demonstrate real-robot feasibility with fast planning and robustness to camera movements and perturbations.

Proposed method

• Introduce a discrete action parameterization for rearrangement to enable MCTS.
• Use Monte-Carlo Tree Search with a reward based on how many objects are at their target positions to guide planning.
• Couple MCTS with a standard RRT-based local motion planner for execution.
• Develop a calibration-free visual state estimator predicting 2D robot-coordinate positions of multiple objects from a single uncalibrated RGB image.
• Train a multi-object detector using synthetic data with domain randomization, including object masks and dense 2D position fields, without needing object CAD models or markers.

Experimental results

Research questions

• RQ1 Can an MCTS-based planner efficiently solve visually guided multi-object rearrangement without explicit buffer space?
• RQ2 How well can a calibration-free visual state estimator locate and identify multiple unknown objects from a single RGB image and map them to robot coordinates?
• RQ3 Does the integrated pipeline scale to large numbers of objects and operate in real-time on a physical robot?
• RQ4 Is the system robust to camera movements and external perturbations during execution?

Key findings

• MCTS scales to 37 objects with 80% success under a 100000-iteration limit, while the baseline struggles beyond ~33 objects.
• The planner moves significantly fewer objects than the baseline (e.g., for 30 objects, 40 moves with MCTS vs. 60 with the baseline).
• On a laptop with a single CPU core, MCTS finds plans for 25 objects in 60 ms, enabling online replanning after each move.
• The visual state estimator achieves 1.1 cm localization accuracy for a single object and maintains similar accuracy with up to 10 objects, enabling near 100% grasping success in tests up to 10 objects.
• The full pipeline demonstrates successful real-robot rearrangement on a UR-5 platform, solving varied instances and recovering from perturbations.