[Paper Review] Kimera-Multi: a System for Distributed Multi-Robot Metric-Semantic Simultaneous Localization and Mapping

Yun Chang, Yulun Tian+2 more|arXiv (Cornell University)|Nov 8, 2020

Robotics and Sensor-Based Localization74 references75 citations

TL;DR

Kimera-Multi presents the first fully distributed multi-robot system for dense metric-semantic SLAM, enabling teams of visual-inertial robots to collaboratively build accurate, semantically labeled 3D mesh maps in real time. By combining distributed place recognition, incremental maximum clique outlier rejection, Riemannian block-coordinate descent for pose graph optimization, and real-time mesh deformation, the system achieves high accuracy with 80% less communication and lower computation than centralized approaches.

ABSTRACT

We present the first fully distributed multi-robot system for dense metric-semantic Simultaneous Localization and Mapping (SLAM). Our system, dubbed Kimera-Multi, is implemented by a team of robots equipped with visual-inertial sensors, and builds a 3D mesh model of the environment in real-time, where each face of the mesh is annotated with a semantic label (e.g., building, road, objects). In Kimera-Multi, each robot builds a local trajectory estimate and a local mesh using Kimera. Then, when two robots are within communication range, they initiate a distributed place recognition and robust pose graph optimization protocol with a novel incremental maximum clique outlier rejection; the protocol allows the robots to improve their local trajectory estimates by leveraging inter-robot loop closures. Finally, each robot uses its improved trajectory estimate to correct the local mesh using mesh deformation techniques. We demonstrate Kimera-Multi in photo-realistic simulations and real data. Kimera-Multi (i) is able to build accurate 3D metric-semantic meshes, (ii) is robust to incorrect loop closures while requiring less computation than state-of-the-art distributed SLAM back-ends, and (iii) is efficient, both in terms of computation at each robot as well as communication bandwidth.

Motivation & Objective

• To develop a fully distributed multi-robot system for dense metric-semantic SLAM under realistic constraints of communication, computation, and sensing.
• To enable accurate, real-time 3D metric-semantic mesh reconstruction by fusing local robot estimates through inter-robot collaboration.
• To reduce communication bandwidth and computational load while maintaining robustness to incorrect loop closures in distributed SLAM.
• To integrate trajectory optimization with mesh correction via real-time mesh deformation for consistent global map updates.

Proposed method

• Each robot uses Kimera-VIO and Kimera-Semantics to build local trajectory and dense metric-semantic 3D mesh from visual-inertial and semantic segmentation data.
• When within communication range, robots perform distributed place recognition using Bag-of-Words (BoW) descriptors from keyframes.
• A novel incremental maximum clique algorithm is used for robust outlier rejection in the distributed pose graph optimization (PGO) pipeline.
• Distributed PGO is performed via the Riemannian Block-Coordinate Descent (RBCD) method, enabling efficient, scalable optimization across robots.
• Improved trajectory estimates are used to correct local meshes via real-time mesh deformation based on Sumner et al. [52].
• The system uses a decentralized architecture where all processing and optimization occur locally, minimizing reliance on centralized computation or communication.

Experimental results

Research questions

• RQ1 Can a fully distributed multi-robot system achieve accurate, real-time dense metric-semantic SLAM with minimal communication and computation?
• RQ2 How can robust inter-robot loop closure detection and outlier rejection be achieved in a decentralized setting without a central fusion point?
• RQ3 To what extent can distributed PGO with incremental maximum clique rejection improve trajectory accuracy while reducing computational cost compared to centralized or prior distributed methods?
• RQ4 How effective is real-time mesh deformation in correcting local meshes using optimized trajectories from inter-robot collaboration?
• RQ5 What is the trade-off between communication efficiency, computational load, and reconstruction accuracy in large-scale multi-robot metric-semantic mapping?

Key findings

• Kimera-Multi reduces communication by up to 80% compared to a centralized system transmitting raw images or keypoints.
• The system achieves a 2.28 m ATE on the Camp dataset and 0.308 m ATE on Vicon Room 2, demonstrating high trajectory accuracy.
• The incremental maximum clique outlier rejection method improves robustness to incorrect loop closures while reducing computation compared to prior methods.
• The early-stopped variant of RBCD achieves satisfactory trajectory estimates in just 2.76 seconds on average, making it suitable for real-time deployment.
• Mesh deformation via local optimization improves semantic reconstruction accuracy from 67.91% to 83.93% on the merged mesh in the City simulation.
• The system outperforms local PGO and DGS in both ATE and PGO cost, with RBCD yielding lower costs and better accuracy than DGS on all datasets except Vicon Room 2, where noise in loop closures affects performance.