[Paper Review] Kimera-Multi: Robust, Distributed, Dense Metric-Semantic SLAM for Multi-Robot Systems

Yulun Tian, Yun Chang+4 more|arXiv (Cornell University)|Jun 28, 2021

Robotics and Sensor-Based Localization85 references4 citations

TL;DR

Kimera-Multi presents a fully distributed, robust, and communication-efficient multi-robot SLAM system that builds globally consistent, dense metric-semantic 3D mesh maps in real time. By combining distributed place recognition with a novel robust pose graph optimization (D-GNC), it achieves trajectory accuracy comparable to centralized systems while rejecting perceptual aliasing outliers and enabling high-level spatial AI through semantic mesh annotation.

ABSTRACT

This paper presents Kimera-Multi, the first multi-robot system that (i) is robust and capable of identifying and rejecting incorrect inter and intra-robot loop closures resulting from perceptual aliasing, (ii) is fully distributed and only relies on local (peer-to-peer) communication to achieve distributed localization and mapping, and (iii) builds a globally consistent metric-semantic 3D mesh model of the environment in real-time, where faces of the mesh are annotated with semantic labels. Kimera-Multi is implemented by a team of robots equipped with visual-inertial sensors. Each robot builds a local trajectory estimate and a local mesh using Kimera. When communication is available, robots initiate a distributed place recognition and robust pose graph optimization protocol based on a novel distributed graduated non-convexity algorithm. The proposed protocol allows the robots to improve their local trajectory estimates by leveraging inter-robot loop closures while being robust to outliers. Finally, each robot uses its improved trajectory estimate to correct the local mesh using mesh deformation techniques. We demonstrate Kimera-Multi in photo-realistic simulations, SLAM benchmarking datasets, and challenging outdoor datasets collected using ground robots. Both real and simulated experiments involve long trajectories (e.g., up to 800 meters per robot). The experiments show that Kimera-Multi (i) outperforms the state of the art in terms of robustness and accuracy, (ii) achieves estimation errors comparable to a centralized SLAM system while being fully distributed, (iii) is parsimonious in terms of communication bandwidth, (iv) produces accurate metric-semantic 3D meshes, and (v) is modular and can be also used for standard 3D reconstruction (i.e., without semantic labels) or for trajectory estimation (i.e., without reconstructing a 3D mesh).

Motivation & Objective

• To enable multi-robot SLAM systems that are robust to perceptual aliasing and operate under limited communication bandwidth.
• To build globally consistent, dense metric-semantic 3D mesh maps in real time using only peer-to-peer communication.
• To achieve trajectory estimation accuracy comparable to centralized SLAM while maintaining full distribution and low communication overhead.
• To support high-level decision-making by annotating mesh faces with semantic labels (e.g., 'building', 'road').
• To develop a modular system that supports both 3D reconstruction and trajectory estimation independently.

Proposed method

• Each robot uses Kimera to build a local trajectory estimate and a local dense 3D mesh using visual-inertial sensors.
• When communication is available, robots perform distributed place recognition to detect inter- and intra-robot loop closures.
• A novel distributed graduated non-convexity (D-GNC) algorithm performs robust pose graph optimization using a two-stage RBCD-based approach to reject outliers.
• D-GNC enables fully distributed optimization by iteratively updating variables across robots using local peer-to-peer communication.
• Each robot uses the globally optimized trajectory to deform and correct its local mesh, improving metric-semantic accuracy.
• The system is modular and can be used for standard 3D reconstruction or trajectory estimation without semantic labeling.

Experimental results

Research questions

• RQ1 Can a fully distributed multi-robot SLAM system achieve metric-semantic accuracy comparable to centralized systems while minimizing communication bandwidth?
• RQ2 How can perceptual aliasing be effectively mitigated in multi-robot SLAM using robust distributed optimization?
• RQ3 Can dense, globally consistent 3D mesh maps with semantic labels be constructed in real time using only peer-to-peer communication?
• RQ4 How does the performance of the proposed D-GNC algorithm compare to centralized and prior distributed outlier rejection methods in challenging outdoor environments?
• RQ5 To what extent does the system maintain accuracy under sparse inter-robot loop closures and visually degraded conditions?

Key findings

• On the Medfield dataset, Kimera-Multi reduced end-to-end trajectory errors to 0.01–0.13 meters, matching the performance of a centralized solver, while Kimera-VIO accumulated drifts of 14.84–24.55 meters.
• On the more challenging Stata dataset, Kimera-Multi achieved end-to-end errors of 1.17–33.13 meters after 2000 RBCD iterations, significantly improving from initial estimates and approaching centralized performance.
• The system reduced communication costs substantially compared to baseline centralized SLAM, achieving high accuracy with minimal bandwidth usage through peer-to-peer loop closure exchange.
• The D-GNC algorithm successfully rejected outlier loop closures caused by perceptual aliasing, enabling robust optimization even with limited inter-robot loop closures.
• The final optimized 3D meshes produced by Kimera-Multi showed improved metric-semantic accuracy, with faces correctly labeled and geometric fidelity preserved across robots.
• Kimera-Multi demonstrated scalability and robustness on long trajectories (up to 860 meters per robot), validating its real-world applicability in outdoor environments.