[Paper Review] M2DGR: A Multi-sensor and Multi-scenario SLAM Dataset for Ground Robots

Jie Yin, Ang Li+3 more|arXiv (Cornell University)|Dec 19, 2021

Robotics and Sensor-Based Localization9 citations

TL;DR

M2DGR is a large-scale, multi-sensor, multi-scenario SLAM dataset for ground robots, featuring 36 sequences (1TB) with synchronized data from 12 sensors including fisheye, sky-pointing, event, infrared, LiDAR, IMU, and GNSS. The dataset reveals significant performance drops in state-of-the-art SLAM systems under real-world challenges like low illumination, lift transitions, and outdoor-indoor switching, highlighting critical gaps in current algorithms.

ABSTRACT

We introduce M2DGR: a novel large-scale dataset collected by a ground robot with a full sensor-suite including six fish-eye and one sky-pointing RGB cameras, an infrared camera, an event camera, a Visual-Inertial Sensor (VI-sensor), an inertial measurement unit (IMU), a LiDAR, a consumer-grade Global Navigation Satellite System (GNSS) receiver and a GNSS-IMU navigation system with real-time kinematic (RTK) signals. All those sensors were well-calibrated and synchronized, and their data were recorded simultaneously. The ground truth trajectories were obtained by the motion capture device, a laser 3D tracker, and an RTK receiver. The dataset comprises 36 sequences (about 1TB) captured in diverse scenarios including both indoor and outdoor environments. We evaluate state-of-the-art SLAM algorithms on M2DGR. Results show that existing solutions perform poorly in some scenarios. For the benefit of the research community, we make the dataset and tools public. The webpage of our project is https://github.com/SJTU-ViSYS/M2DGR.

Motivation & Objective

• To address the lack of comprehensive, large-scale SLAM datasets tailored for ground robots operating in complex indoor and outdoor environments.
• To capture realistic, challenging scenarios—such as entering lifts, moving through dark environments, and transitioning between outdoor and indoor spaces—commonly faced by service and logistics robots.
• To evaluate the performance of state-of-the-art SLAM algorithms under practical conditions using a richly instrumented ground robot with synchronized, calibrated multi-sensor data.
• To expose the limitations of existing visual and LiDAR SLAM systems in real-world failure cases, thereby guiding future algorithm development.
• To establish a benchmark for multi-sensor SLAM research that supports fair comparison and drives innovation in robust robot localization and mapping.

Proposed method

• The dataset was collected using a ground robot equipped with six fisheye cameras, one sky-pointing RGB camera, a forward-looking pinhole camera, an event camera, an infrared camera, a 32-beam LiDAR, an IMU, and dual GNSS receivers (including RTK).
• All sensors were precisely calibrated and time-synchronized to ensure multi-modal data alignment for joint processing.
• Ground truth trajectories were obtained using a motion capture system, a laser 3D tracker, and an RTK GNSS receiver, providing high-accuracy reference poses.
• The dataset includes 36 sequences across diverse environments, including halls, corridors, lifts, roads, and indoor-outdoor transitions, with challenging conditions such as low illumination and dynamic motion.
• A comprehensive benchmark was conducted by evaluating multiple state-of-the-art SLAM systems—ORB-SLAM3, LIO-SAM, VINS-Mono, LVI-SAM, and others—on the dataset using standard metrics like ATE and APE.
• The evaluation focused on failure cases such as lift transitions, darkness, and abrupt motion, with qualitative and quantitative analysis of system robustness and drift.

Figure 1: Our dataset for ground robots was captured by a rich suite of sensors within various scenarios. Some sensory data are visualized.

Experimental results

Research questions

• RQ1 How do state-of-the-art visual and LiDAR SLAM systems perform in real-world ground robot scenarios involving low illumination, such as in dark corridors or nighttime streets?
• RQ2 To what extent can existing SLAM systems maintain accuracy and robustness when a robot enters and exits a lift, where visual and LiDAR sensors perceive the environment as static?
• RQ3 How do SLAM systems handle the transition from outdoor to indoor environments, particularly when GNSS signals degrade or disappear?
• RQ4 What are the failure modes of multi-camera visual SLAM systems when processing highly distorted fisheye images in dynamic or complex scenes?
• RQ5 How does the integration of complementary sensors—such as event cameras, thermal-infrared, and RTK-GNSS—improve SLAM robustness in challenging conditions?

Key findings

• ORB-SLAM3 failed to extract sufficient features in low-illumination scenes like Roomdark06 and Street07, even with adaptive histogram equalization, due to insufficient texture and reliance on distant bright objects.
• Thermal-infrared cameras improved robustness in dark scenes, but failed to detect texture-less objects like flat curtains, indicating no universal advantage over RGB cameras.
• None of the tested SLAM systems successfully tracked or mapped the entire trajectory during lift transitions (e.g., lift04), with LIO-SAM showing severe drift due to IMU-LiDAR odometry mismatch.
• Visual SLAM systems produced large ATE errors in outdoor-indoor transitions (e.g., Door01), despite GNSS availability, as GNSS signals degraded near doors and indoor positioning failed.
• Abrupt dynamic motion in Street07, including quick turns and speed changes, caused failures in both visual and LiDAR SLAM systems, with high ATE errors across all tested algorithms.
• Multi-camera visual SLAM (Multicol-SLAM) failed in nearly all sequences due to false feature matches from high distortion in fisheye images, underscoring challenges in multi-camera SLAM with non-pinhhole optics.

Figure 2: Our ground robot for data collection. (a) Side view of the robot. (b) Sensors on the top layer. (c) Sensors on the middle layer.