[Paper Review] Multi-Camera LiDAR Inertial Extension to the Newer College Dataset

Lintong Zhang, Marco Camurri+2 more|arXiv (Cornell University)|Dec 16, 2021

Remote Sensing and LiDAR Applications28 citations

TL;DR

This paper presents a multi-camera LiDAR inertial handheld dataset extending Newer College with hardware-synchronized cameras, IMU, and a 128-channel LiDAR, plus high-frequency 6 DoF ground truth and an example multi-camera visual-inertial odometry method.

ABSTRACT

We present a multi-camera LiDAR inertial dataset of 4.5 km walking distance as an expansion of the Newer College Dataset. The global shutter multi-camera device is hardware synchronized with both the IMU and LiDAR, which is more accurate than the original dataset with software synchronization. This dataset also provides six Degrees of Freedom (DoF) ground truth poses at LiDAR frequency (10 Hz). Three data collections are described and an example use case of multi-camera visual-inertial odometry is demonstrated. This expansion dataset contains small and narrow passages, large scale open spaces, as well as vegetated areas, to test localization and mapping systems. Furthermore, some sequences present challenging situations such as abrupt lighting change, textureless surfaces, and aggressive motion. The dataset is available at: https://ori-drs.github. io/newer-college-dataset/

Motivation & Objective

• Provide a comprehensive multi-camera visual-inertial and LiDAR dataset extended from Newer College.
• Offer high-frequency 6-DoF ground truth poses using a detailed prior map.
• Demonstrate the advantages of hardware synchronization for robust SLAM in challenging environments.

Proposed method

• Assemble a handheld device with a four-camera Alphasense system, a 128-channel Ouster LiDAR, and a healthily synchronized IMU.
• Capture 4.5 km of walking trajectories across varied environments including narrow passages, open spaces, and vegetation.
• Generate high-frequency ground-truth poses by registering LiDAR scans to a Leica BLK360-based prior map with IMU distortion correction.
• Calibrate intrinsics and extrinsics with Kalibr for camera-IMU-LiDAR synchronization.
• Provide dual-PTP synchronized data streams to achieve sub-microsecond timing accuracy across sensors.
• Demonstrate multi-camera visual-inertial odometry with VILENS-MC as an example usage.

Experimental results

Research questions

• RQ1 How can a multi-camera, LiDAR, and inertial sensor setup improve robust SLAM in challenging outdoor and indoor environments?
• RQ2 What are the benefits of hardware synchronization over software synchronization for high-frequency pose estimation?
• RQ3 How does multi-camera visual-inertial odometry perform on sequences with rapid lighting changes, textureless surfaces, and aggressive motion?

Key findings

• The dataset extends Newer College with a 4.5 km walking distance and high-frequency 6-DoF ground truth at LiDAR rate (10 Hz).
• Hardware synchronization using PTP provides sub-microsecond accuracy between LiDAR, cameras, and IMU.
• The collection includes diverse scenes such as stairs, corridors, parks, and textured/textureless regions to challenge SLAM systems.
• The paper demonstrates multi-camera visual-inertial odometry (VILENS-MC) outperforming mono/stereo methods in challenging sequences.
• Open comparisons show that methods like ORB-SLAM3 and OpenVINS struggle on textureless or fast-motion sequences, while VILENS-MC benefits from cross-camera feature tracking.
• A detailed calibration protocol with Kalibr and per-collection calibration files is provided.