[Paper Review] Efficient LiDAR data compression for embedded V2I or V2V data handling

P. Caillet, Yohan Dupuis|arXiv (Cornell University)|Apr 11, 2019

Algorithms and Data Compression9 references4 citations

TL;DR

This paper proposes a lightweight, bit-mask-based LiDAR data compression method for real-time V2V/V2I applications on low-power embedded systems. By zeroing the least significant bits of raw 16-bit range measurements and applying standard lossless compression, it achieves high compression ratios (up to 63.8% of original size) with minimal processing overhead, outperforming H.264 and PNG-based methods in speed and efficiency while preserving accuracy within vendor-specified tolerances.

ABSTRACT

LiDAR are increasingly being used in intelligent vehicles (IV) or intelligent transportation systems (ITS). Storage and transmission of data generated by LiDAR sensors are one of the most challenging aspects of their deployment. In this paper we present a method that can be used to efficiently compress LiDAR data in order to facilitate storage and transmission in V2V or V2I applications. This method can be used to perform lossless or lossy compression and is specifically designed for embedded applications with low processing power. This method is also designed to be easily applicable to existing processing chains by keeping the structure of the data stream intact. We benchmarked our method using several publicly available datasets and compared it with state-of-the-art LiDAR data compression methods from the literature.

Motivation & Objective

• Address the challenge of high-bandwidth LiDAR data transmission and storage in embedded V2V/V2I systems with limited processing power.
• Enable real-time, low-latency compression of raw LiDAR point clouds for efficient data transmission over constrained wireless networks.
• Maintain compatibility with existing data processing pipelines by preserving the original data stream structure.
• Achieve high compression ratios with controlled loss through bit masking, exploiting measurement redundancy and vendor-specified accuracy margins.
• Demonstrate superior performance over existing methods like H.264 and PNG in processing time and compression efficiency on embedded platforms.

Proposed method

• Apply a bit mask to zero the n least significant bits (LSBs) of each 16-bit LiDAR range measurement, reducing precision and creating repeating zero patterns.
• Use standard lossless compression (e.g., bzip2) on the masked data to exploit the redundancy introduced by bit masking.
• Leverage the fact that most LiDAR sensors use 16-bit integers with a step size (e.g., 2 mm) much smaller than their stated accuracy (e.g., ±3 cm), allowing for precision reduction without loss of meaningful information.
• Design the method to be fully compatible with existing data processing chains by preserving the original data packet structure and format.
• Tune the number of zeroed bits (n) to balance compression ratio and introduced error, enabling both lossy and lossless compression modes.
• Implement the method using only bitwise operations and standard compression, ensuring low computational cost suitable for embedded systems.

Experimental results

Research questions

• RQ1 Can bit masking of LiDAR range measurements significantly reduce data size while maintaining accuracy within vendor-specified tolerances?
• RQ2 How does the proposed bit-masking compression method compare in compression ratio and processing time to state-of-the-art methods like H.264 and PNG on embedded and desktop systems?
• RQ3 To what extent can the method be integrated into existing V2V/V2I data processing pipelines without modifying data structure?
• RQ4 What is the optimal number of zeroed bits (n) that balances compression efficiency and introduced measurement error across diverse LiDAR datasets?
• RQ5 Can the method achieve real-time compression on low-power embedded hardware, making it viable for roadside units and intelligent vehicles?

Key findings

• The proposed method achieved a relative file size of 35.1% for the office dataset and 63.8% for the test track dataset when 4 bits were zeroed, outperforming H.264-based compression in both compression ratio and processing time.
• Processing time for the proposed method was 105% of dataset duration on the office dataset, compared to 128% for the H.264 method, indicating significantly better real-time feasibility.
• The method achieved a mean error of 14.8 mm and standard deviation of 9.8 mm on the office dataset with 4 bits masked, remaining within the typical ±3 cm accuracy of the Velodyne VLP-16 sensor.
• PNG-based compression achieved 47.5% file size reduction on the office dataset but required 24.1% of the dataset duration for processing, making it less suitable for real-time embedded use.
• H.264-based compression on a single CPU thread was 2–3 times slower on embedded hardware than on desktop systems, rendering it impractical for real-time embedded deployment.
• The method demonstrated consistent performance across heterogeneous datasets (office, test track, Velodyne sample), proving robustness in real-world conditions.