[Paper Review] Real-Time Spatio-Temporal LiDAR Point Cloud Compression

Yu Feng, Shaoshan Liu+1 more|arXiv (Cornell University)|Aug 16, 2020

Computer Graphics and Visualization Techniques26 references4 citations

TL;DR

This paper proposes a real-time spatio-temporal LiDAR point cloud compression system that leverages both spatial redundancy within a key frame and temporal redundancy across consecutive frames by reusing spatially encoded planes for temporal encoding. It achieves up to 90× compression with real-time performance (>10 FPS) and maintains high accuracy in end-to-end applications like object detection and registration, outperforming MPEG's standard in rate, speed, and accuracy.

ABSTRACT

Compressing massive LiDAR point clouds in real-time is critical to autonomous machines such as drones and self-driving cars. While most of the recent prior work has focused on compressing individual point cloud frames, this paper proposes a novel system that effectively compresses a sequence of point clouds. The idea to exploit both the spatial and temporal redundancies in a sequence of point cloud frames. We first identify a key frame in a point cloud sequence and spatially encode the key frame by iterative plane fitting. We then exploit the fact that consecutive point clouds have large overlaps in the physical space, and thus spatially encoded data can be (re-)used to encode the temporal stream. Temporal encoding by reusing spatial encoding data not only improves the compression rate, but also avoids redundant computations, which significantly improves the compression speed. Experiments show that our compression system achieves 40x to 90x compression rate, significantly higher than the MPEG's LiDAR point cloud compression standard, while retaining high end-to-end application accuracies. Meanwhile, our compression system has a compression speed that matches the point cloud generation rate by today LiDARs and out-performs existing compression systems, enabling real-time point cloud transmission.

Motivation & Objective

• To address the growing need for real-time, high-efficiency compression of massive LiDAR point cloud data in autonomous systems like self-driving cars and drones.
• To simultaneously improve compression rate, compression speed, and end-to-end application accuracy—three metrics often traded off in prior work.
• To exploit both spatial redundancy (within a frame) and temporal redundancy (across frames) in sequential LiDAR point clouds, which is underexplored in prior compression methods.
• To enable real-time transmission and cloud offloading by matching or exceeding the point cloud generation rate of modern LiDARs (5–20 Hz).
• To evaluate compression quality not by traditional metrics like PSNR, but by end-to-end application accuracy in tasks such as registration, object detection, and segmentation.

Proposed method

• The system converts raw point clouds into range images to enable efficient, structured processing and accelerate compression.
• It identifies a key frame (K-frame) in a sequence and performs spatial encoding by iteratively fitting planes to groups of points, leveraging the fact that many points in real-world scenes lie on planar surfaces.
• Subsequent frames (P-frames) are transformed into the K-frame’s coordinate system using IMU measurements to align overlapping regions.
• Temporal encoding reuses the planes identified in the K-frame to encode points in P-frames, reducing redundant computation and improving compression efficiency.
• The method compensates for IMU inaccuracies by leveraging spatial consistency across frames, enhancing robustness.
• A parallel implementation across multiple CPU cores accelerates processing, achieving high throughput (up to 52.1 FPS on PC, 20.5 FPS on Jetson TX2).

Fig. 1: Overview of our compression system, which compresses a sequence of consecutive point clouds. All the points clouds are converted to range images to accelerate the compression speed. We first spatially encode the key point-cloud (K-frame) in the sequence, typically the middle one. The spatial

Experimental results

Research questions

• RQ1 Can jointly exploiting spatial and temporal redundancies in LiDAR point cloud sequences lead to higher compression rates than current state-of-the-art methods?
• RQ2 Can such a system achieve real-time compression speeds matching or exceeding modern LiDAR generation rates (5–20 Hz)?
• RQ3 Does the compression method preserve sufficient geometric fidelity to maintain high accuracy in downstream robotic applications like object detection and registration?
• RQ4 How does the reuse of spatially encoded planes in temporal encoding affect computational efficiency and compression performance?
• RQ5 What is the impact of encoding multiple consecutive frames together on compression rate, speed, and application accuracy?

Key findings

• The proposed system achieves up to 90× compression rate, significantly outperforming MPEG’s LiDAR point cloud compression standard.
• Compression speeds reach up to 52.1 FPS on a PC and 20.5 FPS on a mobile Jetson TX2, exceeding the typical 5–20 Hz generation rate of modern LiDARs.
• The method maintains end-to-end application accuracy comparable to the original point cloud in registration, object detection, and segmentation tasks, outperforming existing compression schemes.
• The compression speed is highly sensitive to the number of consecutive frames processed, with parallelization across frames enabling up to 5.4× speedup on a four-core CPU.
• Application accuracy remains largely insensitive to the number of encoded frames, while compression rate remains stable due to consistent proportions of spatially and temporally encoded points.
• The system demonstrates that reusing spatially encoded planes for temporal encoding reduces redundant computation and improves both speed and compression efficiency.

Fig. 2: Different data structures used in our compression. The raw point clouds are converted to range images. After spatial and temporal encoding, most of the tiles in the range images are plane-encoded; the unfit tiles are left in the residual maps.