[论文解读] Streaming Object Detection for 3-D Point Clouds

Wei Han, Zhengdong Zhang+8 more|arXiv (Cornell University)|May 4, 2020

Advanced Neural Network Applications参考文献 65被引用 6

一句话总结

本文提出了一种用于3D LiDAR点云的流式目标检测框架，能够以增量方式处理随时间到达的数据，与传统批处理相比，延迟降低高达15倍。通过将局部感受野、有状态的非极大值抑制（NMS）和循环时间建模整合到单阶段检测架构中，该方法在显著降低峰值计算需求的同时，实现了最先进的mAP（行人达到60.1），并提升了推理效率。

ABSTRACT

Autonomous vehicles operate in a dynamic environment, where the speed with which a vehicle can perceive and react impacts the safety and efficacy of the system. LiDAR provides a prominent sensory modality that informs many existing perceptual systems including object detection, segmentation, motion estimation, and action recognition. The latency for perceptual systems based on point cloud data can be dominated by the amount of time for a complete rotational scan (e.g. 100 ms). This built-in data capture latency is artificial, and based on treating the point cloud as a camera image in order to leverage camera-inspired architectures. However, unlike camera sensors, most LiDAR point cloud data is natively a streaming data source in which laser reflections are sequentially recorded based on the precession of the laser beam. In this work, we explore how to build an object detector that removes this artificial latency constraint, and instead operates on native streaming data in order to significantly reduce latency. This approach has the added benefit of reducing the peak computational burden on inference hardware by spreading the computation over the acquisition time for a scan. We demonstrate a family of streaming detection systems based on sequential modeling through a series of modifications to the traditional detection meta-architecture. We highlight how this model may achieve competitive if not superior predictive performance with state-of-the-art, traditional non-streaming detection systems while achieving significant latency gains (e.g. 1/15'th - 1/3'rd of peak latency). Our results show that operating on LiDAR data in its native streaming formulation offers several advantages for self driving object detection -- advantages that we hope will be useful for any LiDAR perception system where minimizing latency is critical for safe and efficient operation.

研究动机与目标

• 通过在完整LiDAR扫描完成后立即处理点云数据，消除由完整扫描完成带来的延迟。
• 通过在扫描持续时间内分布推理负载，降低峰值计算需求。
• 与非流式基线相比，保持或提升检测精度（mAP）。
• 通过原生LiDAR数据流式传输，实现实时、低延迟感知，适用于自动驾驶车辆。
• 证明在计算资源受限的情况下，流式架构可优于非流式模型。

提出的方法

• 将LiDAR点云按时间片段（例如每扫描4至64个片段）增量式处理，而非等待360°完整旋转完成。
• 应用局部感受野，将空间上下文限制在当前和最近的扫描片段内，从而降低计算负载。
• 引入有状态的非极大值抑制（NMS），在片段间保留检测历史，以提升跟踪一致性。
• 引入循环神经网络（RNN）以建模时间状态，提升随时间推移的检测稳定性。
• 通过增加特征图分辨率或隐藏单元数量来扩展模型容量，同时保持流式推理能力。
• 在扫描时间上实现计算流水线化，降低峰值FLOPS，实现更低延迟的推理。

实验结果

研究问题

• RQ1 能否通过利用LiDAR数据的原生流式特性，在3D LiDAR点云上实现显著降低的延迟？
• RQ2 在流式设置中，时间建模和有状态操作如何提升检测精度和一致性？
• RQ3 在仅使用非流式基线所需峰值计算资源的一小部分时，流式模型能否实现具有竞争力或更优的mAP？
• RQ4 为适应标准检测元架构的流式推理，需要进行哪些架构修改？
• RQ5 在相同计算预算下，流式设置中的模型扩展在多大程度上可超越非流式性能？

主要发现

• 该流式模型在行人检测上达到60.1 mAP，超过非流式基线的54.9 mAP。
• 对于车辆检测，流式模型达到51.0 mAP，与非流式基线持平，但延迟显著降低。
• 系统将预期检测延迟从基线的约120 ms降低至约30 ms，实现4倍减少（峰值延迟的1/15）。
• 峰值计算需求降低至非流式基线的1/10，同时保持或提升性能。
• 即使仅使用非流式模型峰值FLOPS的1/3，有状态RNN与NMS变体仍保持高精度，并实现超过3倍的延迟降低。
• 更大的流式模型在精度上超过非流式基线（例如，60.1 vs 54.9 mAP），同时仅需非流式模型峰值FLOPS的1/2.25。