[论文解读] Unsupervised Sequence Forecasting of 100,000 Points for Unsupervised Trajectory Forecasting.

Xinshuo Weng, Jianren Wang+3 more|arXiv (Cornell University)|Mar 18, 2020

3D Shape Modeling and Analysis参考文献 19被引用 9

一句话总结

本文提出 SPCSFNet，一种新颖的无监督方法，用于从 LiDAR 数据中预测密集 3D 场景点云（超过 100,000 个点），实现端到端的未来预测，无需对象级标注。通过在无标签传感器数据上进行训练并利用场景级预测，该方法在下游对象轨迹预测任务中达到最先进性能，优于需要轨迹标签的监督方法。

ABSTRACT

Predicting the future is a crucial first step to effective control, since systems that can predict the future can select plans that lead to desired outcomes. In this work, we study the problem of future prediction at the level of 3D scenes, represented by point clouds captured by a LiDAR sensor, i.e., directly learning to forecast the evolution of >100,000 points that comprise a complete scene. We term this Scene Point Cloud Sequence Forecasting (SPCSF). By directly predicting the densest-possible 3D representation of the future, the output contains richer information than other representations such as future object trajectories. We design a method, SPCSFNet, evaluate it on the KITTI and nuScenes datasets, and find that it demonstrates excellent performance on the SPCSF task. To show that SPCSF can benefit downstream tasks such as object trajectory forecasting, we present a new object trajectory forecasting pipeline leveraging SPCSFNet. Specifically, instead of forecasting at the object level as in conventional trajectory forecasting, we propose to forecast at the sensor level and then apply detection and tracking on the predicted sensor data. As a result, our new pipeline can remove the need of object trajectory labels and enable large-scale training with unlabeled sensor data. Surprisingly, we found our new pipeline based on SPCSFNet was able to outperform the conventional pipeline using state-of-the-art trajectory forecasting methods, all of which require future object trajectory labels. Finally, we propose a new evaluation procedure and two new metrics to measure the end-to-end performance of the trajectory forecasting pipeline. Our code will be made publicly available at this https URL

研究动机与目标

• 解决在自动驾驶中使用密集点云进行长时程、高分辨率 3D 场景预测的挑战。
• 通过从对象级预测转向传感器级预测，消除对对象轨迹标注的依赖。
• 开发一种可扩展的无监督学习框架，能够泛化于多样的 3D 场景动态。
• 证明场景级预测可优于传统的对象级轨迹预测方法。

提出的方法

• 该方法 SPCSFNet 直接从输入的 LiDAR 扫描中预测未来的 3D 点云序列，建模每个场景中超过 100,000 个点的演化过程。
• 采用一种自监督学习目标，鼓励在时间与几何增强下预测的点云序列保持一致性。
• 网络架构设计用于保留长序列中精细的几何细节与空间一致性。
• 引入一种新型流程，将预测的点云作为输入送入对象检测与跟踪模块，从而无需轨迹标签。
• 利用大规模无标签 LiDAR 数据进行预训练，使模型能够泛化至多样的驾驶场景。
• 提出一种新的评估协议，通过预测的点云来衡量端到端轨迹预测性能。

实验结果

研究问题

• RQ1 无监督 3D 场景点云预测能否优于需要监督的轨迹预测方法？
• RQ2 在无对象级监督的情况下，传感器级预测在捕捉动态场景演化方面有多有效？
• RQ3 在原始 LiDAR 序列上进行自监督学习，能在多大程度上提升下游轨迹预测性能？
• RQ4 哪些关键设计选择使得对超过 100,000 个点的密集点云实现高保真预测成为可能？
• RQ5 所提出的评估流程为何能比标准轨迹指标更准确地反映真实世界性能？

主要发现

• SPCSFNet 在 SPCSF 任务上达到最先进性能，展示了对超过 100,000 个点的密集 3D 场景的高保真预测能力。
• 所提出的传感器级预测流程优于现有最先进轨迹预测方法，后者需要未来轨迹标注。
• 该方法在 KITTI 和 nuScenes 数据集上验证了其在多样化驾驶场景中的良好泛化能力。
• 在无标签 LiDAR 数据上进行自监督训练，可实现有效的特征学习，而无需任何轨迹级别监督。
• 新的评估流程与指标能更准确地评估端到端轨迹预测性能。