[论文解读] SCOOP: Self-Supervised Correspondence and Optimization-Based Scene Flow

Itai Lang, Dror Aiger+3 more|arXiv (Cornell University)|Nov 25, 2022

Advanced Vision and Imaging被引用 4

一句话总结

SCOOP 提出了一种自监督的3D场景光流估计方法，通过将对应关系学习与光流优化分离：轻量级网络学习点特征以初始化软对应关系，进而通过运行时优化残差光流更新。该方法在极少数据上进行训练，无需真实光流监督，仅使用少量数据即在 KITTI 数据集上达到最先进精度，推理速度比纯优化方法快 8 倍，且在远少于监督方法的训练数据量下表现更优。

ABSTRACT

Scene flow estimation is a long-standing problem in computer vision, where the goal is to find the 3D motion of a scene from its consecutive observations. Recently, there have been efforts to compute the scene flow from 3D point clouds. A common approach is to train a regression model that consumes source and target point clouds and outputs the per-point translation vector. An alternative is to learn point matches between the point clouds concurrently with regressing a refinement of the initial correspondence flow. In both cases, the learning task is very challenging since the flow regression is done in the free 3D space, and a typical solution is to resort to a large annotated synthetic dataset. We introduce SCOOP, a new method for scene flow estimation that can be learned on a small amount of data without employing ground-truth flow supervision. In contrast to previous work, we train a pure correspondence model focused on learning point feature representation and initialize the flow as the difference between a source point and its softly corresponding target point. Then, in the run-time phase, we directly optimize a flow refinement component with a self-supervised objective, which leads to a coherent and accurate flow field between the point clouds. Experiments on widespread datasets demonstrate the performance gains achieved by our method compared to existing leading techniques while using a fraction of the training data. Our code is publicly available at https://github.com/itailang/SCOOP.

研究动机与目标

• 解决在缺乏或仅有少量真实光流标注的情况下，3D 点云中场景光流估计的挑战。
• 通过在小规模真实数据集上实现有效训练，减少对大规模合成数据集的依赖。
• 通过将对应关系学习与光流优化分离，提升光流估计的精度与效率。
• 在不进行端到端光流回归训练的前提下，实现高精度的场景光流估计。
• 通过结合轻量级对应关系学习与高效的运行时优化，平衡推理速度与精度。

提出的方法

• 训练一个自监督神经网络，仅用于学习源点云与目标点云之间软对应关系的点特征，采用置信度感知的距离损失。
• 光流初始化通过计算每个源点与其软对应目标点之间的差异得到，其中对应关系基于学习到的特征相似性对目标点进行加权求和来定义。
• 在推理阶段，通过自监督目标直接优化残差光流更新，以保证光流的一致性并使结果接近目标点云。
• 优化目标包含平滑性先验，用于正则化邻近点具有相似的光流向量，从而提升场的一致性。
• 通过将特征学习（可训练）与光流优化（在测试时优化）分离，避免了光流回归的端到端训练。
• 在训练过程中，使用对应网络输出的置信度分数对距离损失进行加权，突出可靠匹配。

Figure 1 : Flow accuracy on the KITTI benchmark vs . the train set size. Our method is trained on one or two orders of magnitude less data while surpassing the performance of the competing techniques [ 20 , 25 , 29 , 37 , 17 , 18 , 36 , 4 ] by a large margin. Please see Table 1 for the complete deta

实验结果

研究问题

• RQ1 是否仅使用少量真实世界数据且无真实光流监督，也能实现准确的场景光流估计？
• RQ2 将对应关系学习与光流优化分离，是否能提升性能并降低对数据的依赖？
• RQ3 运行时对残差光流更新的优化，是否能在精度与效率上超越基于端到端学习的方法？
• RQ4 置信度感知的对应关系学习如何影响光流初始化的鲁棒性与精度？
• RQ5 在优化阶段调整优化步数时，推理时间与精度之间存在何种权衡？

主要发现

• SCOOP 在 KITTI 测试集上实现了 0.039 的 EPE，优于当前最先进的监督与自监督方法，且训练数据仅为极小部分。
• 仅使用 FT3D 训练集的 10% 数据，SCOOP 即可发挥全部性能潜力，展现出极强的数据效率。
• 与仅使用前馈网络的方法相比，SCOOP + 将 EPE 降低超过 50%，且推理速度比纯优化方法 Neural Prior 快 8 倍。
• 消融实验表明，若移除光流优化模块，EPE 上升至 0.115，证实其在提升光流一致性和精度方面具有关键作用。
• 若省略平滑性损失，EPE 上升至 0.056，表明该正则化项对学习鲁棒点表示至关重要。
• 仅使用传输最高的目标点作为候选点（而非所有点）可提升性能，表明聚焦对应关系能带来更优的初始化。

Figure 2 : Comparison of scene flow approaches. Given a pair of point clouds, FlowNet3D [ 20 ] learns to regress the flow in the free 3D space, and the trained model is frozen for testing. FLOT [ 29 ] concurrently trains two network components: one that computes point correspondence and another that