[论文解读] Point Cloud Pre-training by Mixing and Disentangling

Chao Sun, Zhedong Zheng+3 more|arXiv (Cornell University)|Sep 1, 2021

3D Shape Modeling and Analysis参考文献 62被引用 7

一句话总结

本文提出了一种自监督预训练方法——混合与解耦（MD），用于点云处理，该方法通过从混合输入中重建原始点云来训练模型。通过采用简单的编码器-解码器架构并结合实例自适应解码，MD实现了快速收敛，并在分类与分割任务中表现出色，即使在标注数据有限的情况下，其性能也显著优于从零开始训练的模型。

ABSTRACT

The annotation for large-scale point clouds is still time-consuming and unavailable for many real-world tasks. Point cloud pre-training is one potential solution for obtaining a scalable model for fast adaptation. Therefore, in this paper, we investigate a new self-supervised learning approach, called Mixing and Disentangling (MD), for point cloud pre-training. As the name implies, we explore how to separate the original point cloud from the mixed point cloud, and leverage this challenging task as a pretext optimization objective for model training. Considering the limited training data in the original dataset, which is much less than prevailing ImageNet, the mixing process can efficiently generate more high-quality samples. We build one baseline network to verify our intuition, which simply contains two modules, encoder and decoder. Given a mixed point cloud, the encoder is first pre-trained to extract the semantic embedding. Then an instance-adaptive decoder is harnessed to disentangle the point clouds according to the embedding. Albeit simple, the encoder is inherently able to capture the point cloud keypoints after training and can be fast adapted to downstream tasks including classification and segmentation by the pre-training and fine-tuning paradigm. Extensive experiments on two datasets show that the encoder + ours (MD) significantly surpasses that of the encoder trained from scratch and converges quickly. In ablation studies, we further study the effect of each component and discuss the advantages of the proposed self-supervised learning strategy. We hope this self-supervised learning attempt on point clouds can pave the way for reducing the deeply-learned model dependence on large-scale labeled data and saving a lot of annotation costs in the future.

研究动机与目标

• 解决三维深度学习中大规模标注点云数据集稀缺的挑战。
• 通过开发一种自监督预训练方法，降低对昂贵人工标注数据的依赖。
• 通过一种新颖的掩蔽任务，提升模型泛化能力及下游任务（如分类、分割）的性能。
• 探索混合与解耦作为学习鲁棒点云表征的掩蔽任务的有效性。
• 证明基于简单编码器-解码器框架并结合MD预训练，可实现快速微调与优异性能。

提出的方法

• 提出一种自监督掩蔽任务，将两个点云混合形成复合输入。
• 训练编码器从混合点云中提取语义嵌入。
• 使用实例自适应解码器，基于编码后的嵌入从混合输入中解耦出原始点云。
• 通过解耦输出的重建损失，端到端训练编码器与解码器。
• 利用混合过程高效增强训练数据，尤其适用于大规模点云标注稀缺的情况。
• 通过微调将预训练编码器应用于下游任务，仅需少量标注数据。

实验结果

研究问题

• RQ1 混合与解耦点云能否作为自监督预训练的有效掩蔽任务？
• RQ2 MD方法在下游任务上的收敛速度与性能方面，相较于从零开始训练有何差异？
• RQ3 所提出方法在多大程度上能捕捉到有意义的点云特征（如关键点）？
• RQ4 MD框架的各个组成部分（如混合策略、实例自适应解码）对性能的贡献如何？
• RQ5 预训练模型是否能通过极少微调，良好泛化到多样化的3D任务（如分类与分割）？

主要发现

• MD预训练方法在分类与分割任务上均显著优于从零开始训练。
• 与随机初始化相比，预训练编码器在下游任务中收敛更快且准确率更高。
• 模型能够学习到点云中的关键点结构，表明其具备有效的特征学习能力。
• 消融实验证实，混合策略与实例自适应解码对性能提升均至关重要。
• 该方法通过混合生成高质量增强样本，有效缓解了数据稀缺问题。
• 编码器可快速微调以适应下游任务，展现出强大的迁移能力与可扩展性。