[论文解读] Learning Representations and Generative Models for 3D Point Clouds

Panos Achlioptas, Olga Diamanti+2 more|arXiv (Cornell University)|Jul 8, 2017

3D Shape Modeling and Analysis被引用 538

一句话总结

论文开发了一个深度自编码器，用于学习3D点云的紧凑潜在表示，并在该潜在空间中研究了几种生成模型（原始点GANs、潜在空间GANs和高斯混合模型），并给出新的保真度/覆盖度指标，显示潜在空间GMMs往往表现最好。

ABSTRACT

Three-dimensional geometric data offer an excellent domain for studying representation learning and generative modeling. In this paper, we look at geometric data represented as point clouds. We introduce a deep AutoEncoder (AE) network with state-of-the-art reconstruction quality and generalization ability. The learned representations outperform existing methods on 3D recognition tasks and enable shape editing via simple algebraic manipulations, such as semantic part editing, shape analogies and shape interpolation, as well as shape completion. We perform a thorough study of different generative models including GANs operating on the raw point clouds, significantly improved GANs trained in the fixed latent space of our AEs, and Gaussian Mixture Models (GMMs). To quantitatively evaluate generative models we introduce measures of sample fidelity and diversity based on matchings between sets of point clouds. Interestingly, our evaluation of generalization, fidelity and diversity reveals that GMMs trained in the latent space of our AEs yield the best results overall.

研究动机与目标

• Develop an autoencoder (AE) architecture that yields high reconstruction quality and strong generalization for 3D point clouds.
• Enable semantic operations in latent space such as interpolation, shape editing, and completion.
• Investigate and compare generative models (r-GAN, l-GAN, and GMMs) for point clouds with robust evaluation metrics.
• Propose and validate metrics for fidelity, coverage, and diversity of generated point clouds.

提出的方法

• Design a 3D point cloud autoencoder operating on 2048-point inputs with a 128-dimensional latent bottleneck.
• Use permutation-invariant losses (EMD or Chamfer distance) as reconstruction objectives (AE-EMD and AE-CD).
• Train a raw-point GAN (r-GAN) directly on 2048x3 point clouds.
• Train latent-space GANs (l-GAN) in the AE latent space, decoding with the AE decoder to produce point clouds.
• Fit Gaussian Mixture Models (GMMs) in the AE latent space and generate samples via the decoder.
• Introduce evaluation metrics for generative models: Jensen-Shannon Divergence (JSD), Coverage (COV-CD/EMD), and Minimum Matching Distance (MMD-CD/EMD).
• Perform extensive experiments on ShapeNet data, comparing class-specific and multi-class settings, and analyze Chamfer vs. EMD fidelity.
• Demonstrate shape editing, interpolation, and completion tasks in the AE latent space.

实验结果

研究问题

• RQ1 How well can a deep autoencoder learn compact and meaningful latent representations for 3D point clouds?
• RQ2 Which generative models (r-GAN, l-GAN, GMM) in the latent space or on raw data provide the best fidelity and coverage for point clouds?
• RQ3 Do latent-space models enable meaningful semantic manipulations and shape completion for 3D objects?
• RQ4 How do different point-cloud reconstruction/evaluation metrics (EMD vs. Chamfer) behave in practice for generative tasks?
• RQ5 Is a simple Gaussian Mixture Model in the AE latent space competitive with adversarial approaches for 3D point cloud generation?

主要发现

• The autoencoder achieves good generalization to unseen shapes with small MMD-CD/MMD-EMD gaps between train and test.
• Latent representations enable semantic operations such as interpolation and attribute manipulation, and support competitive 3D object classification via linear SVMs.
• Latent-space GANs improve fidelity and coverage over raw-point GANs, but can suffer from mode collapse; WGAN approaches mitigate some issues.
• Gaussian Mixture Models in the AE latent space achieve strong fidelity and competitive coverage, often rivaling or surpassing adversarial models in this setting.
• Chamfer distance can be misleading for evaluating generated point clouds, whereas EMD-based metrics align better with visual fidelity and diversity.
• Voxel-based generators underperform compared to point-cloud–centered approaches in terms of fidelity and coverage for the same object classes.