[Paper Review] Multiresolution Tree Networks for 3D Point Cloud Processing

Matheus Gadelha, Rui Wang+1 more|arXiv (Cornell University)|Jul 10, 2018

3D Shape Modeling and Analysis4 references42 citations

TL;DR

MRTNet introduces a multiresolution, tree-structured network for processing 3D point clouds, enabling efficient, accurate shape classification and direct point-cloud generation, with strong performance on ModelNet40 and ShapeNet tasks.

ABSTRACT

We present multiresolution tree-structured networks to process point clouds for 3D shape understanding and generation tasks. Our network represents a 3D shape as a set of locality-preserving 1D ordered list of points at multiple resolutions. This allows efficient feed-forward processing through 1D convolutions, coarse-to-fine analysis through a multi-grid architecture, and it leads to faster convergence and small memory footprint during training. The proposed tree-structured encoders can be used to classify shapes and outperform existing point-based architectures on shape classification benchmarks, while tree-structured decoders can be used for generating point clouds directly and they outperform existing approaches for image-to-shape inference tasks learned using the ShapeNet dataset. Our model also allows unsupervised learning of point-cloud based shapes by using a variational autoencoder, leading to higher-quality generated shapes.

Motivation & Objective

• Develop a multiresolution, tree-structured network to process 3D point clouds directly without voxelization or view-based surrogates.
• Enable accurate shape classification, image-to-shape inference, and unsupervised shape learning via an encoder–decoder (and VAE) framework.
• Maintain locality and scalability through a 3-scale, multi-grid architecture that preserves 3D structure in 1D point orderings.
• Investigate the benefits of multiresolution processing for fast convergence and reduced memory footprint during training.

Proposed method

• Represent a 3D shape as a locality-preserving 1D ordered list of points at multiple resolutions using a space-partitioning tree (KD-tree or RP-tree).
• Process the 1D point sequence with 1D convolutions, incorporating a three-scale, multi-grid MR-CONV block that fuses information across resolutions via upsampling and pooling.
• Use an encoder to produce a latent z (512-D) for tasks like classification; attach a fully connected layer for ModelNet40 classification.
• Employ a multi-resolution decoder (MR-CONV-T blocks) to generate point clouds directly, allowing Chamfer distance-based losses.
• Optionally combine with a pretrained image encoder (VGG-11) for image-to-shape inference, training with Chamfer distance as reconstruction loss.
• Extend to unsupervised learning with a MR-VAE where the encoder outputs z and the decoder reconstructs point clouds, with a moment-matching regularizer to Gaussian N(0,I).

Experimental results

Research questions

• RQ1 Can a multiresolution, tree-structured representation of point clouds improve classification accuracy compared to single-scale point-based models?
• RQ2 How well can MRTNet encode and decode point clouds to enable direct shape generation and image-to-shape inference without voxelization or view-based synthesis?
• RQ3 Does a multiresolution architecture improve convergence speed and memory efficiency during training?
• RQ4 Can unsupervised learning via MR-VAE produce high-quality, latent-shape representations useful for downstream tasks?
• RQ5 How does MRTNet perform on shape segmentation and cross-domain generation (e.g., image-to-3D) relative to prior approaches?

Key findings

• MRTNet achieves 91.7% accuracy on ModelNet40 when using 4K points, outperforming several point-based methods that use XYZ data only.
• With 1K points MRTNet reaches 91.2% accuracy, surpassing baseline single-resolution variants and many prior point-based architectures.
• The multi-resolution decoder enables direct point-cloud generation, delivering higher-quality shapes than several competing image-to-shape approaches on ShapeNet, using Chamfer distance as the loss.
• Unsupervised MR-VAE training yields 86.4% accuracy on ModelNet40 when using features from the MR-VAE encoder for downstream classification, indicating strong learned representations.
• MR-Net variants demonstrate faster convergence and lower memory usage compared to single-scale baselines, while maintaining or improving accuracy across tasks.
• Qualitative results show MRTNet preserves coherent spatial structure in generated point clouds and can interpolate between shapes in latent space.