[Paper Review] Adaptive O-CNN: A Patch-based Deep Representation of 3D Shapes

Peng‐Shuai Wang, Chunyu Sun+2 more|arXiv (Cornell University)|Sep 21, 2018

3D Shape Modeling and AnalysisEngineering51 references137 citations

TL;DR

Adaptive O-CNN encodes 3D shapes with a patch-guided adaptive octree, enabling efficient high-resolution 3D shape encoding/decoding and improved shape generation compared to prior octree/CNN approaches. It achieves memory and time savings with competitive accuracy across tasks including classification, autoencoding, single-image shape prediction, and completion.

ABSTRACT

We present an Adaptive Octree-based Convolutional Neural Network (Adaptive O-CNN) for efficient 3D shape encoding and decoding. Different from volumetric-based or octree-based CNN methods that represent a 3D shape with voxels in the same resolution, our method represents a 3D shape adaptively with octants at different levels and models the 3D shape within each octant with a planar patch. Based on this adaptive patch-based representation, we propose an Adaptive O-CNN encoder and decoder for encoding and decoding 3D shapes. The Adaptive O-CNN encoder takes the planar patch normal and displacement as input and performs 3D convolutions only at the octants at each level, while the Adaptive O-CNN decoder infers the shape occupancy and subdivision status of octants at each level and estimates the best plane normal and displacement for each leaf octant. As a general framework for 3D shape analysis and generation, the Adaptive O-CNN not only reduces the memory and computational cost, but also offers better shape generation capability than the existing 3D-CNN approaches. We validate Adaptive O-CNN in terms of efficiency and effectiveness on different shape analysis and generation tasks, including shape classification, 3D autoencoding, shape prediction from a single image, and shape completion for noisy and incomplete point clouds.

Motivation & Objective

• Motivate and enable efficient, high-fidelity 3D shape representation and generation beyond uniform-resolution voxel or fixed-octree methods.
• Develop an encoder that processes planar patch normals and displacements across adaptive octree levels.
• Develop a decoder that predicts octant occupancy status and learns local planar patches to reconstruct shapes.
• Demonstrate efficiency and effectiveness on multiple downstream tasks including classification, autoencoding, and single-image shape prediction.
• Show that adaptive patch-based octrees can achieve high-quality shapes with lower memory and computation than comparable methods.

Proposed method

• Introduce a patch-guided adaptive octree representation for 3D shapes where each non-empty leaf octant stores a planar patch.
• Define a Hausdorff-distance-based patch approximation error to guide octree subdivision and ensure bounded approximation quality.
• Extend an O-CNN-inspired encoder to operate on adaptive octrees by incorporating inputs from local plane parameters into a multi-level CNN processing scheme.
• Design a 3D decoder that (i) predicts octant statuses (empty, surface-well-approximated, surface-poorly-approximated) and (ii) regresses planar patch parameters for leaf octants, with deconvolution to refine descendants.
• Employ a joint loss combining a structure loss (cross-entropy over octant statuses) and a patch loss (plane parameter regression with an explicit d* constraint).
• Adapt prior octree CNN infrastructure to support bottom-up leaf-level feature aggregation and top-down patch-based decoding, achieving efficiency gains.

Experimental results

Research questions

• RQ1 Can a patch-guided adaptive octree representation yield high-quality 3D shapes with reduced memory and computation compared to uniform voxel or fixed-depth octrees?
• RQ2 Does incorporating planar patch parameters at leaf octants improve surface fidelity and detail in reconstructed shapes?
• RQ3 How well does the Adaptive O-CNN perform on shape classification, 3D autoencoding, and single-image shape prediction tasks relative to state-of-the-art methods?
• RQ4 What are the trade-offs in accuracy versus efficiency as octree depth and patch fidelity are varied?
• RQ5 Can the proposed decoder reliably infer octant occupancy and adaptive subdivision to produce watertight, high-quality surfaces?

Key findings

• Adaptive O-CNN achieves comparable classification accuracy to O-CNN across resolutions while offering substantially lower memory and computation costs, especially at high resolutions (e.g., memory and time savings noted for 2563 vs 1283 inputs).
• On ModelNet40 classification, Adaptive O-CNN is competitive with state-of-the-art methods, exhibiting similar accuracy to O-CNN and plateaus near other strong baselines.
• In 3D autoencoding on ShapeNet Core v2, Adaptive O-CNN achieves the best Chamfer-distance performance among compared methods (AtlasNet variants and PSG) on average, indicating higher surface fidelity and volume structure reconstruction for many categories.
• The decoder’s joint structure and patch losses enable better handling of thin parts and preserve volume structure, reducing distortions seen with patch-based or pure Chamfer-distance-driven approaches.
• The method demonstrates shape prediction from a single image and shape completion for noisy/incomplete point clouds, validating the patch-guided adaptive octree as a general framework for 3D shape analysis and generation.
• Compared to prior octree-based methods, the patch-guided adaptation yields higher sparsity and efficiency without sacrificing qualitative or quantitative shape quality.