[Paper Review] 3D Object Classification via Spherical Projections

Zhangjie Cao, Qixing Huang+1 more|arXiv (Cornell University)|Dec 12, 2017

3D Shape Modeling and Analysis24 references3 citations

TL;DR

This paper proposes a novel 3D object classification method using spherical projections to combine the strengths of image-based and 3D-based deep learning. By projecting 3D objects onto a sphere centered at their barycenter, the approach enables high-resolution feature learning and pre-training on ImageNet while capturing global cross-view dependencies through two complementary projections: depth variation and contour information. The method achieves state-of-the-art performance on ModelNet40 and ShapeNetCore benchmarks.

ABSTRACT

In this paper, we introduce a new method for classifying 3D objects. Our main idea is to project a 3D object onto a spherical domain centered around its barycenter and develop neural network to classify the spherical projection. We introduce two complementary projections. The first captures depth variations of a 3D object, and the second captures contour-information viewed from different angles. Spherical projections combine key advantages of two main-stream 3D classification methods: image-based and 3D-based. Specifically, spherical projections are locally planar, allowing us to use massive image datasets (e.g, ImageNet) for pre-training. Also spherical projections are similar to voxel-based methods, as they encode complete information of a 3D object in a single neural network capturing dependencies across different views. Our novel network design can fully utilize these advantages. Experimental results on ModelNet40 and ShapeNetCore show that our method is superior to prior methods.

Motivation & Objective

• To address the limitations of existing 3D classification methods that either sacrifice resolution (3D-based) or fail to capture cross-view dependencies (image-based).
• To leverage the advantages of image-based methods—high-resolution processing and ImageNet pre-training—while overcoming their view-dependent and discontinuous projection issues.
• To develop a spherical projection framework that encodes complete 3D shape information in a continuous, globally consistent representation.
• To design a neural network architecture that performs convolutions on cylindrical patches of the spherical projection, enabling use of pre-trained 2D CNNs.

Proposed method

• Project 3D objects onto a spherical domain centered at their barycenter to create a continuous, globally consistent representation.
• Introduce two complementary spherical projections: one capturing depth variation across viewpoints, and another capturing contour information from different angles.
• Define cylindrical patches on the spherical surface to enable standard 2D convolution operations, minimizing patch count while preserving cross-view dependency capture.
• Use pre-trained ImageNet models to extract features from the spherical projections, enabling transfer learning and high-resolution feature learning.
• Design a network architecture that processes both depth-based and contour-based spherical projections using shared or parallel branches for improved generalization.
• Apply a principled sampling strategy for cylindrical patches to reduce computational cost while maintaining geometric fidelity and view-invariance.

Experimental results

Research questions

• RQ1 Can spherical projections effectively combine the high-resolution capability of image-based methods with the global consistency of 3D-based methods for 3D classification?
• RQ2 How can spherical projections be designed to preserve both depth and contour information across multiple viewpoints?
• RQ3 To what extent can pre-trained 2D CNNs on ImageNet be leveraged for 3D classification via spherical projections?
• RQ4 How do different projection resolutions and patch sampling strategies affect classification accuracy and computational efficiency?
• RQ5 Can the proposed method achieve view-invariant classification without explicit data augmentation or orientation estimation?

Key findings

• The proposed method achieves 90.49% accuracy on ModelNet40, outperforming prior state-of-the-art methods.
• On ShapeNetCore, the method attains 87.54% accuracy, demonstrating strong generalization across diverse 3D object categories.
• Reducing the depth projection resolution from 30° to 15° and 60° results in a 0.2% and 0.5% drop in accuracy, respectively, justifying the use of 30° for efficiency.
• Increasing the contour projection grid from 3×6 to 3×12 improves accuracy by ~1.0%, but further increase to 3×24 yields less than 0.2% improvement.
• The method is robust to elevation degree variations in horizontal strips, with performance stabilizing beyond 60°, where distortion reduces effectiveness of pre-trained models.
• Rendering and inference for a single 3D object take approximately 1.0–1.3 seconds, with training times around 4–7 hours on a single GPU, dominated by projection generation.