[Paper Review] Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline

Ankit Goyal, Hei Law+3 more|arXiv (Cornell University)|Jun 9, 2021

3D Shape Modeling and Analysis66 references59 citations

TL;DR

The paper shows that evaluation protocol factors largely drive point-cloud classification performance and introduces SimpleView, a simple projection-based baseline that matches or exceeds state-of-the-art results under controlled protocols while using fewer parameters.

ABSTRACT

Processing point cloud data is an important component of many real-world systems. As such, a wide variety of point-based approaches have been proposed, reporting steady benchmark improvements over time. We study the key ingredients of this progress and uncover two critical results. First, we find that auxiliary factors like different evaluation schemes, data augmentation strategies, and loss functions, which are independent of the model architecture, make a large difference in performance. The differences are large enough that they obscure the effect of architecture. When these factors are controlled for, PointNet++, a relatively older network, performs competitively with recent methods. Second, a very simple projection-based method, which we refer to as SimpleView, performs surprisingly well. It achieves on par or better results than sophisticated state-of-the-art methods on ModelNet40 while being half the size of PointNet++. It also outperforms state-of-the-art methods on ScanObjectNN, a real-world point cloud benchmark, and demonstrates better cross-dataset generalization. Code is available at https://github.com/princeton-vl/SimpleView.

Motivation & Objective

• Assess how protocol factors (evaluation scheme, data augmentation, losses) influence point-cloud classification performance independent of architecture.
• Reevaluate common architectures (PointNet, PointNet++, DGCNN, RSCNN) under controlled protocols.
• Introduce and evaluate SimpleView, a simple projection-based baseline.
• Demonstrate SimpleView's performance on ModelNet40 and real-world ScanObjectNN and its generalization across datasets.

Proposed method

• Define and analyze a protocol framework that isolates architecture from controllable factors such as augmentation and loss functions.
• Reproduce and compare PointNet, PointNet++, DGCNN, and RSCNN under standardized protocols.
• Implement SimpleView, a projection-based baseline that converts 3D points to depth maps on orthogonal planes and processes them with a lightweight CNN (ResNet18/4).
• Use fixed 1024 points per object, non-use of ImageNet pretraining, and non-ensembled predictions for fair comparisons.
• Conduct ablations on number of views, projection type (perspective/orthographic), depth computation (min vs weighted average), and feature fusion (concat vs pooling).
• Perform experiments on ModelNet40 and ScanObjectNN, including cross-dataset transfer tests.

Experimental results

Research questions

• RQ1 How much do protocol choices (augmentation, loss, evaluation protocol) affect reported performance across point-cloud classification methods?
• RQ2 Can a simple projection-based baseline match or surpass state-of-the-art point-based networks when protocols are controlled?
• RQ3 How does SimpleView perform on ModelNet40 and real-world ScanObjectNN, and does it generalize across datasets?

Key findings

• Protocol choices account for large performance differences, potentially obscuring architectural gains.
• When controlling for protocol, PointNet++ is competitive with newer architectures.
• SimpleView achieves on-par or better results than state-of-the-art methods on ModelNet40 and outperforms them on ScanObjectNN.
• SimpleView uses fewer parameters than competing networks and does not rely on ImageNet pretraining.
• SimpleView demonstrates better cross-dataset generalization compared to prior methods.