[Paper Review] Benchmarking and Analyzing Point Cloud Classification under Corruptions

Jiawei Ren, Liang Pan+1 more|arXiv (Cornell University)|Feb 7, 2022

3D Surveying and Cultural Heritage31 citations

TL;DR

This paper introduces ModelNet-C, a comprehensive corruption benchmark for point cloud classification, analyzes robustness across architectures, and proposes RPC and WOLFMix to improve robustness.

ABSTRACT

3D perception, especially point cloud classification, has achieved substantial progress. However, in real-world deployment, point cloud corruptions are inevitable due to the scene complexity, sensor inaccuracy, and processing imprecision. In this work, we aim to rigorously benchmark and analyze point cloud classification under corruptions. To conduct a systematic investigation, we first provide a taxonomy of common 3D corruptions and identify the atomic corruptions. Then, we perform a comprehensive evaluation on a wide range of representative point cloud models to understand their robustness and generalizability. Our benchmark results show that although point cloud classification performance improves over time, the state-of-the-art methods are on the verge of being less robust. Based on the obtained observations, we propose several effective techniques to enhance point cloud classifier robustness. We hope our comprehensive benchmark, in-depth analysis, and proposed techniques could spark future research in robust 3D perception.

Motivation & Objective

• Motivate robust point cloud classification for real-world deployments under sensor and processing corruptions.
• Provide a standardized, comprehensive corruption test suite for ModelNet40-based evaluation.
• Systematically benchmark a wide range of architectures, pretraining, and augmentations for robustness.
• Identify design principles and augmentation strategies that improve corruption robustness.

Proposed method

• Define a taxonomy of seven atomic corruptions (Add Global, Add Local, Drop Global, Drop Local, Rotate, Scale, Jitter).
• Create ModelNet-C by applying severity levels to these corruptions on ModelNet40 test set.
• Evaluate 14 methods (9 architectures, 3 pretrains, 2 augmentations) under corruption scenarios with mean Corruption Error (mCE).
• Analyze architecture components (local operations, advanced grouping, featurizer) and their impact on robustness.
• Examine self-supervised pretraining (OcCo, Point-BERT) and augmentation strategies (PointMixUp, RSMix, PointWOLF) on robustness.
• Propose RPC (Robust Point Cloud Classifier) and WOLFMix augmentations as robustness-enhancing approaches.

Experimental results

Research questions

• RQ1 How robust are current point cloud classifiers to realistic corruptions across object-, sensor-, and processing-level artefacts?
• RQ2 What architectural choices and training augmentations most improve robustness to corruptions?
• RQ3 Can standardized corruption benchmarks like ModelNet-C reveal robustness gaps not evident on clean data?
• RQ4 Do self-supervised pretraining and novel augmentation strategies transfer robustness benefits across architectures?
• RQ5 What is the interaction (synergy) between architecture design and augmentation for robustness?

Key findings

• ModelNet-C provides a comprehensive, standardized suite covering seven atomic corruptions across five severity levels.
• State-of-the-art models on clean data show notable fragility under common corruptions (non-negligible mCE variance).
• Architectural choices such as 3D representations with k-NN, frequency grouping, and self-attention (RPC) yield the best standalone robustness (lowest mCE).
• Advanced groupings (frequency and curve) consistently improve robustness but add computation overhead.
• Self-supervised pretraining offers targeted robustness benefits for certain corruptions (e.g., rotation with OcCo; local occlusion with Point-BERT).
• WOLFMix, a two-stage augmentation (deform then mix), provides substantial robustness gains across architectures, with synergy varying by model.