[Paper Review] PointAugment: an Auto-Augmentation Framework for Point Cloud Classification

Ruihui Li, Xianzhi Li+2 more|arXiv (Cornell University)|Feb 25, 2020

3D Shape Modeling and AnalysisEngineering53 references18 citations

TL;DR

PointAugment is an end-to-end, sample-aware auto-augmentation framework for 3D point cloud classification that jointly optimizes an augmentor and classifier via adversarial learning. It learns shape-wise transformations and point-wise displacements tailored to individual samples, significantly improving classification accuracy across multiple networks on ModelNet40 and SHREC16 by dynamically adapting augmentation based on classifier feedback.

ABSTRACT

We present PointAugment, a new auto-augmentation framework that automatically optimizes and augments point cloud samples to enrich the data diversity when we train a classification network. Different from existing auto-augmentation methods for 2D images, PointAugment is sample-aware and takes an adversarial learning strategy to jointly optimize an augmentor network and a classifier network, such that the augmentor can learn to produce augmented samples that best fit the classifier. Moreover, we formulate a learnable point augmentation function with a shape-wise transformation and a point-wise displacement, and carefully design loss functions to adopt the augmented samples based on the learning progress of the classifier. Extensive experiments also confirm PointAugment's effectiveness and robustness to improve the performance of various networks on shape classification and retrieval.

Motivation & Objective

• To address the limited diversity and small scale of 3D point cloud datasets, which hinder deep network generalization.
• To overcome the suboptimal performance of conventional data augmentation methods that apply fixed, non-adaptive transformations to all samples.
• To develop an end-to-end framework that jointly optimizes data augmentation and classification training for improved robustness and accuracy.
• To enable the augmentor to generate sample-specific, adaptive augmentations that maximize classifier performance across training stages.

Proposed method

• The framework employs an adversarial learning strategy to jointly train an augmentor network and a classifier network in an end-to-end manner.
• The augmentor learns a shape-wise transformation matrix and a point-wise displacement vector from per-point features using shared MLPs.
• A learnable augmentation function combines global transformation and local displacement to generate diverse, realistic point cloud variations.
• A dynamic loss function balances classifier and augmentor objectives, with a hyperparameter λ that adjusts augmentation magnitude based on classifier learning progress.
• The augmentor uses feedback from classifier loss to refine its augmentation policy, ensuring generated samples challenge the classifier without overfitting.
• The method is modular and compatible with various backbone networks, including PointNet, PointNet++, DGCNN, and RSCNN.

Experimental results

Research questions

• RQ1 Can an auto-augmentation framework that jointly optimizes augmentation and classification improve 3D point cloud classification beyond fixed-data-augmentation baselines?
• RQ2 How does sample-aware augmentation—where augmentation is tailored to individual point cloud complexity—impact model generalization and accuracy?
• RQ3 What is the optimal trade-off between augmentation strength and classifier learning progress, and how can it be dynamically controlled during training?
• RQ4 How does the proposed framework perform across diverse network architectures and benchmark datasets?

Key findings

• PointAugment achieves consistent accuracy gains across four state-of-the-art networks on ModelNet40, with improvements ranging from 0.8% to 1.4% over conventional data augmentation.
• On ModelNet40, PointNet++ achieves 92.9% accuracy with PointAugment, compared to 92.1% without, demonstrating significant performance gains.
• The framework improves performance on the SHREC16 retrieval benchmark, confirming its effectiveness beyond classification.
• Ablation studies show that using MLPs for feature extraction yields better performance than more complex units like EdgeConv or DenseConv, due to reduced overfitting risk.
• The optimal hyperparameter λ = 1.0 balances classifier and augmentor objectives, as values too low or too high degrade performance.
• The framework is robust to different feature extraction units and maintains strong generalization, indicating its modular and adaptable design.