[Paper Review] Anomaly Detection in 3D Point Clouds using Deep Geometric Descriptors

Paul Bergmann, David Sattlegger|arXiv (Cornell University)|Feb 23, 2022

Remote Sensing and LiDAR Applications4 citations

TL;DR

This paper proposes 3D-ST, a novel unsupervised anomaly detection method for 3D point clouds that adapts the student-teacher framework to 3D geometry. By pretraining a teacher network via self-supervised reconstruction of local receptive fields, it learns dense geometric descriptors that enable high-resolution anomaly localization with state-of-the-art performance on the MVTec 3D-AD dataset.

ABSTRACT

We present a new method for the unsupervised detection of geometric anomalies in high-resolution 3D point clouds. In particular, we propose an adaptation of the established student-teacher anomaly detection framework to three dimensions. A student network is trained to match the output of a pretrained teacher network on anomaly-free point clouds. When applied to test data, regression errors between the teacher and the student allow reliable localization of anomalous structures. To construct an expressive teacher network that extracts dense local geometric descriptors, we introduce a novel self-supervised pretraining strategy. The teacher is trained by reconstructing local receptive fields and does not require annotations. Extensive experiments on the comprehensive MVTec 3D Anomaly Detection dataset highlight the effectiveness of our approach, which outperforms the next-best method by a large margin. Ablation studies show that our approach meets the requirements of practical applications regarding performance, runtime, and memory consumption.

Motivation & Objective

• Address the lack of effective unsupervised anomaly detection methods for high-resolution 3D point clouds in industrial and real-world applications.
• Overcome the limitation of existing 3D methods that rely on random weight initialization or require annotated data.
• Develop a pretraining protocol for 3D local geometric descriptors that generalizes well to anomaly detection without human annotations.
• Enable dense, high-resolution anomaly localization without intermediate downsampling or voxelization.
• Establish a new state-of-the-art benchmark for 3D anomaly detection using only anomaly-free training data.

Proposed method

• Adapt the 2D student-teacher anomaly detection framework to 3D by training a student network to regress the geometric descriptors of a pretrained teacher network on anomaly-free data.
• Design a self-supervised pretraining strategy for the teacher network that reconstructs local receptive fields in 3D point clouds without annotations.
• Use a local feature aggregation module to extract dense, pointwise geometric descriptors from each input point using a limited receptive field.
• Train the teacher network via a reconstruction head that predicts the original local geometry from the learned descriptors.
• Compute anomaly scores during inference as the L2 regression error between student-predicted and teacher-targeted descriptors for each 3D point.
• Enable inference on full-resolution point clouds by avoiding voxelization or subsampling, preserving spatial fidelity.

Experimental results

Research questions

• RQ1 Can a self-supervised pretraining strategy for 3D local geometric descriptors improve unsupervised anomaly detection performance in 3D point clouds?
• RQ2 Does the student-teacher framework adapted to 3D outperform existing methods trained from random initialization on the MVTec 3D-AD benchmark?
• RQ3 How robust is the method to variations in pretraining datasets and feature extractors used for initialization?
• RQ4 To what extent does the method maintain high performance, runtime efficiency, and memory efficiency in practical deployment scenarios?
• RQ5 Can dense, high-resolution anomaly localization be achieved without intermediate downsampling or voxelization?

Key findings

• The proposed 3D-ST method achieves state-of-the-art performance on the MVTec 3D Anomaly Detection dataset, outperforming the next-best method by a large margin.
• Ablation studies confirm that the method is computationally efficient, with low memory consumption and fast inference, suitable for real-world deployment.
• The self-supervised pretraining strategy enables the teacher network to learn generic, transferable geometric descriptors that generalize well across different datasets.
• Performance remains stable across different pretraining datasets (e.g., ITODD, 3DMatch), with only a slight drop on KITTI due to domain shift, indicating robustness.
• Replacing the pretrained teacher with features from PPF-FoldNet or FCGF yields better results than training from scratch, but still underperforms the proposed method.
• The method localizes anomalies accurately at the point level, as shown by qualitative results on the MVTec 3D-AD dataset, with high precision in identifying geometric deviations.