[Paper Review] 3D Keypoint Detection Based on Deep Neural Network with Sparse Autoencoder

Xinyu Lin, Ce Zhu+2 more|arXiv (Cornell University)|Apr 30, 2016

3D Shape Modeling and Analysis36 references3 citations

TL;DR

This paper proposes a novel 3D keypoint detection method using a deep neural network (DNN) with sparse autoencoders (SAE) to formulate a regression model that leverages both local geometric features and global Laplacian spectrum information across multi-scale representations. The approach outperforms five state-of-the-art methods in keypoint detection accuracy, particularly under higher localization error tolerances, with an average IOU of 0.275 on Dataset A and 0.272 on Dataset B.

ABSTRACT

Researchers have proposed various methods to extract 3D keypoints from the surface of 3D mesh models over the last decades, but most of them are based on geometric methods, which lack enough flexibility to meet the requirements for various applications. In this paper, we propose a new method on the basis of deep learning by formulating the 3D keypoint detection as a regression problem using deep neural network (DNN) with sparse autoencoder (SAE) as our regression model. Both local information and global information of a 3D mesh model in multi-scale space are fully utilized to detect whether a vertex is a keypoint or not. SAE can effectively extract the internal structure of these two kinds of information and formulate high-level features for them, which is beneficial to the regression model. Three SAEs are used to formulate the hidden layers of the DNN and then a logistic regression layer is trained to process the high-level features produced in the third SAE. Numerical experiments show that the proposed DNN based 3D keypoint detection algorithm outperforms current five state-of-the-art methods for various 3D mesh models.

Motivation & Objective

• To address the limitations of traditional geometric methods in 3D keypoint detection, such as inflexibility and sensitivity to noise and local variations.
• To develop a learning-based framework that can adaptively detect keypoints with high recall and precision across diverse 3D mesh models.
• To integrate both local geometric properties and global spectral features (Laplacian spectrum) into a unified deep learning model for improved detection robustness.
• To demonstrate the superiority of the proposed DNN with SAE over existing state-of-the-art methods using standardized evaluation metrics.

Proposed method

• The method formulates 3D keypoint detection as a regression problem using a deep neural network (DNN) with three stacked sparse autoencoders (SAEs) as the hidden layers.
• Local features are extracted from three geometric properties: Euclidean distance to the tangent plane, normal vector angles, and various curvatures in neighborhood rings.
• Global features are derived from the log-Laplacian spectrum of the 3D mesh model, capturing shape-level structural information.
• The SAEs learn hierarchical, high-level representations of both local and global features, enabling effective abstraction of complex patterns.
• A final logistic regression layer processes the high-level features from the third SAE to predict whether each vertex is a keypoint.
• The entire DNN is trained end-to-end using labeled 3D mesh data, with optimization focused on minimizing regression loss.

Experimental results

Research questions

• RQ1 Can a deep neural network with sparse autoencoders effectively learn and fuse multi-scale local and global features for 3D keypoint detection?
• RQ2 Does the proposed method outperform existing geometric and machine learning-based 3D keypoint detection techniques in terms of detection accuracy and robustness?
• RQ3 How does the integration of Laplacian spectrum features improve the generalization and stability of keypoint detection across diverse 3D models?
• RQ4 To what extent does the model maintain performance under varying localization error tolerances?

Key findings

• The proposed DNN with SAE achieves the highest average Intersection over Union (IOU) score of 0.275 on Dataset A and 0.272 on Dataset B, outperforming five state-of-the-art methods.
• The method shows superior performance under higher localization error tolerances (r), particularly when r is large, indicating robustness to localization uncertainty.
• The FNE (False Negative Error) and FPE (False Positive Error) metrics are significantly lower than those of competing methods, with FNE-A of 0.509 and FPE-A of 0.561 on Dataset A.
• The WME (Weighted Mean Error) metric is also minimized, with values of 0.484 on Dataset A and 0.490 on Dataset B, indicating better alignment with human visual perception.
• The detected keypoints are stable across different mesh models and show strong consistency with human-annotated salient features, especially in complex regions like shoulders and joints.