[Paper Review] Refine-Net: Normal Refinement Neural Network for Noisy Point Clouds

Haoran Zhou, Honghua Chen+7 more|arXiv (Cornell University)|Mar 23, 2022

3D Shape Modeling and Analysis4 citations

TL;DR

Refine-Net is a deep learning framework that refines noisy point cloud normals by fusing multiple geometric feature representations—such as local coordinates, height distances, and curvature—through a novel connection module. It achieves state-of-the-art performance in normal estimation and denoising, significantly improving sharp feature preservation and noise removal on both synthetic and real-scanned datasets.

ABSTRACT

Point normal, as an intrinsic geometric property of 3D objects, not only serves conventional geometric tasks such as surface consolidation and reconstruction, but also facilitates cutting-edge learning-based techniques for shape analysis and generation. In this paper, we propose a normal refinement network, called Refine-Net, to predict accurate normals for noisy point clouds. Traditional normal estimation wisdom heavily depends on priors such as surface shapes or noise distributions, while learning-based solutions settle for single types of hand-crafted features. Differently, our network is designed to refine the initial normal of each point by extracting additional information from multiple feature representations. To this end, several feature modules are developed and incorporated into Refine-Net by a novel connection module. Besides the overall network architecture of Refine-Net, we propose a new multi-scale fitting patch selection scheme for the initial normal estimation, by absorbing geometry domain knowledge. Also, Refine-Net is a generic normal estimation framework: 1) point normals obtained from other methods can be further refined, and 2) any feature module related to the surface geometric structures can be potentially integrated into the framework. Qualitative and quantitative evaluations demonstrate the clear superiority of Refine-Net over the state-of-the-arts on both synthetic and real-scanned datasets. Our code is available at https://github.com/hrzhou2/refinenet.

Motivation & Objective

• To address the challenge of accurate normal estimation in noisy point clouds where traditional methods rely on rigid priors or single-feature inputs.
• To develop a generic, flexible framework that can refine initial normals from any source, including other learning-based methods.
• To enhance geometric fidelity by integrating diverse feature representations—such as local coordinates, grid-like height distances, and curvature—into a unified refinement process.
• To design a multi-scale fitting patch selection (MFPS) scheme that improves initial normal estimation using geometric domain knowledge.
• To enable robust performance across diverse noise distributions and complex surface geometries, including sharp edges and flat regions.

Proposed method

• Refine-Net employs a multi-scale and multi-feature refinement architecture that processes initial normals by extracting and fusing features from different geometric domains.
• A novel connection module integrates multiple feature modules—such as point coordinates, height distance grids, and curvature features—into a shared representation space to jointly refine normals.
• The multi-scale fitting patch selection (MFPS) scheme selects optimal local patches for initial normal estimation by leveraging geometric priors, improving robustness to noise and surface complexity.
• A cluster-based training scheme divides input point clouds into Kc = 4 clusters to enhance feature learning, especially on geometrically challenging regions like edges and corners.
• The framework supports plug-and-play integration of new feature modules, making it extensible for future geometric priors.
• A point-updating denoising algorithm guided by refined normals is used to validate performance, with updates based on normal consistency and spatial proximity.

Experimental results

Research questions

• RQ1 Can a deep learning framework effectively refine noisy point cloud normals by combining multiple geometric feature representations?
• RQ2 How does multi-scale and multi-feature fusion improve normal estimation compared to single-feature or single-scale approaches?
• RQ3 To what extent can Refine-Net improve suboptimal initial normals from other methods, especially in preserving sharp edges and removing noise?
• RQ4 Does the proposed MFPS scheme for initial normal estimation lead to better downstream performance than standard neighborhood fitting?
• RQ5 Can Refine-Net be generalized across diverse noise types and real-world scanning conditions?

Key findings

• Refine-Net achieves state-of-the-art performance on both synthetic and real-scanned datasets, outperforming existing methods in normal estimation accuracy and robustness.
• The framework significantly improves normal quality on MFPS-initialized normals, particularly in recovering sharp edges and reducing perturbations in flat regions.
• Using only point coordinates as input, Refine-Net still produces high-quality denoised normals, but performance improves substantially with multi-feature fusion.
• The cluster-based training scheme enhances feature learning on complex geometric structures, with minimal performance gain beyond Kc = 4 clusters.
• Refine-Net enables superior point cloud denoising and surface reconstruction when used as a normal guidance module, as validated by iterative point updating.
• The method is generic and extensible: it can refine normals from any source and integrate new geometric feature modules, demonstrating broad applicability.