[Paper Review] Variational Relational Point Completion Network

Liang Pan, Xinyi Chen+5 more|arXiv (Cornell University)|Apr 20, 2021

3D Shape Modeling and Analysis30 references4 citations

TL;DR

This paper proposes VRCNet, a variational relational point completion network that combines probabilistic modeling via a dual-path architecture with relational feature learning using point self-attention and selective kernel modules to generate detailed, structurally coherent 3D shapes from incomplete point clouds. VRCNet achieves state-of-the-art performance on standard benchmarks and generalizes robustly to real-world scans, outperforming prior methods in shape detail recovery and structural consistency.

ABSTRACT

Real-scanned point clouds are often incomplete due to viewpoint, occlusion, and noise. Existing point cloud completion methods tend to generate global shape skeletons and hence lack fine local details. Furthermore, they mostly learn a deterministic partial-to-complete mapping, but overlook structural relations in man-made objects. To tackle these challenges, this paper proposes a variational framework, Variational Relational point Completion network (VRCNet) with two appealing properties: 1) Probabilistic Modeling. In particular, we propose a dual-path architecture to enable principled probabilistic modeling across partial and complete clouds. One path consumes complete point clouds for reconstruction by learning a point VAE. The other path generates complete shapes for partial point clouds, whose embedded distribution is guided by distribution obtained from the reconstruction path during training. 2) Relational Enhancement. Specifically, we carefully design point self-attention kernel and point selective kernel module to exploit relational point features, which refines local shape details conditioned on the coarse completion. In addition, we contribute a multi-view partial point cloud dataset (MVP dataset) containing over 100,000 high-quality scans, which renders partial 3D shapes from 26 uniformly distributed camera poses for each 3D CAD model. Extensive experiments demonstrate that VRCNet outperforms state-of-theart methods on all standard point cloud completion benchmarks. Notably, VRCNet shows great generalizability and robustness on real-world point cloud scans.

Motivation & Objective

• To address the limitations of existing point cloud completion methods that generate coarse shapes lacking fine local details.
• To overcome the deterministic nature of current methods by introducing a probabilistic framework that models uncertainty in shape completion.
• To explicitly learn structural relations—such as symmetry, regularity, and surface smoothness—using relational feature modules for improved geometric fidelity.
• To provide a large-scale, diverse multi-view partial point cloud dataset (MVP) to support training and evaluation of completion models.
• To demonstrate generalizability and robustness on real-world scans from KITTI and ScanNet, beyond synthetic benchmarks.

Proposed method

• VRCNet employs a dual-path architecture: one path reconstructs complete point clouds using a variational autoencoder (VAE) to learn prior distributions, while the other generates complete shapes from partial inputs guided by the reconstructed prior.
• The probabilistic modeling path regularizes the posterior distribution of partial point clouds to align with the prior distribution from complete clouds, enabling principled uncertainty modeling.
• A relational enhancement network (RENet) refines coarse completions using multi-scale local features, with the Point Self-Attention Kernel (PSA) adaptively aggregating relations among neighboring points via learned attention weights.
• The Point Selective Kernel Module (PSK) fuses features from multiple kernel sizes using adaptive channel-wise weighting, enhancing multi-scale feature representation.
• The model is trained end-to-end on a new large-scale MVP dataset with over 100,000 partial and complete scans derived from 26 uniformly distributed camera views per 3D CAD model.
• Fine-tuning is applied on real-world scans (KITTI, ScanNet) to adapt the model to sparse and noisy real-world data without retraining from scratch.

Experimental results

Research questions

• RQ1 Can a variational framework improve shape completion by modeling uncertainty in the completion process, leading to more plausible and diverse outputs?
• RQ2 How effectively can relational feature learning—such as attention-based local relations and multi-scale kernel fusion—recover fine-grained geometric details and structural symmetries?
• RQ3 To what extent does the proposed method generalize to real-world, noisy, and sparse point cloud scans from LiDAR and depth cameras?
• RQ4 Does the inclusion of a large-scale, multi-view partial point cloud dataset (MVP) significantly improve model generalization and robustness compared to existing benchmarks?
• RQ5 Can the proposed relational modules (PSA and PSK) be effectively transferred to other 3D point cloud tasks beyond completion?

Key findings

• VRCNet achieves the lowest Chamfer Distance (CD) loss on the Completion3D benchmark, with a mean CD of 8.12×10⁻⁴, significantly outperforming prior SOTA methods such as GRNet (10.64×10⁻⁴) and TopNet (14.25×10⁻⁴).
• On the MVP dataset, VRCNet generates more detailed and structurally consistent shapes than PCN, NSFA, and AtlasNet, particularly in recovering symmetrical parts like legs of chairs and wheels of motorbikes.
• The model successfully reconstructs plausible complete shapes from real-world scans in KITTI and ScanNet, preserving observed details and generating smooth, coherent surfaces even with high missing ratios.
• Ablation studies confirm that each component—PSA, PSK, and the dual-path architecture—contributes significantly to performance, with the full model outperforming the baseline ECG by a large margin.
• User studies rank VRCNet as the most preferred method for real-scanned point cloud completion, indicating strong perceptual quality and realism of generated outputs.
• The MVP dataset, containing over 100,000 high-quality partial and complete scans, enables better data diversity and model generalization, especially for real-world generalization tasks.