[Paper Review] Cascaded Refinement Network for Point Cloud Completion

Xiaogang Wang, Marcelo H. Ang+1 more|arXiv (Cornell University)|Apr 7, 2020

3D Shape Modeling and AnalysisEngineering53 references19 citations

TL;DR

This paper proposes a cascaded refinement network for 3D point cloud completion that preserves local details from partial inputs while generating high-fidelity missing parts through a coarse-to-fine strategy. By combining a patch-based discriminator with iterative refinement and feature contraction-expansion modules, the method achieves state-of-the-art performance on multiple benchmarks with improved detail fidelity and reduced model size.

ABSTRACT

Point clouds are often sparse and incomplete. Existing shape completion methods are incapable of generating details of objects or learning the complex point distributions. To this end, we propose a cascaded refinement network together with a coarse-to-fine strategy to synthesize the detailed object shapes. Considering the local details of partial input with the global shape information together, we can preserve the existing details in the incomplete point set and generate the missing parts with high fidelity. We also design a patch discriminator that guarantees every local area has the same pattern with the ground truth to learn the complicated point distribution. Quantitative and qualitative experiments on different datasets show that our method achieves superior results compared to existing state-of-the-art approaches on the 3D point cloud completion task. Our source code is available at https://github.com/xiaogangw/cascaded-point-completion.git.

Motivation & Objective

• To address the challenge of generating high-fidelity, detailed 3D shapes from sparse and incomplete point clouds.
• To overcome limitations in existing methods that fail to preserve fine-grained structural details like chair legs in partial inputs.
• To develop a coarse-to-fine framework that jointly optimizes reconstruction and adversarial losses for better geometric realism.
• To learn complex point distributions by enforcing local pattern consistency through a patch-based discriminator.
• To reduce model size while maintaining or improving performance compared to prior state-of-the-art methods.

Proposed method

• The generator uses a three-stage architecture: feature extraction via two MLPs with max-pooling, coarse reconstruction using fully connected layers, and dense reconstruction via a cascaded refinement sub-network with lifting modules.
• A skip connection links the incomplete input to the coarse output, preserving existing structural details.
• An iterative refinement decoder with a feature contraction and expansion unit progressively refines point positions at both local and global levels.
• A patch-based discriminator is introduced to enforce local structural consistency by comparing patches of generated and real point clouds, improving fidelity to ground truth distributions.
• Adversarial training is combined with a reconstruction loss (Chamfer Distance) to jointly optimize for geometric accuracy and perceptual realism.
• Mirror augmentation is applied to partial inputs during training to improve generalization and symmetry learning.

Experimental results

Research questions

• RQ1 Can a cascaded refinement strategy preserve fine-grained structural details from partial point clouds while generating realistic missing parts?
• RQ2 How does a patch-based discriminator improve the modeling of complex point distributions compared to global discriminators?
• RQ3 To what extent does iterative refinement with contraction-expansion units enhance point cloud density and geometric accuracy?
• RQ4 Does the proposed method achieve better performance than state-of-the-art approaches on standard 3D completion benchmarks?
• RQ5 Can the model maintain high performance with fewer parameters and reduced model size?

Key findings

• The proposed method achieves a Chamfer Distance of 5.52 × 10⁻⁴ at 20% occlusion, outperforming PCN (7.69 × 10⁻⁴) and TopNet (8.46 × 10⁻⁴) on the ShapeNet dataset.
• At 70% occlusion, the method achieves a CD of 21.27 × 10⁻⁴, significantly lower than PCN (23.60 × 10⁻⁴) and TopNet (23.20 × 10⁻⁴), demonstrating robustness to high levels of missing data.
• The classification accuracy on generated point clouds reaches 94.17%, improving by 1.59 percentage points over PCN and TopNet, indicating better semantic fidelity.
• The model has 5.14M parameters and a size of 61.90MB, which is smaller than PCN (6.85M, 82.30MB) and TopNet (9.96M, 79.80MB), showing improved efficiency.
• Ablation studies confirm that removing any component—such as the discriminator, mean shape, contraction-expansion unit, or mirror operation—degrades performance, validating each module's contribution.
• Shape interpolation results demonstrate smooth transitions between latent vectors, confirming that the learned features preserve meaningful geometric semantics.