[Paper Review] SRPCN: Structure Retrieval based Point Completion Network

Kaiyi Zhang, Ximing J. Yang+2 more|arXiv (Cornell University)|Feb 6, 2022

3D Shape Modeling and Analysis4 citations

TL;DR

SRPCN proposes a structure retrieval-based point cloud completion network that enhances completion authenticity by retrieving plausible complete shapes from a database using k-means clustering and KL divergence. By modeling structure points as Gaussian distributions and leveraging retrieval-based reconstruction, it achieves more realistic outputs and superior generalization across diverse missing patterns compared to existing methods.

ABSTRACT

Given partial objects and some complete ones as references, point cloud completion aims to recover authentic shapes. However, existing methods pay little attention to general shapes, which leads to the poor authenticity of completion results. Besides, the missing patterns are diverse in reality, but existing methods can only handle fixed ones, which means a poor generalization ability. Considering that a partial point cloud is a subset of the corresponding complete one, we regard them as different samples of the same distribution and propose Structure Retrieval based Point Completion Network (SRPCN). It first uses k-means clustering to extract structure points and disperses them into distributions, and then KL Divergence is used as a metric to find the complete structure point cloud that best matches the input in a database. Finally, a PCN-like decoder network is adopted to generate the final results based on the retrieved structure point clouds. As structure plays an important role in describing the general shape of an object and the proposed structure retrieval method is robust to missing patterns, experiments show that our method can generate more authentic results and has a stronger generalization ability.

Motivation & Objective

• To address the poor authenticity of existing point cloud completion methods, which often produce fuzzy or implausible structures.
• To overcome the limited generalization of current methods that assume fixed missing patterns during training.
• To introduce a new representation—structure distribution—that preserves general shape with fewer points.
• To develop a robust retrieval mechanism based on KL divergence for matching partial inputs to complete, structurally similar shapes.
• To demonstrate improved performance on real-world missing patterns beyond standard benchmarks.

Proposed method

• K-means clustering is applied to extract key structure points from both input partial and reference complete point clouds.
• Each structure point is modeled as an ellipsoidal Gaussian distribution, forming a distribution representation of the object's general shape.
• The overall structure distribution is constructed by merging individual Gaussian components, enabling distribution-level comparison.
• KL divergence is used as a metric to retrieve the most similar complete structure point cloud from a database of reference shapes.
• A PCN-like decoder network is used to upsample the retrieved structure point cloud to generate the final complete point cloud.
• The retrieval process is invariant to missing patterns, enabling robust generalization across diverse input corruptions.

Experimental results

Research questions

• RQ1 Can a structure-based retrieval mechanism improve the authenticity of point cloud completion beyond end-to-end learning?
• RQ2 How well can structure distribution representations generalize across diverse and arbitrary missing patterns?
• RQ3 Does using KL divergence for distribution matching outperform direct shape reconstruction in terms of structural fidelity?
• RQ4 Can retrieval-based completion produce plausible results even when the ground truth is not retrieved?
• RQ5 To what extent does the proposed method reduce visual artifacts and structural ambiguity in completion outputs?

Key findings

• SRPCN achieves superior completion authenticity, as shown in qualitative results where outputs are more plausible and structurally coherent than those of PCN, TopNet, and GRNet.
• Despite performing poorly on Chamfer Distance (a metric sensitive to point-wise alignment), SRPCN produces more visually authentic results, indicating that standard metrics may not reflect perceptual quality.
• The method generalizes well to unseen missing patterns, including those not seen during training, such as view-point-based point removal and depth image back-projection.
• Even when retrieving a shape different from the ground truth, the completion result remains reasonable and structurally sound, demonstrating robustness to retrieval errors.
• The structure distribution representation effectively preserves general shape while reducing instability from sparse structure points, as validated in visualizations of distribution density.
• Generalization experiments confirm that SRPCN maintains high-quality outputs on two new missing patterns, including one that causes failure in baseline models like PF-Net.