[Paper Review] The Whole Is Greater Than the Sum of Its Nonrigid Parts

Oshri Halimi, Ido Imanuel+5 more|arXiv (Cornell University)|Jan 27, 2020

3D Shape Modeling and Analysis61 references11 citations

TL;DR

This paper proposes a deep Siamese autoencoder that unifies non-rigid shape completion and partial correspondence by leveraging a complete shape as a prior to reconstruct a full, detailed 3D model from a single partial scan in a different pose. The method achieves state-of-the-art performance in single-view deformable shape completion and dense correspondence, outperforming prior work by large margins on both synthetic and real data, with robustness to noise, downsampling, and varying projection angles.

ABSTRACT

According to Aristotle, a philosopher in Ancient Greece, "the whole is greater than the sum of its parts". This observation was adopted to explain human perception by the Gestalt psychology school of thought in the twentieth century. Here, we claim that observing part of an object which was previously acquired as a whole, one could deal with both partial matching and shape completion in a holistic manner. More specifically, given the geometry of a full, articulated object in a given pose, as well as a partial scan of the same object in a different pose, we address the problem of matching the part to the whole while simultaneously reconstructing the new pose from its partial observation. Our approach is data-driven, and takes the form of a Siamese autoencoder without the requirement of a consistent vertex labeling at inference time; as such, it can be used on unorganized point clouds as well as on triangle meshes. We demonstrate the practical effectiveness of our model in the applications of single-view deformable shape completion and dense shape correspondence, both on synthetic and real-world geometric data, where we outperform prior work on these tasks by a large margin.

Motivation & Objective

• To address the ill-posed problem of single-view non-rigid shape completion by leveraging a complete shape as a geometric prior.
• To unify partial shape matching and non-rigid shape completion in a single, end-to-end learning framework.
• To enable accurate reconstruction of full, detailed 3D shapes from partial scans without requiring consistent vertex labeling.
• To achieve robust performance under extreme partiality, noise, and varying viewing angles.
• To demonstrate state-of-the-art results on both synthetic (FAUST) and real-world (AMASS) datasets.

Proposed method

• A Siamese encoder-decoder architecture is trained to predict a deformation field that maps a full shape to match a partial scan in a different pose.
• The network is trained in a self-supervised manner using paired full and partial shapes related by non-rigid transformations.
• The deformation is predicted based on geometric features from both the full shape (for identity) and the partial shape (for pose).
• The method operates on unorganized point clouds and triangle meshes without requiring consistent vertex correspondence at inference.
• Dense correspondence is recovered via nearest-neighbor search between the partial shape and the reconstructed full shape.
• The architecture is invariant to azimuthal projection angles and robust to vertex noise and downsampling.

Experimental results

Research questions

• RQ1 Can a deep learning model reconstruct a complete, detailed 3D shape from a single partial scan using a full shape as a prior?
• RQ2 How well can a unified framework jointly solve non-rigid shape completion and partial correspondence under extreme partiality?
• RQ3 To what extent is the method robust to noise, vertex downsampling, and varying projection angles in real-world scans?
• RQ4 Can the model generalize across diverse subjects and poses without requiring consistent vertex labeling?
• RQ5 Does leveraging the intrinsic geometry of the full shape significantly improve reconstruction fidelity compared to methods relying only on partial data?

Key findings

• The method achieves state-of-the-art performance on single-view deformable shape completion, outperforming prior work on both FAUST and AMASS datasets.
• The model maintains high reconstruction accuracy even under 4 cm additive white Gaussian noise, with only a slight decline in performance.
• Robustness to vertex downsampling is demonstrated, with accurate reconstructions preserved even after significant vertex decimation.
• The error distribution across different projection angles is nearly uniform, indicating strong azimuthal invariance and consistent performance regardless of viewing direction.
• Dense non-rigid correspondence is successfully recovered via nearest-neighbor matching between the partial input and the reconstructed shape, with qualitative and quantitative validation.
• The method completes shapes in under one second per inference, demonstrating high efficiency for real-time applications.