[Paper Review] ALIGNet: Partial-Shape Agnostic Alignment via Unsupervised Learning

Rana Hanocka, Noa Fish+4 more|arXiv (Cornell University)|Apr 23, 2018

Image and Object Detection Techniques31 references20 citations

TL;DR

ALIGNet proposes an unsupervised deep learning approach for partial-shape agnostic shape alignment using a convolutional neural network trained to warp complete source shapes to incomplete targets as if they were complete. By leveraging shape datasets to learn a shape-aware prior and enforcing piecewise smooth deformations via anisotropic total variation regularization, the method achieves robust, generalizable alignment even with over 50% of the target shape missing, without requiring ground-truth correspondences.

ABSTRACT

The process of aligning a pair of shapes is a fundamental operation in computer graphics. Traditional approaches rely heavily on matching corresponding points or features to guide the alignment, a paradigm that falters when significant shape portions are missing. These techniques generally do not incorporate prior knowledge about expected shape characteristics, which can help compensate for any misleading cues left by inaccuracies exhibited in the input shapes. We present an approach based on a deep neural network, leveraging shape datasets to learn a shape-aware prior for source-to-target alignment that is robust to shape incompleteness. In the absence of ground truth alignments for supervision, we train a network on the task of shape alignment using incomplete shapes generated from full shapes for self-supervision. Our network, called ALIGNet, is trained to warp complete source shapes to incomplete targets, as if the target shapes were complete, thus essentially rendering the alignment partial-shape agnostic. We aim for the network to develop specialized expertise over the common characteristics of the shapes in each dataset, thereby achieving a higher-level understanding of the expected shape space to which a local approach would be oblivious. We constrain ALIGNet through an anisotropic total variation identity regularization to promote piecewise smooth deformation fields, facilitating both partial-shape agnosticism and post-deformation applications. We demonstrate that ALIGNet learns to align geometrically distinct shapes, and is able to infer plausible mappings even when the target shape is significantly incomplete. We show that our network learns the common expected characteristics of shape collections, without over-fitting or memorization, enabling it to produce plausible deformations on unseen data during test time.

Motivation & Objective

• Address the challenge of aligning geometrically distinct or partially missing shapes where traditional feature-based methods fail due to lack of correspondences.
• Overcome the limitations of local optimization methods that lack global shape priors and are sensitive to missing data or topological variations.
• Develop a data-driven, unsupervised framework that learns a shape-aware prior from large shape collections to guide plausible, structure-preserving deformations.
• Enable alignment robust to significant shape incompleteness by training the network to infer complete shape structures from partial inputs.
• Ensure generalization to unseen shapes by avoiding overfitting or memorization through regularization and self-supervised learning.

Proposed method

• Train a CNN-based warping network using a self-supervised loss that compares the overlap between a warped source shape and a complete target shape, even when the target is partially observed.
• Employ a grid-based free-form deformation (FFD) layer as a differentiable spatial transformer to learn high-order, expressive deformations with controllable smoothness.
• Apply anisotropic total variation (TV) regularization to the deformation field to promote piecewise smoothness and prevent structural distortion in post-processing tasks.
• Use random pairs of shapes from a dataset as training samples, where the source is always complete and the target is randomly masked to simulate incompleteness.
• Leverage large-scale shape collections to learn a global, class-specific shape prior that captures common geometric, topological, and semantic characteristics.
• Train the network end-to-end in an unsupervised manner, without requiring ground-truth correspondences or alignment labels.

Experimental results

Research questions

• RQ1 Can a deep neural network learn to perform plausible shape alignment on incomplete targets without any ground-truth supervision?
• RQ2 To what extent can a shape-aware prior learned from a dataset improve alignment robustness in the presence of missing parts?
• RQ3 How effective is anisotropic total variation regularization in preserving geometric structure during deformation while enabling partial-shape agnosticism?
• RQ4 Can the network generalize to unseen shapes from the same class or similar classes, even when the target is significantly incomplete?
• RQ5 Does the unsupervised, self-supervised training paradigm prevent overfitting or memorization while still enabling accurate reconstruction of symmetries and scale?

Key findings

• ALIGNet successfully aligns complete source shapes to partial targets with over 50% of the shape missing, producing plausible and structurally coherent results.
• The network preserves key shape attributes such as symmetry and scale, even in ambiguous or highly incomplete cases, indicating effective learning of shape priors.
• The anisotropic total variation regularization significantly improves deformation smoothness and prevents structural degradation, especially important for downstream tasks like texture transfer.
• The method generalizes well to unseen test shapes from the same dataset and even to untrained but similar shape classes, demonstrating robustness and generalization capability.
• The unsupervised training scheme avoids memorization and overfitting, as the network learns a shared shape prior across the dataset rather than memorizing specific examples.
• Post-processing with ICP can further refine alignment, suggesting that ALIGNet provides a strong initialization for iterative refinement methods.