[Paper Review] DeformSyncNet: Deformation Transfer via Synchronized Shape Deformation Spaces

Minhyuk Sung, Zhenyu Jiang+3 more|arXiv (Cornell University)|Sep 3, 2020

3D Shape Modeling and Analysis66 references4 citations

TL;DR

DeformSyncNet introduces a neural network framework that enables consistent deformation transfer and projection across a shape category without requiring explicit correspondence or part-level annotations. By learning a shared canonical latent space for all deformations and individual linear deformation spaces per shape, it synchronizes deformations via latent space encoding, achieving path-invariant, plausible edits that transfer instantly across shapes while preserving structural properties like symmetry and part connectivity.

ABSTRACT

Shape deformation is an important component in any geometry processing toolbox. The goal is to enable intuitive deformations of single or multiple shapes or to transfer example deformations to new shapes while preserving the plausibility of the deformed shape(s). Existing approaches assume access to point-level or part-level correspondence or establish them in a preprocessing phase, thus limiting the scope and generality of such approaches. We propose DeformSyncNet, a new approach that allows consistent and synchronized shape deformations without requiring explicit correspondence information. Technically, we achieve this by encoding deformations into a class-specific idealized latent space while decoding them into an individual, model-specific linear deformation action space, operating directly in 3D. The underlying encoding and decoding are performed by specialized (jointly trained) neural networks. By design, the inductive bias of our networks results in a deformation space with several desirable properties, such as path invariance across different deformation pathways, which are then also approximately preserved in real space. We qualitatively and quantitatively evaluate our framework against multiple alternative approaches and demonstrate improved performance.

Motivation & Objective

• To enable consistent, synchronized deformations across a category of 3D shapes without relying on explicit point or part correspondences.
• To learn a shared canonical latent space that encodes all possible deformations between shapes in a category.
• To map each individual shape to a unique, shape-specific linear deformation space that is synchronized with the canonical space.
• To support both deformation projection (snapping edits to plausible shape space) and deformation transfer (applying edits to other shapes in the category) in a unified framework.
• To achieve path-invariance and structural preservation (e.g., symmetry, part connectivity) in deformations without correspondence supervision.

Proposed method

• The framework uses a jointly trained encoder-decoder neural network architecture: a shape-specific encoder maps input shapes into a canonical latent space, while a decoder maps latent vectors back to per-point deformation offsets in 3D space.
• The canonical latent space is shared across all shapes in a category and encodes all possible deformations, ensuring synchronization across shapes.
• Each shape's individual deformation space is learned via a shape-specific decoder that maps latent vectors to 3D point offsets, enabling shape-aware deformation.
• The network is trained using Chamfer distance between reconstructed and ground-truth deformed shapes, with deformation handle projections used as a supervisory signal in the loss function.
• The architecture enforces inductive bias toward path-invariance and structural consistency by design, with regularization to ensure unique and meaningful latent representations.
• An extension to non-linear deformations is explored using circular trajectory modeling in the latent space, improving fitting accuracy on articulated shapes.

Experimental results

Research questions

• RQ1 Can deformation transfer be achieved across a shape category without requiring explicit correspondence between shapes or their parts?
• RQ2 Can a shared canonical latent space synchronize individual shape-specific deformation spaces to ensure consistent and plausible edits across shapes?
• RQ3 How can path-invariance and structural preservation (e.g., symmetry, part connectivity) be achieved in deformation transfer without correspondence supervision?
• RQ4 Can the framework generalize to non-linear deformations, such as rotational motions in articulated shapes?
• RQ5 To what extent does the absence of correspondence information affect the quality and consistency of deformation transfer and projection?

Key findings

• DeformSyncNet achieves consistent deformation transfer across multiple shapes in the same category without using any correspondence information during training.
• The framework preserves structural properties such as symmetry and part connectivity during editing, as demonstrated by visualizations showing maintained red and green circles in Figure 1.
• On the Shape2Motion dataset, learning circular trajectories in the latent space yields lower fitting errors (0.03) compared to linear trajectories (0.04) when the latent dimension matches the degrees of freedom.
• With a large latent dimension (k=64), the linear trajectory approach also achieves low fitting error (0.08), showing that high-dimensional spaces can encode complex motions.
• The network successfully discovers independent rotational motions in articulated shapes, as visualized in Figure 14, where each learned dictionary element corresponds to a distinct folding part.
• Neural Cages, which also uses linear offsets, fails on the Shape2Motion dataset, indicating DeformSyncNet's superior generalization and modeling capability.