[Paper Review] Efficient Globally Optimal 2D-to-3D Deformable Shape Matching

Zorah Lähner, Emanuele Rodolà+3 more|arXiv (Cornell University)|Jan 22, 2016

3D Shape Modeling and Analysis32 references4 citations

TL;DR

This paper proposes a globally optimal, polynomial-time algorithm for 2D-to-3D deformable shape matching, modeling the problem as finding the shortest circular path on the product 3-manifold of the 2D and 3D shapes. It achieves a worst-case complexity of 𝒪(mn² log n) and provides both exact and ε-approximate solutions, enabling efficient sketch-based 3D shape retrieval with state-of-the-art performance on deformable shapes including humans and cats with topological changes.

ABSTRACT

We propose the first algorithm for non-rigid 2D-to-3D shape matching, where the input is a 2D shape represented as a planar curve and a 3D shape represented as a surface; the output is a continuous curve on the surface. We cast the problem as finding the shortest circular path on the product 3-manifold of the surface and the curve. We prove that the optimal matching can be computed in polynomial time with a (worst-case) complexity of $O(mn^2\log(n))$, where $m$ and $n$ denote the number of vertices on the template curve and the 3D shape respectively. We also demonstrate that in practice the runtime is essentially linear in $m\!\cdot\! n$ making it an efficient method for shape analysis and shape retrieval. Quantitative evaluation confirms that the method provides excellent results for sketch-based deformable 3D shape retrieval.

Motivation & Objective

• To address the lack of methods for non-rigid 2D-to-3D shape matching, particularly for deformable shapes such as humans and animals with topological changes.
• To develop a globally optimal algorithm that computes continuous, closed matching curves on 3D surfaces from 2D query contours.
• To ensure computational efficiency for large-scale shapes (up to 40,000 vertices) while maintaining high retrieval accuracy.
• To support sketch-based 3D shape retrieval by providing a robust 2D-to-3D similarity criterion based on learned correspondences.

Proposed method

• The problem is formulated as minimizing an energy functional representing the shortest circular path on the product 3-manifold of the 2D and 3D shapes.
• The method uses a discrete formulation based on a graph representation of the product manifold, where nodes represent pairs of 2D and 3D vertices.
• It applies Dijkstra’s algorithm to compute shortest paths on this graph, ensuring globally optimal solutions via branch-and-bound when needed.
• An ε-approximate variant is introduced to reduce runtime by limiting the search space, trading off minimal precision for significant speedup.
• The method supports different local feature descriptors for 2D and 3D data, including spectral features, enabling semantically meaningful correspondences.
• The matching energy is used as a similarity measure for 3D shape retrieval, enabling efficient k-NN search in embedding space.

Figure 2: Examples of correspondences between a cat contour and a 3D cat (left) and human contours and 3D humans (middle and right). The human shape (middle) exhibit a topological change along the hands which is handled well by our method. The contour on the right is handdrawn. The dashed lines deno

Experimental results

Research questions

• RQ1 Can a globally optimal 2D-to-3D matching method be designed for deformable shapes with non-rigid transformations and topological changes?
• RQ2 What is the computational complexity of computing globally optimal 2D-to-3D correspondences, and can it be made practical for large-scale shapes?
• RQ3 How does the use of an ε-approximate solution affect retrieval performance and runtime in sketch-based 3D shape retrieval?
• RQ4 Can spectral features from 2D and 3D shapes be effectively compared to enable semantically meaningful matching?
• RQ5 To what extent can the matching energy serve as a reliable similarity metric for 3D shape retrieval?

Key findings

• The proposed method achieves a worst-case time complexity of 𝒪(mn² log n), where m and n are the number of vertices on the 2D and 3D shapes, respectively.
• The global algorithm achieves a mean average precision (MAP) of 0.9984 on the extended 2D-to-3D retrieval dataset, outperforming Shape DNA (0.4720) and consensus segmentation (0.5907).
• The ε-approximate variant reduces runtime significantly while maintaining high accuracy, achieving a MAP of 0.9597 on the same dataset.
• For hand-drawn sketches, the method maintains strong performance, with a MAP of 0.9462 for human shapes and 0.9772 for cat shapes.
• Embedding of 3D models using matching energies successfully separates all classes in 2D space, with only one outlier (horse near centaurs), demonstrating the method’s discriminative power.
• The method successfully handles topological changes, such as limbs in human poses, by maintaining continuous, semantically meaningful correspondences even when parts are occluded or missing.

Figure 3: Matching between a 2D query shape (left) and a 3D target shape achieved by solving a LAP between the same point-wise features our method uses (middle) and our method (right).