[Paper Review] Mesh Variational Autoencoders with Edge Contraction Pooling

Yujie Yuan, Yu‐Kun Lai+3 more|arXiv (Cornell University)|Aug 7, 2019

3D Shape Modeling and Analysis39 references4 citations

TL;DR

This paper proposes a mesh variational autoencoder with a novel edge contraction pooling operation based on a modified mesh simplification algorithm, enabling effective hierarchical feature learning on 3D meshes with consistent connectivity. The method achieves superior generalization, handles high-resolution models, and outperforms prior work in shape generation, interpolation, and embedding by reducing parameters and preserving geometric detail.

ABSTRACT

3D shape analysis is an important research topic in computer vision and graphics. While existing methods have generalized image-based deep learning to meshes using graph-based convolutions, the lack of an effective pooling operation restricts the learning capability of their networks. In this paper, we propose a novel pooling operation for mesh datasets with the same connectivity but different geometry, by building a mesh hierarchy using mesh simplification. For this purpose, we develop a modified mesh simplification method to avoid generating highly irregularly sized triangles. Our pooling operation effectively encodes the correspondence between coarser and finer meshes in the hierarchy. We then present a variational auto-encoder structure with the edge contraction pooling and graph-based convolutions, to explore probability latent spaces of 3D surfaces. Our network requires far fewer parameters than the original mesh VAE and thus can handle denser models thanks to our new pooling operation and convolutional kernels. Our evaluation also shows that our method has better generalization ability and is more reliable in various applications, including shape generation, shape interpolation and shape embedding.

Motivation & Objective

• To address the lack of effective pooling operations in mesh-based deep learning, which limits receptive field growth and model generalization.
• To enable efficient, parameter-efficient learning on high-resolution 3D meshes with consistent connectivity.
• To develop a pooling mechanism that preserves correspondence between coarse and fine mesh levels for end-to-end training.
• To enhance the performance of mesh variational autoencoders by integrating graph convolutions and a modified mesh simplification process.
• To demonstrate improved capabilities in shape generation, interpolation, and low-dimensional embedding compared to existing methods.

Proposed method

• Proposes a modified mesh simplification algorithm that avoids highly irregular triangles by prioritizing edge collapse based on geometric and topological constraints.
• Introduces an edge contraction pooling operation that maintains explicit vertex correspondence between successive mesh levels in a hierarchy.
• Employs a variational autoencoder architecture with graph convolutional layers for feature learning on vertex-based deformation representations.
• Uses per-vertex feature representations (deformation gradients) as input instead of raw 3D coordinates to enable consistent processing across meshes with same connectivity.
• Designs a de-pooling operation that reconstructs higher-resolution meshes from latent codes using the stored correspondence maps.
• Applies graph convolutions to capture local geometric patterns while enabling hierarchical feature abstraction through the mesh hierarchy.

Experimental results

Research questions

• RQ1 Can a mesh simplification-based pooling operation effectively encode correspondence between coarse and fine mesh levels for deep learning?
• RQ2 How does edge contraction pooling improve generalization and parameter efficiency in mesh VAEs compared to fully connected layers?
• RQ3 To what extent can the proposed method handle high-resolution meshes that are infeasible for previous mesh VAEs?
• RQ4 How does the method perform in shape interpolation and latent space embedding compared to state-of-the-art approaches?
• RQ5 Can the framework maintain geometric fidelity and structural consistency across diverse 3D shape deformations?

Key findings

• The proposed method achieves better generalization and reconstruction quality than the original MeshVAE, which relies on fully connected layers and struggles with high-resolution models.
• The network can process dense meshes (e.g., 42,321 vertices) that are infeasible for MeshVAE due to memory constraints, thanks to efficient pooling and convolutional operations.
• Interpolation results show smoother, more natural motion sequences without artifacts or redundant motions, outperforming both MeshVAE and a data-driven method [8].
• The method produces high-quality, detailed shape generations and interpolations, even between complex or non-human shapes, as demonstrated in Fig. 12 and Fig. 13.
• Latent space visualization reveals that the method successfully captures intrinsic cyclic motion patterns, forming a coherent circle in 2D embedding, unlike t-SNE and PCA.
• The framework demonstrates robustness in shape embedding, grouping similar shapes and poses together while preserving structural and deformational relationships.