[Paper Review] Fully Convolutional Mesh Autoencoder using Efficient Spatially Varying Kernels

Yi Zhou, Chenglei Wu+6 more|arXiv (Cornell University)|Jun 8, 2020

3D Shape Modeling and Analysis29 references47 citations

TL;DR

Introduces a non-template-specific, fully convolutional mesh autoencoder with spatially varying kernels learned from a global weight basis and local coefficients, achieving state-of-the-art reconstruction on diverse mesh types including tetrahedrons and non-manifold meshes.

ABSTRACT

Learning latent representations of registered meshes is useful for many 3D tasks. Techniques have recently shifted to neural mesh autoencoders. Although they demonstrate higher precision than traditional methods, they remain unable to capture fine-grained deformations. Furthermore, these methods can only be applied to a template-specific surface mesh, and is not applicable to more general meshes, like tetrahedrons and non-manifold meshes. While more general graph convolution methods can be employed, they lack performance in reconstruction precision and require higher memory usage. In this paper, we propose a non-template-specific fully convolutional mesh autoencoder for arbitrary registered mesh data. It is enabled by our novel convolution and (un)pooling operators learned with globally shared weights and locally varying coefficients which can efficiently capture the spatially varying contents presented by irregular mesh connections. Our model outperforms state-of-the-art methods on reconstruction accuracy. In addition, the latent codes of our network are fully localized thanks to the fully convolutional structure, and thus have much higher interpolation capability than many traditional 3D mesh generation models.

Motivation & Objective

• Motivate learning latent representations for registered meshes with general topology (including non-manifold meshes).
• Develop a template-free, fully convolutional autoencoder for arbitrary meshes without relying on UV mapping or template templates.
• Introduce efficient locally varying convolution and pooling operators that handle irregular mesh connectivity while sharing parameters globally.

Proposed method

• Define vcConv and vcTransConv: convolution and transpose convolution where weights lie in a shared Basis B and are sampled per neighbor via locally variant coefficients (AV,i,j).
• Use Weight Basis B with M bases; per neighbor weights W_{i,j} = sum_k α_{i,j,k} B_k to reduce parameters.
• Introduce variant densities ρ' for pooling/unpooling to account for irregular sampling; ρ' normalized across neighborhood.
• Employ Monte Carlo-inspired vdPool and vdUnpool for feature aggregation, with learnable densities ρ.
• Construct fully convolutional autoencoder using residual blocks built from vcConv/vcTransConv + vdDownRes/vdUpRes, enabling down/up-sampling without fully connected layers.
• Enable localized latent feature interpolation by placing latent vertices (e.g., head/torso/limbs) and interpolating their latent codes.

Experimental results

Research questions

• RQ1 Can a template-free, fully convolutional mesh autoencoder achieve state-of-the-art reconstruction across arbitrary mesh topologies (including tetrahedral and non-manifold meshes)?
• RQ2 Do spatially varying convolution kernels with a global weight basis and locally learned coefficients provide superior reconstruction quality and memory efficiency compared to existing graph/convolution operators?
• RQ3 Can the model support localized latent code interpolation for semantically meaningful control over mesh regions?
• RQ4 What is the impact of different pooling/unpooling strategies and kernel basis sizes on reconstruction accuracy and memory usage?

Key findings

• The proposed vcConv/vcTransConv with a global weight basis and local variant coefficients achieves state-of-the-art reconstruction on D-FAUST and generalizes to tetrahedral and non-manifold meshes.
• The model supports down/up-sampling and transpose convolution, enabling end-to-end fully convolutional autoencoders without static templates.
• vdPool and vdUnpool with learned variant densities improve pooling stability on irregular meshes, outperforming simple max/average pooling.
• Localized latent interpolation is possible by placing latent vertices at semantically meaningful locations (head, limbs, torso), enabling targeted pose/surface edits.
• On D-FAUST, the model attains lower training and test errors than Neural3DMM and MeshCNN on comparable bottle-neck sizes; the comparison table indicates favorable metrics for the proposed method.
• The approach uses significantly fewer parameters than naïve locally connected layers while maintaining or improving precision; LCConv, though parameter-heavy, performs worse in this setting.