[Paper Review] VConv-DAE: Deep Volumetric Shape Learning Without Object Labels

Abhishek Sharma, Oliver Grau+1 more|arXiv (Cornell University)|Apr 13, 2016

3D Shape Modeling and Analysis24 references4 citations

TL;DR

This paper proposes VConv-DAE, a fully convolutional volumetric autoencoder that learns deep 3D shape representations in an unsupervised manner from noisy, incomplete 3D shapes without object labels. By training to reconstruct voxel occupancy grids from corrupted inputs, it achieves state-of-the-art performance in denoising and shape completion, outperforming supervised methods like ShapeNet while being 200x faster at test time.

ABSTRACT

With the advent of affordable depth sensors, 3D capture becomes more and more ubiquitous and already has made its way into commercial products. Yet, capturing the geometry or complete shapes of everyday objects using scanning devices (e.g. Kinect) still comes with several challenges that result in noise or even incomplete shapes. Recent success in deep learning has shown how to learn complex shape distributions in a data-driven way from large scale 3D CAD Model collections and to utilize them for 3D processing on volumetric representations and thereby circumventing problems of topology and tessellation. Prior work has shown encouraging results on problems ranging from shape completion to recognition. We provide an analysis of such approaches and discover that training as well as the resulting representation are strongly and unnecessarily tied to the notion of object labels. Thus, we propose a full convolutional volumetric auto encoder that learns volumetric representation from noisy data by estimating the voxel occupancy grids. The proposed method outperforms prior work on challenging tasks like denoising and shape completion. We also show that the obtained deep embedding gives competitive performance when used for classification and promising results for shape interpolation.

Motivation & Objective

• To learn meaningful 3D shape representations from raw, noisy, and incomplete 3D scans without relying on object-level annotations.
• To overcome the limitations of supervised deep learning methods in 3D shape analysis, particularly the need for costly labeling and slow inference.
• To develop a fully convolutional, end-to-end trainable autoencoder that generalizes well to challenging 3D reconstruction tasks like denoising and shape completion.
• To provide a scalable, efficient, and unsupervised alternative to prior deep learning models such as ShapeNet, which require layer-wise pre-training and sampling-based inference.

Proposed method

• The method employs a fully convolutional volumetric autoencoder that maps noisy or incomplete 3D voxel grids to clean, complete reconstructions.
• The network is trained in an unsupervised manner by reconstructing the original voxel occupancy from corrupted inputs, such as randomly corrupted voxels or sliced-out portions.
• The model uses a denoising autoencoder objective, where the input is a corrupted version of the ground truth shape, and the output is the predicted clean shape.
• The architecture is fully convolutional, enabling end-to-end training without pre-training, and avoids sampling-based inference, significantly improving test-time efficiency.
• The learned latent code serves as a deep embedding for downstream tasks such as classification, interpolation, and shape completion.
• The model is trained from scratch on a large-scale CAD dataset without any object labels, part annotations, or symmetry constraints.

Experimental results

Research questions

• RQ1 Can a fully convolutional volumetric autoencoder learn meaningful 3D shape representations without any object-level supervision?
• RQ2 How does unsupervised learning via voxel reconstruction compare to supervised approaches like ShapeNet in terms of performance on shape completion and denoising?
• RQ3 Can the learned latent space support downstream tasks such as shape classification and interpolation?
• RQ4 Does the model generalize to challenging noise patterns like slicing noise, where large portions of the shape are missing?
• RQ5 What is the computational efficiency of the proposed method compared to prior work, especially at test time?

Key findings

• VConv-DAE outperforms ShapeNet on both denoising and shape completion tasks, achieving lower average reconstruction errors across all object categories.
• On the challenging slicing noise scenario (30% missing voxels), VConv-DAE achieves a mean error of 12.75, significantly lower than ShapeNet’s 14.85, indicating superior completion capability.
• The model reduces test-time inference from 600ms (ShapeNet) to just 3ms, a 200x speedup, due to the absence of sampling at test time.
• The learned latent space enables competitive performance on 3D shape classification, demonstrating the utility of the unsupervised representation.
• Qualitative results show that VConv-DAE effectively reconstructs missing parts, especially in complex shapes like beds and nightstands, with fewer artifacts than ShapeNet.
• The model generalizes well to unseen noise patterns, including random and structured noise, validating its robustness and generalization capacity.