[Paper Review] 3D Reconstruction of Incomplete Archaeological Objects Using a Generative Adversary Network

Renato Hermoza, Ivan Sipiran|arXiv (Cornell University)|Nov 17, 2017

Image Processing and 3D Reconstruction23 references3 citations

TL;DR

This paper proposes ORGAN, a conditional generative adversarial network (GAN) that reconstructs incomplete 3D archaeological objects by leveraging a 3D encoder-decoder architecture with dual loss objectives: completion loss and improved Wasserstein GAN loss. The model conditions on metadata such as culture or region, enabling accurate recovery of missing geometry—even when over 50% of voxels are missing—while minimizing structural errors.

ABSTRACT

We introduce a data-driven approach to aid the repairing and conservation of archaeological objects: ORGAN, an object reconstruction generative adversarial network (GAN). By using an encoder-decoder 3D deep neural network on a GAN architecture, and combining two loss objectives: a completion loss and an Improved Wasserstein GAN loss, we can train a network to effectively predict the missing geometry of damaged objects. As archaeological objects can greatly differ between them, the network is conditioned on a variable, which can be a culture, a region or any metadata of the object. In our results, we show that our method can recover most of the information from damaged objects, even in cases where more than half of the voxels are missing, without producing many errors.

Motivation & Objective

• To address the challenge of reconstructing damaged 3D archaeological objects with minimal human intervention.
• To develop a data-driven approach that generalizes across diverse archaeological object types by conditioning on cultural or regional metadata.
• To improve reconstruction fidelity in cases of severe occlusion or missing geometry, such as when over half the voxels are absent.
• To reduce artifacts and geometric errors in generated reconstructions through a novel loss function design.

Proposed method

• The method employs a 3D encoder-decoder neural network architecture within a generative adversarial network (GAN) framework.
• The generator network predicts the missing geometry of incomplete 3D objects based on input partial scans.
• The model is conditioned on metadata such as culture or region to guide reconstruction with domain-specific priors.
• Two loss functions are used: a completion loss to minimize reconstruction error and an improved Wasserstein GAN loss to stabilize training and improve sample quality.
• The discriminator evaluates the realism of generated shapes by distinguishing between real and generated incomplete objects.
• The network is trained end-to-end to optimize both geometric accuracy and perceptual realism of the reconstructed shapes.

Experimental results

Research questions

• RQ1 Can a conditional GAN effectively reconstruct 3D archaeological objects with more than 50% of their voxels missing?
• RQ2 How well does conditioning on cultural or regional metadata improve reconstruction accuracy and generalization across diverse artifact types?
• RQ3 To what extent does combining completion loss with improved Wasserstein GAN loss reduce artifacts and improve structural fidelity in generated reconstructions?
• RQ4 Can the model generalize to unseen object classes without fine-tuning, given only metadata conditioning?

Key findings

• The proposed method successfully recovers most of the missing geometry in incomplete 3D archaeological objects, even when over half the voxels are missing.
• Conditioning the model on metadata such as culture or region significantly improves reconstruction accuracy and structural consistency.
• The combination of completion loss and improved Wasserstein GAN loss leads to fewer artifacts and more realistic shape generation compared to baseline methods.
• The model generalizes well across diverse archaeological object types without requiring retraining for each class.