[Paper Review] Generative adversarial interpolative autoencoding: adversarial training on latent space interpolations encourage convex latent distributions

Tim Sainburg, Marvin Thielk+3 more|arXiv (Cornell University)|Jul 17, 2018

Advanced Image Processing Techniques40 references43 citations

TL;DR

GAIA combines autoencoder and GAN elements to train on latent-space interpolations, promoting convex latent distributions and realistic cross-space reconstructions with bidirectional mapping.

ABSTRACT

We present a neural network architecture based upon the Autoencoder (AE) and Generative Adversarial Network (GAN) that promotes a convex latent distribution by training adversarially on latent space interpolations. By using an AE as both the generator and discriminator of a GAN, we pass a pixel-wise error function across the discriminator, yielding an AE which produces non-blurry samples that match both high- and low-level features of the original images. Interpolations between images in this space remain within the latent-space distribution of real images as trained by the discriminator, and therfore preserve realistic resemblances to the network inputs. Code available at https://github.com/timsainb/GAIA

Motivation & Objective

• Motivate generative modeling for exploring perceptual and psychophysical representations.
• Promote convexity in latent space by adversarially training interpolations.
• Achieve bidirectional X↔Z mapping with pixel-wise reconstruction through the discriminator.
• Preserve high-dimensional structure in latent space while maintaining sharp, realistic samples.

Proposed method

• Use an autoencoder as both generator and discriminator within a GAN framework.
• Train with a pixel-wise (L1) loss passed across the discriminator to reduce blurriness.
• Train on interpolations in latent space (z_int) to enforce realism in convex combinations of latent samples.
• Incorporate a distance-preserving regularization that aligns pairwise distances in X and Z (multidimensional scaling style).
• Compare GAIA to AE and VAE on 2D datasets and CELEBA-HQ, evaluating interpolation realism and latent-space structure.

Experimental results

Research questions

• RQ1 Does adversarial training on latent-space interpolations produce a convex latent distribution that yields realistic samples from interpolations?
• RQ2 Can GAIA achieve bidirectional X↔Z mapping with preserved high-dimensional structure while delivering sharp samples?
• RQ3 How does GAIA perform relative to AE and VAE on low-dimensional and high-dimensional datasets in terms of interpolation realism and latent-space structure?
• RQ4 Do latent-space interpolations in GAIA lie within the true data distribution more often than in standard AEs or VAEs?
• RQ5 Can high-level attributes be linearly manipulated in GAIA’s latent space similar to other generative models?

Key findings

• GAIA interpolations (Gd( z_int )) are most likely to fall inside the data distribution compared to AE and VAE on 2D datasets.
• GAIA with distance loss maintains stronger correspondence between pairwise X and Z distances than AE or VAE.
• GAIA preserves high-dimensional structure while producing sharp, non-blurred reconstructions thanks to pixel-wise loss passed through the discriminator.
• On 2D datasets, GAIA’s interpolation samples match the true data distribution as effectively as a VAE for projecting interpolations, while respecting original data structure.
• GAIA enables realistic latent-space interpolations and demonstrates bidirectional inference with reconstructed samples at pixel level.