[Paper Review] LIA: Latently Invertible Autoencoder with Adversarial Learning

Jiapeng Zhu, Deli Zhao+2 more|arXiv (Cornell University)|Sep 25, 2019

Generative Adversarial Networks and Image Synthesis40 references13 citations

TL;DR

This paper proposes Latently Invertible Autoencoder (LIA), a novel GAN-based framework that enables disentangled, invertible encoding of real images by embedding symmetric invertible networks within a VAE's latent space. By training the decoder as a GAN and then learning a partial encoder from a disentangled autoencoder, LIA avoids the entanglement issues of VAE/GANs, achieving high-fidelity image generation and reconstruction on FFHQ and LSUN datasets.

ABSTRACT

Generative Adversarial Networks (GANs) play an increasingly important role in machine learning. However, there is one fundamental issue hindering their practical applications: the absence of capability for encoding real-world samples. The conventional way of addressing this issue is to learn an encoder for GAN via Variational Auto-Encoder (VAE). In this paper, we show that the entanglement of the latent space for the VAE/GAN framework poses the main challenge for encoder learning. To address the entanglement issue and enable inference in GAN we propose a novel algorithm named Latently Invertible Autoencoder (LIA). The framework of LIA is that an invertible network and its inverse mapping are symmetrically embedded in the latent space of VAE. The decoder of LIA is first trained as a standard GAN with the invertible network and then the partial encoder is learned from a disentangled autoencoder by detaching the invertible network from LIA, thus avoiding the entanglement problem caused by the random latent space. Experiments conducted on the FFHQ face dataset and three LSUN datasets validate the effectiveness of LIA/GAN.

Motivation & Objective

• To address the fundamental limitation of GANs in encoding real-world images due to the lack of invertible inference.
• To identify latent space entanglement in VAE/GAN frameworks as the primary obstacle to effective encoder learning.
• To develop a method that enables disentangled, invertible encoding by decoupling the encoder training from the entangled latent space of VAE.
• To achieve high-quality image generation and reconstruction by combining GAN training with invertible autoencoding.
• To validate the framework on diverse benchmarks, including FFHQ and LSUN datasets, demonstrating improved performance over conventional VAE/GAN approaches.

Proposed method

• LIA embeds a symmetric invertible network and its inverse within the latent space of a VAE, enabling exact reconstruction from latent codes.
• The decoder is first trained as a standard GAN using the invertible network to map latent codes to real images.
• After GAN training, the invertible network is detached, and a partial encoder is trained on the disentangled latent space to map real images to latent codes.
• The disentangled latent space is preserved by training the encoder independently, avoiding entanglement caused by random noise in standard VAEs.
• The framework leverages adversarial learning for image quality while maintaining invertibility and disentanglement through symmetric invertible mappings.
• The method ensures that the learned encoder can reconstruct real images with high fidelity by leveraging the invertible structure and disentangled representation.

Experimental results

Research questions

• RQ1 Can a GAN-based framework achieve invertible and disentangled image encoding by decoupling encoder learning from the entangled latent space of VAE?
• RQ2 Does the use of symmetric invertible networks in the latent space improve the fidelity and disentanglement of image reconstruction in GANs?
• RQ3 How does LIA compare to standard VAE/GAN frameworks in terms of image generation quality and reconstruction accuracy?
• RQ4 Can the proposed method generalize across diverse datasets such as FFHQ and LSUN without architectural modifications?
• RQ5 What is the impact of removing entanglement through disentangled autoencoder training on the performance of GAN-based image generation?

Key findings

• LIA successfully enables invertible and disentangled encoding in GANs by decoupling encoder learning from the entangled latent space of VAE.
• The framework achieves high-fidelity image generation and reconstruction on the FFHQ face dataset, demonstrating strong perceptual quality.
• LIA outperforms standard VAE/GAN frameworks in reconstruction fidelity by avoiding entanglement through disentangled autoencoder training.
• The use of symmetric invertible networks ensures exact reconstruction from latent codes, enabling reliable inference in GANs.
• Experiments on LSUN datasets confirm the generalization capability of LIA across diverse image domains.
• The disentangled latent space learned by LIA supports meaningful interpolation and manipulation of image attributes, indicating improved disentanglement.