[Paper Review] Structured Disentangled Representations

Babak Esmaeili, Hao Wu+7 more|arXiv (Cornell University)|Apr 6, 2018

Generative Adversarial Networks and Image Synthesis34 references58 citations

TL;DR

The paper introduces hierarchically factorized VAEs (HFVAEs) with a two-level objective to control independence between blocks and within blocks, enabling disentanglement of discrete factors and better generalization to unseen factor combinations.

ABSTRACT

Deep latent-variable models learn representations of high-dimensional data in an unsupervised manner. A number of recent efforts have focused on learning representations that disentangle statistically independent axes of variation by introducing modifications to the standard objective function. These approaches generally assume a simple diagonal Gaussian prior and as a result are not able to reliably disentangle discrete factors of variation. We propose a two-level hierarchical objective to control relative degree of statistical independence between blocks of variables and individual variables within blocks. We derive this objective as a generalization of the evidence lower bound, which allows us to explicitly represent the trade-offs between mutual information between data and representation, KL divergence between representation and prior, and coverage of the support of the empirical data distribution. Experiments on a variety of datasets demonstrate that our objective can not only disentangle discrete variables, but that doing so also improves disentanglement of other variables and, importantly, generalization even to unseen combinations of factors.

Motivation & Objective

• Motivate the need for disentangled representations in deep generative models.
• Develop a two-level hierarchical objective to control independence across variable blocks and within blocks.
• Generalize the VAE objective to explicitly manage trade-offs among data–representation mutual information, prior matching, and data distribution coverage.
• Show that disentangling discrete factors can improve disentanglement of other variables and generalization.

Proposed method

• Reinterpret the VAE objective as a KL divergence between the generative model and the inference model.
• Decompose the VAE objective into four terms representing data–z identifiability, mutual information control, and marginal distribution consistency.
• Introduce a two-level hierarchical decomposition to enforce statistical independence via total correlation (TC) within blocks and within-individual variables.
• Provide a unified view of generalized VAE objectives to illustrate trade-offs among terms and their effects.
• Demonstrate that HFVAE can disentangle discrete factors and improve generalization to unseen factor combinations.

Experimental results

Research questions

• RQ1 Can a two-level hierarchical objective disentangle discrete factors and improve disentanglement of continuous variables?
• RQ2 Does explicitly controlling total correlation within blocks and within variables enhance generalization to unseen factor combinations?
• RQ3 How do trade-offs among mutual information, prior matching, and data distribution coverage affect disentanglement quality?
• RQ4 Do HFVAEs recover latent feature combinations not present in training data?

Key findings

• HFVAEs can disentangle discrete variables and improve disentanglement of other variables.
• The hierarchical objective yields better generalization to unseen combinations of latent factors.
• The approach provides a unified perspective on trade-offs in VAE objective modifications.
• Empirical results on datasets like dSprites, MNIST, Fashion-MNIST, CelebA, and 20Newsgroups support improved disentanglement and generalization.