[Paper Review] Learning Deep Disentangled Embeddings with the F-Statistic Loss

Karl Ridgeway, Michael C. Mozer|arXiv (Cornell University)|Feb 14, 2018

Face and Expression Recognition43 citations

TL;DR

Introduces an F-statistic based loss for deep embeddings that promotes class separation on a subset of dimensions to support few-shot learning while encouraging disentanglement.

ABSTRACT

Deep-embedding methods aim to discover representations of a domain that make explicit the domain's class structure and thereby support few-shot learning. Disentangling methods aim to make explicit compositional or factorial structure. We combine these two active but independent lines of research and propose a new paradigm suitable for both goals. We propose and evaluate a novel loss function based on the $F$ statistic, which describes the separation of two or more distributions. By ensuring that distinct classes are well separated on a subset of embedding dimensions, we obtain embeddings that are useful for few-shot learning. By not requiring separation on all dimensions, we encourage the discovery of disentangled representations. Our embedding method matches or beats state-of-the-art, as evaluated by performance on recall@$k$ and few-shot learning tasks. Our method also obtains performance superior to a variety of alternatives on disentangling, as evaluated by two key properties of a disentangled representation: modularity and explicitness. The goal of our work is to obtain more interpretable, manipulable, and generalizable deep representations of concepts and categories.

Motivation & Objective

• Seek a representation that supports few-shot learning by separating classes on embedding dimensions.
• Encourage disentangled representations by not requiring all dimensions to separate all classes.
• Bridge deep embedding and disentangling literatures with a probabilistic, dimension-wise separation criterion.

Proposed method

• Define an F-statistic based separation measure between class means and within-class variance in embedding dimensions.
• Extend to multi-dimensional embeddings by selecting the top d dimensions with strongest separation for each class pair.
• Formulate a loss that minimizes the negative log probability of separation across selected dimensions.
• Use a class-aware or unnamed-factor oracle to train embeddings for either identity classification or disentangling factors.
• Train with ADAM and evaluate on recall@1 and few-shot tasks, comparing with histogram, triplet, binomial deviance, and LSSS losses.

Experimental results

Research questions

• RQ1 Can the F-statistic loss produce embeddings that perform competitively on recall@1/few-shot tasks against state-of-the-art losses?
• RQ2 Does enforcing separation on a subset of embedding dimensions yield more disentangled representations?
• RQ3 How does the F-statistic loss trade off modularity, compactness, and explicitness of the learned factors?
• RQ4 Is the approach robust across supervised (class-aware) and weakly supervised (unnamed-factor) settings?

Key findings

• F-statistic loss matches or surpasses state-of-the-art on recall@1 across CUHK03, Market-1501, and CUB-200-2011 datasets.
• On disentanglement metrics, the F-statistic loss achieves superior modularity and explicitness, particularly on the small NORB dataset.
• The method shows strong modularity and explicitness with both unnamed-factor and class-aware supervision, outperforming β-VAE baselines in most settings.
• Increasing the number of separating dimensions d yields robustness, with performance relatively insensitive to d in a broad range.