[論文レビュー] Unsupervised Discovery of Interpretable Directions in the GAN Latent Space

Andrey Voynov, Artem Babenko|arXiv (Cornell University)|Feb 10, 2020

Image Processing and 3D Reconstruction参考文献 29被引用数 142

ひとこと要約

本論文は、事前学習済みのGANにおける解釈可能な潜在空間の方向を検出するための、教師なしでモデルに依存しない手法を提示する。これによりラベルなしで意味的な画像操作を実現できる。また、弱教師付き顕著性検出への実用的な応用も示す。

ABSTRACT

The latent spaces of GAN models often have semantically meaningful directions. Moving in these directions corresponds to human-interpretable image transformations, such as zooming or recoloring, enabling a more controllable generation process. However, the discovery of such directions is currently performed in a supervised manner, requiring human labels, pretrained models, or some form of self-supervision. These requirements severely restrict a range of directions existing approaches can discover. In this paper, we introduce an unsupervised method to identify interpretable directions in the latent space of a pretrained GAN model. By a simple model-agnostic procedure, we find directions corresponding to sensible semantic manipulations without any form of (self-)supervision. Furthermore, we reveal several non-trivial findings, which would be difficult to obtain by existing methods, e.g., a direction corresponding to background removal. As an immediate practical benefit of our work, we show how to exploit this finding to achieve competitive performance for weakly-supervised saliency detection.

研究の動機と目的

• Identify semantically meaningful directions in a pretrained GAN latent space without supervision.
• Learn a set of disentangled latent directions that induce easy-to-interpret image transformations.
• Showcase practical uses of discovered directions, such as background removal for downstream tasks.

提案手法

• Fix a pretrained generator G and learn a latent-direction matrix A and a reconstructor R end-to-end while keeping G fixed.
• Sample pairs of latent codes z and z + A(ε e_k) and feed both through G to obtain image pairs.
• Train R to predict the direction index k and the shift magnitude ε from the image pair.
• Use a classification loss on k and a regression loss on ε to encourage disentangled, interpretable directions.
• Ensure columns of A are unit-norm or orthonormal to promote diversity and stability of directions.
• Choose K to match latent dimensionality (or a chosen subset) and experiment with unit-norm vs orthonormal column constraints.

実験結果

リサーチクエスチョン

• RQ1 Can we discover semantically meaningful, interpretable latent directions in GANs without supervision?
• RQ2 Do unsupervised directions tend to be human-interpretable and diverse across datasets and generators?
• RQ3 Can discovered directions enable practical tasks such as weakly supervised saliency detection?

主な発見

• The method identifies non-trivial, human-interpretable latent directions across multiple generators and datasets.
• Some discovered directions correspond to meaningful manipulations like background removal.
• Discovered directions can be leveraged to generate synthetic data for weakly supervised saliency detection with competitive performance.
• Using orthonormal versus unit-norm column constraints influences diversity and interpretability of directions across datasets.
• The approach remains completely unsupervised and model-agnostic, requiring no generator re-training.
• Qualitative results show interpretable transformations across MNIST, AnimeFaces, CelebA-HQ, and BigGAN.