[Paper Review] StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation

Zongze Wu, Dani Lischinski+1 more|arXiv (Cornell University)|Nov 25, 2020

Generative Adversarial Networks and Image SynthesisComputer Science39 references42 citations

TL;DR

The paper analyzes StyleSpace (StyleGAN2’s channel-wise style parameters), showing it is highly disentangled, introduces methods to discover local style channels, and demonstrates attribute-specific and real-image manipulations with an attribution-dependent disentanglement metric.

ABSTRACT

We explore and analyze the latent style space of StyleGAN2, a state-of-the-art architecture for image generation, using models pretrained on several different datasets. We first show that StyleSpace, the space of channel-wise style parameters, is significantly more disentangled than the other intermediate latent spaces explored by previous works. Next, we describe a method for discovering a large collection of style channels, each of which is shown to control a distinct visual attribute in a highly localized and disentangled manner. Third, we propose a simple method for identifying style channels that control a specific attribute, using a pretrained classifier or a small number of example images. Manipulation of visual attributes via these StyleSpace controls is shown to be better disentangled than via those proposed in previous works. To show this, we make use of a newly proposed Attribute Dependency metric. Finally, we demonstrate the applicability of StyleSpace controls to the manipulation of real images. Our findings pave the way to semantically meaningful and well-disentangled image manipulations via simple and intuitive interfaces.

Motivation & Objective

• Assess disentanglement across StyleGAN2 latent spaces (Z, W, S) using real-data trained models.
• Identify and characterize locally-active StyleSpace channels that control specific image regions.
• Develop unsupervised or minimally supervised methods to detect channels controlling attributes and regions.
• Propose an attribute dependency metric to quantify disentanglement in manipulations.
• Demonstrate applicability of StyleSpace controls to real-image manipulation and inversion strategies.

Proposed method

• Quantitatively compare latent spaces Z, W, and StyleSpace S using DCI metrics (disentanglement, completeness, informativeness) on FFHQ with 40 CelebA-based classifiers.
• Develop gradient-based localization by back-propagating through StyleSpace channels, thresholding gradient maps, and measuring overlaps with semantic maps to identify locally-active channels.
• Propose a simple exemplar-based method to detect channels controlling a target attribute by comparing exemplar-mean style vectors to population statistics.
• Introduce Attribute Dependency (AD) as a metric to quantify how manipulating one attribute affects others, enabling cross-method comparisons.
• Evaluate localization and attribute control across FFHQ, LSUN Car, and LSUN Bedroom datasets using segmentation networks for semantic maps.
• Discuss inversion strategies in W, W+, and S spaces and propose a hybrid approach (encoder initialization + sparse latent optimization) for real-image manipulation.

Experimental results

Research questions

• RQ1 Is StyleSpace S more disentangled than Z, W, or W+ for StyleGAN2 trained on real data?
• RQ2 Can we identify a large set of localized StyleSpace channels that each control a distinct visual attribute or region?
• RQ3 Can a small number of positive exemplars reliably identify channels controlling a target attribute?
• RQ4 Does StyleSpace enable more disentangled attribute manipulation on real images compared to GANSpace or InterFaceGAN?
• RQ5 How can real images be inverted and manipulated within StyleSpace to preserve realism and controllability?

Key findings

• StyleSpace S shows higher disentanglement and completeness than Z, W, and W+ according to DCI metrics on FFHQ; informativeness is high for all but is not the distinguishing factor.
• A large set of locally-active StyleSpace channels (1871 in FFHQ model out of 6048 non-tRGB channels) control localized semantic regions like hair, mouth, clothes, beds, and wheels, with high regional localization and disentanglement.
• Attribute-specific channels can be identified with as few as 10–30 positive exemplars, achieving high top-5 detection accuracy (over 92% in some cases when restricting to locally-active channels).
• StyleSpace-based manipulations achieve lower Attribute Dependency (AD) than GANSpace or InterFaceGAN, indicating more disentangled edits in real images.
• Real-image inversion in S space yields best reconstruction fidelity among W, W+, and S, while preserving more natural manipulation when combined with a brief optimization sequence.
• The proposed gradient-overlap and exemplar-mean methods provide practical, scalable means to discover and rank interpretable, localized StyleSpace channels for semantic editing.