[Paper Review] DualDis: Dual-Branch Disentangling with Adversarial Learning

Thomas Robert, Nicolas Thome+1 more|arXiv (Cornell University)|Jun 3, 2019

Digital Media Forensic Detection45 references4 citations

TL;DR

DualDis proposes a dual-branch autoencoder with adversarial training to disentangle class and attribute information into separate, linearized latent subspaces. By using adversarial classifiers to enforce orthogonal separation, it enables precise semantic image editing and effective semi-supervised data augmentation, achieving state-of-the-art disentangling and 10% accuracy gains on Yale-B with generated data.

ABSTRACT

In computer vision, disentangling techniques aim at improving latent representations of images by modeling factors of variation. In this paper, we propose DualDis, a new auto-encoder-based framework that disentangles and linearizes class and attribute information. This is achieved thanks to a two-branch architecture forcing the separation of the two kinds of information, accompanied by a decoder for image reconstruction and generation. To effectively separate the information, we propose to use a combination of regular and adversarial classifiers to guide the two branches in specializing for class and attribute information respectively. We also investigate the possibility of using semi-supervised learning for an effective disentangling even using few labels. We leverage the linearization property of the latent spaces for semantic image editing and generation of new images. We validate our approach on CelebA, Yale-B and NORB by measuring the efficiency of information separation via classification metrics, visual image manipulation and data augmentation.

Motivation & Objective

• To develop a disentangled representation learning framework that explicitly separates identity (class) and visual attribute information in image data.
• To enforce orthogonality between class and attribute factors in the latent space using adversarial training.
• To linearize factors of variation in each latent subspace for effective semantic image manipulation and generation.
• To enable semi-supervised learning with limited labels while maintaining strong disentangling performance.
• To demonstrate practical utility through semantic data augmentation that improves downstream classification accuracy.

Proposed method

• DualDis uses a two-branch encoder-decoder architecture, with one branch encoding identity (class) and the other encoding attributes, each guided by a dedicated classifier.
• Adversarial classifiers are trained to detect the opposite domain's information in each latent space, forcing the encoder to remove cross-domain signals and achieve disentanglement.
• Reconstruction loss ensures the decoder can generate realistic images from the combined latent codes.
• Linear classifiers are used to guide the linearization of factors in each latent subspace, enabling semantic navigation via vector arithmetic.
• Semi-supervised learning is applied by leveraging unlabeled data during training to reduce annotation dependency.
• Image editing is performed by modifying the attribute latent code along learned linear directions, preserving identity.

Experimental results

Research questions

• RQ1 Can adversarial training effectively disentangle class and attribute factors in a shared image representation?
• RQ2 Does enforcing orthogonality between latent subspaces improve disentanglement quality and downstream classification accuracy?
• RQ3 To what extent can linearized latent spaces enable precise semantic image editing through vector arithmetic?
• RQ4 Can DualDis achieve strong disentangling performance with limited labeled data using semi-supervised learning?
• RQ5 Can generated images from the model improve identity classification via semantic data augmentation?

Key findings

• DualDis achieved state-of-the-art disentangling performance on CelebA, with high classification accuracy for both identity and attributes.
• On the Yale-B dataset, data augmentation using generated images improved identity classification accuracy by nearly 10 percentage points when using 60 generated images per class.
• The model successfully performed semantic image editing by modifying attributes such as gender, eyeglasses, and smile intensity along learned linear directions.
• Mixing identity from one image and attributes from another produced plausible, semantically consistent results, confirming effective disentanglement.
• Semi-supervised training maintained strong disentangling performance even with limited labels, demonstrating robustness to annotation scarcity.
• The linearized latent space enabled fine-grained control over attribute changes, such as varying the intensity of a smile or hair color.