[Paper Review] Learning Debiased Representation via Disentangled Feature Augmentation

Jungsoo Lee, Eungyeup Kim+3 more|arXiv (Cornell University)|Jul 3, 2021

Domain Adaptation and Few-Shot Learning35 references46 citations

TL;DR

The paper proposes a feature-level data augmentation method that disentangles intrinsic and bias attributes to synthesize diverse bias-conflicting samples, improving debiasing and generalization across synthetic and real datasets.

ABSTRACT

Image classification models tend to make decisions based on peripheral attributes of data items that have strong correlation with a target variable (i.e., dataset bias). These biased models suffer from the poor generalization capability when evaluated on unbiased datasets. Existing approaches for debiasing often identify and emphasize those samples with no such correlation (i.e., bias-conflicting) without defining the bias type in advance. However, such bias-conflicting samples are significantly scarce in biased datasets, limiting the debiasing capability of these approaches. This paper first presents an empirical analysis revealing that training with "diverse" bias-conflicting samples beyond a given training set is crucial for debiasing as well as the generalization capability. Based on this observation, we propose a novel feature-level data augmentation technique in order to synthesize diverse bias-conflicting samples. To this end, our method learns the disentangled representation of (1) the intrinsic attributes (i.e., those inherently defining a certain class) and (2) bias attributes (i.e., peripheral attributes causing the bias), from a large number of bias-aligned samples, the bias attributes of which have strong correlation with the target variable. Using the disentangled representation, we synthesize bias-conflicting samples that contain the diverse intrinsic attributes of bias-aligned samples by swapping their latent features. By utilizing these diversified bias-conflicting features during the training, our approach achieves superior classification accuracy and debiasing results against the existing baselines on synthetic and real-world datasets.

Motivation & Objective

• Motivate the need for diverse bias-conflicting samples to improve debiasing and generalization.
• Propose a disentangled latent representation for intrinsic and bias attributes.
• Develop a feature-swapping augmentation mechanism to synthesize diverse bias-conflicting samples.
• Introduce a scheduling strategy to apply augmentation after disentanglement progresses.
• Empirically validate on synthetic and real-world biased datasets against baselines.

Proposed method

• Train two encoders to split image representations into intrinsic (z_i) and bias (z_b) latent vectors.
• Use linear classifiers to predict labels from z_i and z_b separately with a joint loss that emphasizes bias-conflicting samples.
• Compute a relative difficulty weight W(z) to identify bias-conflicting samples and drive intrinsic-attribute learning (L_dis).
• Swap latent components between samples (z_swap = [z_i; swapped z_b]) to generate bias-conflicting augmented features (L_swap).
• Schedule augmentation to start after partial disentanglement to avoid noisy early-stage samples (algorithmic workflow).
• Evaluate against baselines on Colored MNIST, Corrupted CIFAR-10, and Biased FFHQ (BFFHQ).

Experimental results

Research questions

• RQ1 Can diverse bias-conflicting samples beyond the training set improve debiasing performance?
• RQ2 Does disentangling intrinsic and bias attributes enable effective latent-space augmentation for debiasing?
• RQ3 What is the impact of feature augmentation scheduling on debiasing performance?
• RQ4 How does the proposed method perform relative to bias-specific and bias-agnostic baselines on synthetic and real-world biases?

Key findings

• Diverse bias-conflicting samples significantly improve debiasing performance across datasets.
• Disentangled latent representations enable effective generation of diverse bias-conflicting samples via feature swapping.
• Scheduled augmentation (after partial disentanglement) yields the best unbiased test accuracy across experiments.
• The proposed method outperforms baselines including bias-explicit and bias-agnostic approaches on Colored MNIST, Corrupted CIFAR-10, and Biased FFHQ.
• On Colored MNIST, the method reaches higher test accuracy than several baselines even without bias-type predefinition.