[Paper Review] Model Selection for Generalized Zero-shot Learning

Hongguang Zhang, Piotr Koniusz|arXiv (Cornell University)|Nov 8, 2018

Domain Adaptation and Few-Shot Learning5 citations

TL;DR

This paper proposes ModelSel, a novel model selection framework for generalized zero-shot learning that separates classification into seen and unseen class tasks using GAN-generated auxiliary data. By training a selector to distinguish seen from unseen data and employing specialized classifiers for each, the method reduces bias from class imbalance, achieving state-of-the-art performance across four benchmark datasets.

ABSTRACT

In the problem of generalized zero-shot learning, the datapoints from unknown classes are not available during training. The main challenge for generalized zero-shot learning is the unbalanced data distribution which makes it hard for the classifier to distinguish if a given testing sample comes from a seen or unseen class. However, using Generative Adversarial Network (GAN) to generate auxiliary datapoints by the semantic embeddings of unseen classes alleviates the above problem. Current approaches combine the auxiliary datapoints and original training data to train the generalized zero-shot learning model and obtain state-of-the-art results. Inspired by such models, we propose to feed the generated data via a model selection mechanism. Specifically, we leverage two sources of datapoints (observed and auxiliary) to train some classifier to recognize which test datapoints come from seen and which from unseen classes. This way, generalized zero-shot learning can be divided into two disjoint classification tasks, thus reducing the negative influence of the unbalanced data distribution. Our evaluations on four publicly available datasets for generalized zero-shot learning show that our model obtains state-of-the-art results.

Motivation & Objective

• Address the challenge of class imbalance in generalized zero-shot learning, where classifiers tend to favor seen classes due to data distribution skew.
• Overcome the limitation of existing methods that treat generalized zero-shot learning as a single unified classification task, leading to poor generalization on unseen classes.
• Leverage generative adversarial networks (GANs) to synthesize auxiliary data for unseen classes to improve model robustness and generalization.
• Develop a model selection mechanism that decouples the classification of seen and unseen classes, enabling more accurate and balanced predictions.
• Achieve state-of-the-art performance on multiple standard generalized zero-shot learning benchmarks through improved handling of data imbalance and classifier specialization.

Proposed method

• Train a linear SVM classifier (M_sel) to distinguish between original training data (labeled as 1) and GAN-generated data for unseen classes (labeled as -1), creating a selector for data origin.
• Train a separate classifier M_s on original seen-class data and M_u on GAN-generated unseen-class data, enabling specialized learning for each class type.
• Use a SoftmaxLog classifier for M_s, M_u, and a joint classifier M_t trained on combined seen and unseen data for comparison.
• Introduce two variants: ModelSel-2Way (binary selector) and ModelSel-3Way (ternary selector), with ModelSel-2Way-SA incorporating soft assignment via sigmoid on SVM scores.
• Apply data normalization (mean and standard deviation) to both original and generated data before training M_sel to mitigate distribution imbalance.
• Optimize all models using Adam with a learning rate of 1e-4, batch size 60, and 50 training epochs, with hyperparameters tuned via cross-validation.

Experimental results

Research questions

• RQ1 Can a model selection mechanism that separates seen and unseen class classification improve performance in generalized zero-shot learning?
• RQ2 How does decoupling the training of classifiers for seen and unseen classes affect accuracy and robustness compared to joint training?
• RQ3 To what extent does using GAN-generated auxiliary data for unseen classes reduce the negative impact of class imbalance in generalized zero-shot learning?
• RQ4 What is the impact of soft assignment via sigmoid on SVM scores in improving model generalization and accuracy?
• RQ5 How does the proposed ModelSel framework compare to state-of-the-art methods like f-CLSWGAN and ZSKL across diverse benchmark datasets?

Key findings

• ModelSel-3Way achieves a harmonic mean accuracy of 66.4% on the FLO dataset, outperforming the previous SOTA f-CLSWGAN by 0.8 percentage points.
• On AWA1, ModelSel-3Way achieves 62.4% harmonic mean accuracy, representing a 2.8 percentage point improvement over f-CLSWGAN.
• On AWA2, ModelSel-3Way achieves 63.7% harmonic mean accuracy, a 3.6 percentage point improvement over f-CLSWGAN.
• ModelSel-2Way-SA achieves 42.3% harmonic mean accuracy on APY, a significant improvement from ZSKL’s 20.5%, demonstrating strong performance on challenging splits.
• The soft assignment variant (ModelSel-2Way-SA) shows consistent gains across datasets, indicating that Sigmoid-based score calibration enhances generalization.
• The model selection mechanism effectively mitigates class imbalance, as evidenced by improved performance on both seen and unseen class splits, particularly on previously difficult benchmarks like APY.