[Paper Review] Fine-Grained Zero-Shot Learning with DNA as Side Information

Sarkhan Badirli, Zeynep Akata+3 more|arXiv (Cornell University)|Sep 29, 2021

Domain Adaptation and Few-Shot LearningComputer Science45 references20 citations

TL;DR

This paper introduces DNA as a novel side information source for fine-grained zero-shot learning (ZSL), using mitochondrial DNA barcodes to model class hierarchies and improve unseen class classification. The authors propose a hierarchical Bayesian model with DNA-learned embeddings that outperforms state-of-the-art methods on a large insect dataset and rivals word vectors on CUB, demonstrating DNA's superiority as a scalable, biologically grounded alternative to visual attributes or word embeddings.

ABSTRACT

Fine-grained zero-shot learning task requires some form of side-information to transfer discriminative information from seen to unseen classes. As manually annotated visual attributes are extremely costly and often impractical to obtain for a large number of classes, in this study we use DNA as side information for the first time for fine-grained zero-shot classification of species. Mitochondrial DNA plays an important role as a genetic marker in evolutionary biology and has been used to achieve near-perfect accuracy in the species classification of living organisms. We implement a simple hierarchical Bayesian model that uses DNA information to establish the hierarchy in the image space and employs local priors to define surrogate classes for unseen ones. On the benchmark CUB dataset, we show that DNA can be equally promising yet in general a more accessible alternative than word vectors as a side information. This is especially important as obtaining robust word representations for fine-grained species names is not a practicable goal when information about these species in free-form text is limited. On a newly compiled fine-grained insect dataset that uses DNA information from over a thousand species, we show that the Bayesian approach outperforms state-of-the-art by a wide margin.

Motivation & Objective

• To address the challenge of fine-grained zero-shot species classification when visual attributes or word embeddings are impractical due to limited textual or morphological data.
• To explore DNA as a biologically grounded, highly specific side information source for class similarity modeling in ZSL, especially for large-scale, fine-grained taxonomic groups.
• To develop a hierarchical Bayesian model that leverages DNA similarity to establish surrogate classes for unseen species, improving generalization beyond seen classes.
• To demonstrate that DNA-based side information outperforms traditional semantic side information (e.g., word vectors, WordNet) in highly fine-grained, large-scale ZSL settings.
• To validate the model on both benchmark (CUB) and a newly compiled, large-scale insect dataset with over 1,000 species, showing robust performance.

Proposed method

• The method uses a Convolutional Neural Network (CNN) to learn DNA barcode embeddings from short mitochondrial DNA sequences (e.g., COI gene), treating these as side information.
• A hierarchical Bayesian model is proposed where unseen classes are assigned to surrogate classes based on DNA similarity, with local priors defining class centers.
• The model uses two hyperparameters, κ₀ and κ₁, to control the dispersion of surrogate-class centers and actual-class centers within each local prior, respectively.
• Class prediction is performed using Bayesian inference, where the likelihood of an image belonging to a class is computed based on its distance to the class center and the prior distribution.
• The model is trained end-to-end using stochastic gradient descent, with the objective of maximizing seen-class accuracy while enabling knowledge transfer to unseen classes via DNA-based hierarchy.
• The approach is evaluated on the CUB dataset and a new insect dataset with over 1,000 species, using both DNA embeddings and word vectors for comparison.

Experimental results

Research questions

• RQ1 Can DNA barcodes serve as a reliable and effective side information source for fine-grained zero-shot learning in species classification?
• RQ2 How does a DNA-based hierarchical Bayesian model compare to word vector-based models in terms of zero-shot accuracy on fine-grained datasets?
• RQ3 To what extent does DNA similarity enable better class hierarchy modeling than semantic or visual attributes in highly fine-grained taxonomic settings?
• RQ4 How sensitive is the model's performance to hyperparameters κ₀ and κ₁, and what optimal settings maximize unseen class accuracy?
• RQ5 Can DNA embeddings trained on short sequences achieve competitive performance compared to word vectors trained on massive text corpora?

Key findings

• On the CUB dataset, DNA embeddings trained on less than 30,000 short mitochondrial sequences achieved zero-shot classification performance competitive with word vectors trained on massive text corpora.
• The hierarchical Bayesian model using DNA side information significantly outperformed state-of-the-art ZSL methods on a newly compiled insect dataset with over 1,000 species, especially in unseen class accuracy.
• The model’s performance was highly sensitive to the hyperparameter κ₁, with optimal unseen class accuracy achieved when κ₁ ∈ [0.1, 1], indicating that moderate dispersion of actual-class centers within a local prior is crucial.
• When κ₁ was too small or too large, performance degraded sharply due to unrealistic prior assumptions, highlighting the importance of proper hyperparameter tuning.
• The model showed robustness in closed-set classification even when training and test species were mutually exclusive, demonstrating the specificity and generalization capability of DNA embeddings.
• The ablation study confirmed that the inclusion of DNA-based hierarchy and local priors significantly improved performance, with the Bayesian framework effectively leveraging biological similarity for knowledge transfer.