[Paper Review] Fine-Grain Few-Shot Vision via Domain Knowledge as Hyperspherical Priors

B. Haney, Alexander Lavin|arXiv (Cornell University)|May 23, 2020

Domain Adaptation and Few-Shot Learning16 references7 citations

TL;DR

This paper proposes a hyperspherical prototype network that integrates domain knowledge as informative priors to improve few-shot fine-grained visual classification. By embedding class prototypes on a hypersphere using a priori attributes (e.g., bird color, wing pattern), the method achieves state-of-the-art 1-shot accuracy of 59.48% on CUB-200-2011 and a 5x speedup in training time compared to standard Euclidean prototypical networks.

ABSTRACT

Prototypical networks have been shown to perform well at few-shot learning tasks in computer vision. Yet these networks struggle when classes are very similar to each other (fine-grain classification) and currently have no way of taking into account prior knowledge (through the use of tabular data). Using a spherical latent space to encode prototypes, we can achieve few-shot fine-grain classification by maximally separating the classes while incorporating domain knowledge as informative priors. We describe how to construct a hypersphere of prototypes that embed a-priori domain information, and demonstrate the effectiveness of the approach on challenging benchmark datasets for fine-grain classification, with top results for one-shot classification and 5x speedups in training time.

Motivation & Objective

• To address the challenge of few-shot fine-grained classification where small interclass differences hinder model performance.
• To improve prototype generalization by incorporating a priori domain knowledge (e.g., binary attributes) into the latent space.
• To leverage non-Euclidean hyperspherical geometry to achieve better prototype separation and robustness under high-dimensional embedding.
• To reduce training time by enabling data-independent prototype optimization through precomputed priors.
• To demonstrate that semantic priors enhance clustering and inference stability in fine-grained tasks.

Proposed method

• The method uses a hyperspherical latent space where class prototypes are embedded with maximal angular separation to improve discriminability.
• Domain knowledge is encoded as binary attributes (e.g., wing color, back color) and used to compute prototype positions on the hypersphere via cosine similarity maximization.
• Prototypes are initialized a priori using a constrained optimization that minimizes the maximum cosine similarity between any two class prototypes.
• Inference is performed by computing cosine similarity between the image embedding and all class prototypes, selecting the class with highest similarity.
• The model uses a ResNet32 backbone with SGD optimization and standard data augmentation (random crop, horizontal flip).
• The hyperspherical prototype space is trained once and reused, enabling data-independent optimization and faster training.

Experimental results

Research questions

• RQ1 Can domain knowledge encoded as binary attributes improve few-shot fine-grained classification performance?
• RQ2 Does using a hyperspherical latent space with priors lead to better prototype separation and robustness than Euclidean or maximally spread spherical spaces?
• RQ3 Can a data-independent prototype initialization reduce training time without sacrificing accuracy?
• RQ4 Does the hyperspherical model avoid the surface area collapse problem in high dimensions?
• RQ5 How does the integration of semantic priors affect clustering and generalization in fine-grained tasks?

Key findings

• The proposed method achieves a 1-shot five-way accuracy of 59.48% on the CUB-200-2011 dataset, outperforming both standard Euclidean and maximally spread hyperspherical prototypes.
• The method achieves a 5x speedup in training time, reducing training from 96 seconds per epoch to 20 seconds per epoch on a GeForce RTX 2080 Ti.
• The model maintains stable performance across high-dimensional hyperspheres, with no signs of surface area collapse or instability at 64 or 128 dimensions.
• The inclusion of domain priors significantly improves prototype localization, as shown by T-SNE visualizations where semantically similar classes (e.g., sparrows) cluster together.
• The method achieves 71.19% accuracy in the 5-shot five-way setting, demonstrating strong generalization from few examples.
• The model shows robustness to input noise, correctly classifying a noisy Baird Sparrow instance as a sparrow and assigning low confidence to a semantically nearby but distinct class (American Pipit).