[Paper Review] Learning with Imprinted Weights

H. Jerry Qi, Matthew A. Brown+1 more|arXiv (Cornell University)|Dec 19, 2017

Domain Adaptation and Few-Shot Learning12 references10 citations

TL;DR

This paper introduces weight imprinting, a method that directly sets final-layer weights in ConvNets using scaled embeddings of few-shot training examples, enabling rapid classification of novel categories. Unlike proxy-based methods, it uses a single learned weight vector per class, achieving better generalization than nearest-neighbor instance embeddings through averaging of imprinted weights.

ABSTRACT

Human vision is able to immediately recognize novel visual categories after seeing just one or a few training examples. We describe how to add a similar capability to ConvNet classifiers by directly setting the final layer weights from novel training examples during low-shot learning. We call this process weight imprinting as it directly sets weights for a new category based on an appropriately scaled copy of the embedding layer activations for that training example. The imprinting process provides a valuable complement to training with stochastic gradient descent, as it provides immediate good classification performance and an initialization for any further fine-tuning in the future. We show how this imprinting process is related to proxy-based embeddings. However, it differs in that only a single imprinted weight vector is learned for each novel category, rather than relying on a nearest-neighbor distance to training instances as typically used with embedding methods. Our experiments show that using averaging of imprinted weights provides better generalization than using nearest-neighbor instance embeddings.

Motivation & Objective

• To enable ConvNet classifiers to rapidly recognize novel visual categories after seeing only one or a few training examples.
• To address the limitation of stochastic gradient descent in low-shot learning by providing an immediate, high-quality initialization for new classes.
• To improve generalization over existing embedding-based methods that rely on nearest-neighbor distances to training instances.
• To explore a direct, non-iterative method for initializing new class weights using feature activations from novel examples.

Proposed method

• The method directly sets the final fully connected layer weights for a new class using a scaled copy of the embedding layer activations from a single training example.
• The imprinted weight vector is derived by projecting the input example’s activation through a learned transformation, effectively creating a prototype for the new class.
• The approach avoids relying on nearest-neighbor comparisons by learning a single, fixed weight vector per novel category.
• Averaging multiple imprinted weights across different examples of the same class improves generalization performance.
• The method is conceptually related to proxy-based embeddings but differs by using a single vector per class rather than distance-based retrieval.

Experimental results

Research questions

• RQ1 Can direct weight initialization via imprinted embeddings achieve superior few-shot classification performance compared to standard fine-tuning?
• RQ2 How does using a single prototype vector per class compare to nearest-neighbor-based instance embeddings in terms of generalization?
• RQ3 Does averaging imprinted weights across multiple examples of the same class lead to better performance than using a single example?
• RQ4 How does weight imprinting compare to proxy-based embedding methods in low-shot learning scenarios?

Key findings

• Weight imprinting provides immediate, high-quality classification performance for novel categories after seeing only one or a few examples.
• Averaging imprinted weights across multiple examples of the same class leads to better generalization than using a single imprinted vector.
• The method outperforms nearest-neighbor instance embedding approaches by avoiding reliance on distance comparisons to training instances.
• The imprinting process serves as an effective initialization for subsequent fine-tuning, improving convergence and final accuracy.