[Paper Review] Unsupervised Embedding Learning via Invariant and Spreading Instance Feature

Mang Ye, Xu Zhang+2 more|arXiv (Cornell University)|Apr 6, 2019

Domain Adaptation and Few-Shot Learning52 references80 citations

TL;DR

The paper introduces an instance-wise softmax embedding trained with a Siamese network to learn data augmentation invariant and instance spread-out features for unsupervised embedding, achieving fast convergence and strong performance on seen and unseen categories.

ABSTRACT

This paper studies the unsupervised embedding learning problem, which requires an effective similarity measurement between samples in low-dimensional embedding space. Motivated by the positive concentrated and negative separated properties observed from category-wise supervised learning, we propose to utilize the instance-wise supervision to approximate these properties, which aims at learning data augmentation invariant and instance spread-out features. To achieve this goal, we propose a novel instance based softmax embedding method, which directly optimizes the `real' instance features on top of the softmax function. It achieves significantly faster learning speed and higher accuracy than all existing methods. The proposed method performs well for both seen and unseen testing categories with cosine similarity. It also achieves competitive performance even without pre-trained network over samples from fine-grained categories.

Motivation & Objective

• Motivate learning discriminative embeddings without labels by leveraging instance-wise supervision.
• Enforce invariance of an image instance under data augmentation while ensuring features of different instances are spread apart.
• Develop an efficient instance-feature based softmax formulation that directly optimizes the embedding features.
• Demonstrate improved performance on standard unsupervised embedding benchmarks including seen and unseen categories.

Proposed method

• Propose an instance feature-based softmax embedding that directly optimizes real instance features rather than classifier weights or memory banks.
• Use random data augmentations per instance to enforce augmentation invariance across the same instance.
• Adopt a small batch approach where m instances are sampled and treated via a binary classification view using a Siamese network.
• Define probabilities P(i|x̂i) and P(i|xj) based on inner products of instance features and augmented features under a temperature parameter τ.
• Minimize the negative log-likelihood J combining P(i|x̂i) and 1−P(i|xj) over all instances in a batch to achieve augmentation invariance and spread-outness.
• Provide theoretical rationale showing how maximizing P(i|x̂i) and minimizing P(i|xj) encourage intra-instance invariance and inter-instance dispersion.

Experimental results

Research questions

• RQ1 Can instance-wise supervision substitute category-level labels for learning discriminative embeddings in an unsupervised setting?
• RQ2 Do data augmentation invariance and instance spread-outness jointly improve embedding quality for both seen and unseen categories?
• RQ3 Does directly optimizing over instance features via a softmax formulation yield faster convergence and better accuracy than weight- or memory-based methods?
• RQ4 How well do the learned embeddings generalize to unseen categories in fine-grained datasets?

Key findings

• Achieves state-of-the-art performance among unsupervised embedding methods on CIFAR-10 and STL-10 in kNN and linear evaluations.
• Demonstrates faster learning speed, reaching higher accuracy earlier (e.g., CIFAR-10 kNN 83.6% with 128-d embedding) and requiring fewer epochs to reach competitive accuracy than competitors.
• Outperforms exemplar-classifier and memory-bank softmax variants by directly optimizing over real instance features.
• Shows strong generalization to unseen categories on CUB200, Product, and Car196 datasets with competitive or superior retrieval (R@K) and clustering (NMI) metrics.
• Ablation studies confirm the importance of both data augmentation (especially RandomResizedCrop) and the hard negative strategy for performance gains.