[Paper Review] Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination

Zhirong Wu, Yuanjun Xiong+2 more|arXiv (Cornell University)|May 5, 2018

Domain Adaptation and Few-Shot LearningComputer Science46 references172 citations

TL;DR

Learn an unsupervised image representation by discriminating each instance non-parametrically, using memory bank embeddings and noise-contrastive estimation, achieving strong ImageNet and Places results and good transfer to semi-supervised learning and object detection.

ABSTRACT

Neural net classifiers trained on data with annotated class labels can also capture apparent visual similarity among categories without being directed to do so. We study whether this observation can be extended beyond the conventional domain of supervised learning: Can we learn a good feature representation that captures apparent similarity among instances, instead of classes, by merely asking the feature to be discriminative of individual instances? We formulate this intuition as a non-parametric classification problem at the instance-level, and use noise-contrastive estimation to tackle the computational challenges imposed by the large number of instance classes. Our experimental results demonstrate that, under unsupervised learning settings, our method surpasses the state-of-the-art on ImageNet classification by a large margin. Our method is also remarkable for consistently improving test performance with more training data and better network architectures. By fine-tuning the learned feature, we further obtain competitive results for semi-supervised learning and object detection tasks. Our non-parametric model is highly compact: With 128 features per image, our method requires only 600MB storage for a million images, enabling fast nearest neighbour retrieval at the run time.

Motivation & Objective

• Motivate learning image representations by discriminating individual instances rather than semantic classes.
• Develop a scalable non-parametric softmax classifier for large-scale instance discrimination.
• Stabilize training with noise-contrastive estimation and proximal regularization.
• Show that the learned features generalize to semi-supervised learning and object detection.
• Demonstrate the efficiency and compactness of the learned 128-dimensional embeddings.

Proposed method

• Formulate instance-level discrimination as a non-parametric softmax over all training instances with L2-normalized features.
• Maintain a memory bank V of instance embeddings to compute P(i|v) without storing per-class weights.
• Use noise-contrastive estimation (NCE) to approximate the softmax with a noise distribution, reducing cost from O(n) to O(1) per sample.
• Apply proximal regularization to stabilize optimization by penalizing large changes in representations across iterations.
• Classify test images using k-nearest neighbors with cosine similarity to memory bank embeddings, enabling consistency between training and testing.

Experimental results

Research questions

• RQ1 Can discriminating individual instances in an unsupervised setting learn a feature space that preserves apparent instance similarity?
• RQ2 Does a non-parametric softmax with a memory bank outperform parametric softmax in unsupervised feature learning?
• RQ3 How do NCE and proximal regularization affect training stability and representation quality?
• RQ4 Do the learned features transfer well to semi-supervised tasks and object detection?

Key findings

• The non-parametric softmax with an instance memory bank improves CIFAR-10 classification by notable margins over parametric softmax.
• On ImageNet, the method achieves 46.5% top-1 accuracy with linear evaluation and 41.0–46.5% with kNN across architectures, outperforming several unsupervised baselines.
• The approach yields strong generalization to Places 205 with 41.6–45.5% top-1 depending on protocol and architecture.
• Embedding size around 128 dimensions provides a compact representation with ~600 MB for 1 million images and fast nearest-neighbor retrieval (~20 ms per image).
• Semi-supervised learning with limited labeled data benefits significantly, often outperforming supervised learning trained on the same small labeled subset.
• For object detection, the method achieves competitive mAP on PASCAL VOC 2007, improving with deeper networks (e.g., 65.4% mAP with ResNet-50).