[Paper Review] Unsupervised Deep Learning by Neighbourhood Discovery

Jiabo Huang, Qi Dong+2 more|arXiv (Cornell University)|Apr 25, 2019

Advanced Image and Video Retrieval TechniquesComputer Science68 citations

TL;DR

AND proposes Anchor Neighbourhood Discovery, an unsupervised method that learns discriminative features by progressively discovering class-consistent local neighbourhoods anchored to individual samples, enabling end-to-end training without labels.

ABSTRACT

Deep convolutional neural networks (CNNs) have demonstrated remarkable success in computer vision by supervisedly learning strong visual feature representations. However, training CNNs relies heavily on the availability of exhaustive training data annotations, limiting significantly their deployment and scalability in many application scenarios. In this work, we introduce a generic unsupervised deep learning approach to training deep models without the need for any manual label supervision. Specifically, we progressively discover sample anchored/centred neighbourhoods to reason and learn the underlying class decision boundaries iteratively and accumulatively. Every single neighbourhood is specially formulated so that all the member samples can share the same unseen class labels at high probability for facilitating the extraction of class discriminative feature representations during training. Experiments on image classification show the performance advantages of the proposed method over the state-of-the-art unsupervised learning models on six benchmarks including both coarse-grained and fine-grained object image categorisation.

Motivation & Objective

• Motivate unsupervised learning to scale beyond labeled data by discovering local class-consistent neighbourhoods.
• Introduce Anchor Neighbourhood Discovery (AND) to combine clustering and sample specificity advantages while mitigating their weaknesses.
• Develop a curriculum-based, progressive neighbourhood discovery to maximize class consistency during training.
• Provide an end-to-end differentiable learning framework with a neighbourhood supervision loss.
• Demonstrate state-of-the-art performance on multiple coarse- and fine-grained image classification benchmarks.

Proposed method

• Define anchor neighbourhoods as k-nearest neighbour sets anchored to each sample and assume locality implies shared unseen class labels.
• Initialize with instance-specific supervision (treat each sample as its own class) using a non-parametric softmax cross-entropy loss.
• Introduce a neighbourhood supervision loss that encourages within-neighbourhood label consistency using a differentiable aggregation of neighbour similarities.
• Employ curriculum learning to progressively select more class-consistent neighbourhoods based on entropy of the similarity distribution, in R rounds.
• Update feature memory with exponential moving average to enable stable neighbourhood discovery across iterations.
• Optimize the joint objective combining instance-level and neighbourhood-level losses in a differentiable end-to-end training loop.

Experimental results

Research questions

• RQ1 Can local, class-consistent neighbourhoods provide reliable supervision signals for unsupervised deep learning without manual labels?
• RQ2 Does a progressive curriculum for neighbourhood discovery improve the quality of learned representations compared to one-shot neighbourhoods or pure instance-level learning?
• RQ3 How does AND compare to clustering, self-supervised, and instance-based unsupervised methods across standard benchmarks?
• RQ4 What effect do neighbourhood size, network capacity, and curriculum rounds have on the quality of the learned features?

Key findings

• AND often achieves superior unsupervised representation learning compared to several state-of-the-art methods on multiple benchmarks (e.g., CIFAR10, CIFAR100, SVHN, ImageNet, and fine-grained datasets).
• Smaller neighbourhoods (k=1) tend to yield better class-consistent neighbourhoods and higher performance.
• A curriculum of progressive neighbourhood discovery (across R rounds) improves learning over a one-off neighbourhood discovery strategy.
• Stronger network backbones (e.g., ResNet variants) and better initialisation enhance AND performance.
• The approach yields discriminative features that directly improve kNN-based classification without extra post-processing.