[Paper Review] Unsupervised feature learning by augmenting single images

Alexey Dosovitskiy, Jost Tobias Springenberg+1 more|arXiv (Cornell University)|Dec 19, 2013

Advanced Image and Video Retrieval Techniques5 citations

TL;DR

This paper proposes an unsupervised feature learning method that uses data augmentation as the core training signal by treating random image patches as single-image surrogate classes. By applying diverse transformations to these patches and training a CNN to distinguish between them, the model learns powerful, transferable features that achieve competitive performance on STL-10, CIFAR-10, and Caltech-101 without any labeled data.

ABSTRACT

When deep learning is applied to visual object recognition, data augmentation is often used to generate additional training data without extra labeling cost. It helps to reduce overfitting and increase the performance of the algorithm. In this paper we investigate if it is possible to use data augmentation as the main component of an unsupervised feature learning architecture. To that end we sample a set of random image patches and declare each of them to be a separate single-image surrogate class. We then extend these trivial one-element classes by applying a variety of transformations to the initial 'seed' patches. Finally we train a convolutional neural network to discriminate between these surrogate classes. The feature representation learned by the network can then be used in various vision tasks. We find that this simple feature learning algorithm is surprisingly successful, achieving competitive classification results on several popular vision datasets (STL-10, CIFAR-10, Caltech-101).

Motivation & Objective

• To explore whether data augmentation alone can serve as the primary signal for unsupervised feature learning.
• To address the challenge of learning rich visual representations without labeled data in object recognition tasks.
• To develop a simple yet effective architecture that leverages image transformations to create surrogate classes for self-supervised learning.
• To evaluate the transferability and performance of features learned through this method on standard benchmark datasets.

Proposed method

• Random image patches are sampled from training images and treated as individual one-element classes, forming surrogate classes.
• Each patch is augmented using a variety of transformations such as color jittering, cropping, and flipping to generate multiple views of the same patch.
• A convolutional neural network is trained to classify these augmented patches into their respective surrogate classes, learning invariant features through contrastive learning.
• The feature representations extracted from the trained network are evaluated on downstream classification tasks using linear probes.
• The method relies solely on data augmentation and does not require explicit noise injection or complex contrastive objectives.

Experimental results

Research questions

• RQ1 Can data augmentation alone serve as the primary supervisory signal for unsupervised feature learning?
• RQ2 How effective is a self-supervised method based on single-image patch augmentation compared to existing contrastive or predictive pretraining approaches?
• RQ3 To what extent can features learned through this method generalize to downstream vision tasks on standard benchmarks?

Key findings

• The proposed method achieves competitive classification accuracy on STL-10, CIFAR-10, and Caltech-101 using only unsupervised pretraining with data augmentation.
• The model learns robust, transferable features despite the simplicity of treating each patch as a separate class.
• Performance is comparable to more complex self-supervised methods, demonstrating the effectiveness of augmentation as a primary learning signal.
• The approach generalizes well across different datasets, indicating strong feature quality and invariance learning.