[Paper Review] Unsupervised Learning of Invariant Representations in Hierarchical Architectures

Fabio Anselmi, Joel Z. Leibo+4 more|arXiv (Cornell University)|Nov 17, 2013

Image Retrieval and Classification TechniquesComputer Science56 references65 citations

TL;DR

This paper proposes a hierarchical, unsupervised learning framework based on Hubel-Wiesel-like modules that automatically learn invariant, discriminative representations for visual object recognition with low sample complexity. By computing invariant signatures through pooling dot-product distributions between image patches and learned templates, the architecture achieves translation, scale, and pose invariance while maintaining discriminability, enabling recognition from few labeled examples—mirroring principles of the ventral visual stream in primates.

ABSTRACT

The present phase of Machine Learning is characterized by supervised learning algorithms relying on large sets of labeled examples ($n o \infty$). The next phase is likely to focus on algorithms capable of learning from very few labeled examples ($n o 1$), like humans seem able to do. We propose an approach to this problem and describe the underlying theory, based on the unsupervised, automatic learning of a ``good'' representation for supervised learning, characterized by small sample complexity ($n$). We consider the case of visual object recognition though the theory applies to other domains. The starting point is the conjecture, proved in specific cases, that image representations which are invariant to translations, scaling and other transformations can considerably reduce the sample complexity of learning. We prove that an invariant and unique (discriminative) signature can be computed for each image patch, $I$, in terms of empirical distributions of the dot-products between $I$ and a set of templates stored during unsupervised learning. A module performing filtering and pooling, like the simple and complex cells described by Hubel and Wiesel, can compute such estimates. Hierarchical architectures consisting of this basic Hubel-Wiesel moduli inherit its properties of invariance, stability, and discriminability while capturing the compositional organization of the visual world in terms of wholes and parts. The theory extends existing deep learning convolutional architectures for image and speech recognition. It also suggests that the main computational goal of the ventral stream of visual cortex is to provide a hierarchical representation of new objects/images which is invariant to transformations, stable, and discriminative for recognition---and that this representation may be continuously learned in an unsupervised way during development and visual experience.

Motivation & Objective

• To address the challenge of learning from very few labeled examples (n→1), as humans do, by reducing sample complexity in visual recognition.
• To develop a theory for unsupervised, automatic learning of invariant representations that are stable and discriminative.
• To formalize how hierarchical architectures of Hubel-Wiesel modules can achieve invariance to local affine transformations, including translation, scaling, and viewpoint changes.
• To bridge insights from primate visual cortex with deep learning, suggesting that the ventral stream’s core function is to build such invariant representations through continuous unsupervised learning.

Proposed method

• The method uses a hierarchical architecture composed of Hubel-Wiesel (HW) modules, each consisting of simple cells (filtering) and complex cells (pooling) to compute invariant signatures.
• Each image patch is represented by a signature vector derived from the empirical distribution of dot-products between the patch and a set of learned templates stored during unsupervised pre-training.
• Pooling over local receptive fields—using sum or max operations—produces invariance to translations and scaling, emulating complex cell behavior.
• The architecture inherits invariance, stability, and discriminability through hierarchical composition, where higher layers integrate lower-level invariant features into global representations.
• The framework extends standard convolutional networks by embedding invariance as a structural property rather than a learned outcome, with templates learned unsupervised from unlabeled data.
• For complex transformations like 3D rotation or in-plane pose variation, the method introduces specialized pooling layers that pool over stored views of objects undergoing such transformations.

Experimental results

Research questions

• RQ1 Can unsupervised learning of invariant representations significantly reduce the number of labeled examples needed for accurate visual recognition?
• RQ2 How can hierarchical architectures of Hubel-Wiesel modules achieve invariance to local affine transformations while preserving discriminative information?
• RQ3 To what extent can invariant signatures be computed as unique, stable, and discriminative representations using empirical distributions of dot-products with learned templates?
• RQ4 Does the proposed architecture mimic the computational function of the primate ventral visual stream in generating transformation-invariant, hierarchical representations?
• RQ5 Can invariance to complex transformations like 3D rotation be achieved through learned pooling over stored views, rather than architectural design alone?

Key findings

• The proposed hierarchical architecture achieves invariance to local affine transformations—including translation, scaling, and viewpoint changes—by construction, without requiring labeled data for invariance.
• The signature vectors computed by the HW modules are invariant to deformations within the receptive field, as demonstrated by consistent signature norms under image distortions such as eye proximity changes.
• The signature vectors are Lipschitz-stable with respect to image deformations, meaning small input changes produce bounded changes in the representation, ensuring robustness.
• The architecture maintains discriminability: signatures from different images (e.g., two faces) remain distinct even when images are translated across the visual field, enabling recognition from a single example.
• Empirical results on HMAX-like implementations show that layer-2 signatures remain invariant to global translation and discriminative between different faces, with Euclidean distances between signatures reflecting image similarity.
• Models incorporating specialized pooling for 3D rotation and in-plane pose variations achieved state-of-the-art performance on the Labeled Faces in the Wild dataset, recognizing novel faces from a single view with robustness to rotation in depth.