[Paper Review] Learning-induced categorical perception in a neural network model

Christian Thériault, Fernanda Pérez-Gay Juárez+2 more|arXiv (Cornell University)|May 11, 2018

Categorization, perception, and language21 references3 citations

TL;DR

This paper proposes a neural network model that demonstrates learning-induced categorical perception (CP) through expansion of between-category and compression of within-category distances in hidden-unit space. By training on category-specific inputs, the model replicates CP effects without pre-existing categories, showing that CP emerges naturally from category learning via distributed representations.

ABSTRACT

In human cognition, the expansion of perceived between-category distances and compression of within-category distances is known as categorical perception (CP). There are several hypotheses about the causes of CP (e.g., language, learning, evolution) but no functional model. Whether CP is essential to categorisation or simply a by-product of it is not yet clear, but evidence is accumulating that CP can be induced by category learning. We provide a model for learning-induced CP as expansion and compression of distances in hidden-unit space in neural nets. Basic conditions from which the current model predicts CP are described, and clues as to how these conditions might generalize to more complex kinds of categorization begin to emerge.

Motivation & Objective

• To develop a functional neural network model that explains how categorical perception (CP) can emerge from category learning.
• To investigate whether CP is a by-product or a core mechanism of categorization in artificial neural networks.
• To identify the basic conditions under which CP arises in distributed representations during training.
• To demonstrate that CP can be induced purely through learning, without prior linguistic or evolutionary constraints.

Proposed method

• A feedforward neural network with multiple hidden layers is trained on stimuli belonging to distinct categories.
• The model learns to classify stimuli by adjusting weights during backpropagation, forming category-specific representations in hidden-unit space.
• Categorical perception is measured by analyzing inter- and intra-category distances in the hidden-unit activation space.
• Distances between stimuli from different categories increase (expansion), while distances within the same category decrease (compression), indicating CP.
• The model uses standard backpropagation and gradient descent for training, with no architectural modifications to enforce CP.
• The analysis focuses on the geometry of hidden-layer representations before and after training to detect CP effects.

Experimental results

Research questions

• RQ1 Can categorical perception emerge in a neural network purely through category learning, without predefined categories?
• RQ2 What specific conditions in the training process lead to the expansion of between-category and compression of within-category distances in hidden-unit space?
• RQ3 Is categorical perception a by-product of categorization or a fundamental mechanism enabling it in neural networks?
• RQ4 How do the geometric properties of hidden-unit representations change during category learning to produce CP?

Key findings

• The model exhibits clear categorical perception: between-category distances in hidden-unit space increase after training, while within-category distances decrease.
• CP emerges naturally from the learning process without any explicit mechanism to enforce category boundaries.
• The degree of CP correlates with the model's classification accuracy, suggesting a functional role in categorization.
• The model's hidden-unit representations become more separable after training, indicating improved category discrimination.
• The results support the hypothesis that CP is a consequence of category learning in distributed representations, not a prerequisite for it.