[Paper Review] Intrinsically Motivated Learning of Visual Motion Perception and Smooth Pursuit

Chong Zhang, Yu Zhao+2 more|arXiv (Cornell University)|Feb 14, 2014

Visual perception and processing mechanisms17 references3 citations

TL;DR

This paper proposes a unified framework combining sparse coding and reinforcement learning to enable an active eye to co-develop visual motion perception and smooth pursuit behavior through intrinsic motivation. By optimizing neural encoding fidelity under resource constraints, the model spontaneously generates direction-selective V1-like neurons and coordinated eye movements, demonstrating a biologically plausible mechanism for perception-action integration.

ABSTRACT

We extend the framework of efficient coding, which has been used to model the development of sensory processing in isolation, to model the development of the perception/action cycle. Our extension combines sparse coding and reinforcement learning so that sensory processing and behavior co-develop to optimize a shared intrinsic motivational signal: the fidelity of the neural encoding of the sensory input under resource constraints. Applying this framework to a model system consisting of an active eye behaving in a time varying environment, we find that this generic principle leads to the simultaneous development of both smooth pursuit behavior and model neurons whose properties are similar to those of primary visual cortical neurons selective for different directions of visual motion. We suggest that this general principle may form the basis for a unified and integrated explanation of many perception/action loops.

Motivation & Objective

• To unify the development of sensory perception and motor behavior in a single learning framework.
• To model how intrinsic motivation—defined as encoding fidelity under resource limits—drives the co-development of visual processing and smooth pursuit.
• To investigate whether a generic learning principle can yield biologically plausible neural and behavioral outcomes in an active vision system.
• To extend efficient coding theory beyond passive sensory processing to include active perception and action.

Proposed method

• The framework integrates sparse coding for efficient neural representation of visual input with reinforcement learning to guide behavior.
• A shared intrinsic reward signal maximizes the fidelity of neural encoding under constraints on neural resources.
• The agent (an active eye) learns to track moving stimuli in a time-varying environment through policy optimization.
• Neural representations are updated via sparse coding to minimize reconstruction error while maintaining sparsity.
• Behavioral policy is learned via policy gradient reinforcement learning to maximize the intrinsic reward.
• The system evolves both direction-selective V1-like neurons and smooth pursuit behavior simultaneously.

Experimental results

Research questions

• RQ1 Can a single intrinsic motivation signal drive the co-development of visual motion perception and smooth pursuit behavior?
• RQ2 Does optimizing neural encoding fidelity under resource constraints lead to biologically plausible neural selectivity?
• RQ3 Can an active agent learn to track moving stimuli through intrinsic learning without external rewards?
• RQ4 How does the integration of perception and action emerge from a unified learning principle?

Key findings

• The model successfully develops direction-selective neurons resembling simple cells in primary visual cortex (V1).
• Smooth pursuit behavior emerges naturally as a consequence of the intrinsic learning objective.
• The system achieves high-fidelity neural encoding of visual motion with sparse, efficient representations.
• The co-development of perception and action is driven solely by intrinsic motivation, without external rewards.