[Paper Review] Concerning the Neural Code

Christoph von der Malsburg|arXiv (Cornell University)|Nov 3, 2018

Neural dynamics and brain function25 references4 citations

TL;DR

This paper proposes that the neural code consists of self-organizing, structured net fragments that form dynamic, self-consistent networks representing mental states. These networks emerge through sensory-driven self-organization, enabling perception, memory, and cognition by maintaining consistency across signal pathways and reasoning chains.

ABSTRACT

The central problem with understanding brain and mind is the neural code issue: understanding the matter of our brain as basis for the phenomena of our mind. The richness with which our mind represents our environment, the parsimony of genetic data, the tremendous efficiency with which the brain learns from scant sensory input and the creativity with which our mind constructs mental worlds all speak in favor of mind as an emergent phenomenon. This raises the further issue of how the neural code supports these processes of organization. The central point of this communication is that the neural code has the form of structured net fragments that are formed by network self-organization, activate and de-activate on the functional time scale, and spontaneously combine to form larger nets with the same basic structure.

Motivation & Objective

• To resolve the mind-body problem by identifying a common functional principle underlying brain and mind.
• To explain how the brain generates rich mental representations from sparse sensory input using self-organized neural networks.
• To clarify the nature of neural coding, memory, and learning mechanisms in terms of network structure and dynamics.
• To provide a framework for understanding consciousness and mental construction as emergent properties of consistent neural network states.

Proposed method

• Models the brain's functional state as a self-consistent network of active neurons and connections, analogous to a logical proof system.
• Proposes that memory is encoded as an overlay of structural net fragments—persistent connectivity patterns formed through self-organization.
• Introduces the idea that sensory input drives network self-organization, favoring consistency across alternative signal pathways.
• Uses computer simulations to demonstrate the simultaneous emergence of projection pathways and object models before and after birth.
• Interprets active neurons as elementary logical propositions and active connections as reasoning rules, forming a coherent web of inference.
• Applies the concept of homeomorphism to identify schema-like net fragments in scenes, enabling schema-based behavior and learning.

Experimental results

Research questions

• RQ1 How does the brain generate the rich, coherent representations of reality and imagination from limited sensory input?
• RQ2 What is the functional role of neural network self-organization in shaping the neural code and mental states?
• RQ3 How are memory traces formed and stored in the brain at the level of network structure?
• RQ4 What mechanism ensures consistency between perception, prediction, and action in a dynamic environment?
• RQ5 How can consciousness and mental construction emerge from the dynamics of neural network consistency?

Key findings

• The neural code is structured as self-consistent net fragments that dynamically form, activate, and deactivate on the functional time scale.
• Memory is encoded as an overlay of structural net fragments, representing stored connectivity patterns rather than isolated synapses.
• Network self-organization, driven by sensory input and favoring consistency, is the core mechanism for forming memory and mental content.
• Active neural networks can be interpreted as logical systems where neurons represent propositions and connections represent inference rules.
• Consistency between sensory signals and internal predictions underlies perceptual confidence and naive realism in perception.
• Schema-based behavior arises from genetically pre-wired net fragments that can be matched to environmental features via homeomorphism and refined through learning.