[Paper Review] A Model for Auto-Programming for General Purposes

Juyang Weng|arXiv (Cornell University)|Oct 12, 2018

Computability, Logic, AI Algorithms80 references4 citations

TL;DR

This paper proposes GENISAMA, a neural network model that learns to auto-program for general purposes by autonomously acquiring and refining Turing Machine-like behaviors through lifelong, incremental, and error-free learning from a teacher. Unlike traditional symbolic systems, GENISAMA uses grounded, emergent patterns instead of fixed symbols, enabling it to self-organize, adapt to new environments, and achieve optimal performance under resource constraints, thus extending the Church-Turing thesis to automatic programming.

ABSTRACT

The Universal Turing Machine (TM) is a model for VonNeumann computers --- general-purpose computers. A human brain can inside-skull-automatically learn a universal TM so that he acts as a general-purpose computer and writes a computer program for any practical purposes. It is unknown whether a machine can accomplish the same. This theoretical work shows how the Developmental Network (DN) can accomplish this. Unlike a traditional TM, the TM learned by DN is a super TM --- Grounded, Emergent, Natural, Incremental, Skulled, Attentive, Motivated, and Abstractive (GENISAMA). A DN is free of any central controller (e.g., Master Map, convolution, or error back-propagation). Its learning from a teacher TM is one transition observation at a time, immediate, and error-free until all its neurons have been initialized by early observed teacher transitions. From that point on, the DN is no longer error-free but is always optimal at every time instance in the sense of maximal likelihood, conditioned on its limited computational resources and the learning experience. This letter also extends the Church-Turing thesis to automatic programming for general purposes and sketchily proved it.

Motivation & Objective

• To identify the minimal set of mechanisms enabling a machine to learn auto-programming for general purposes, emulating biological brains.
• To address the limitations of symbolic Turing Machines in handling dynamic, real-world inputs without pre-programmed symbol sets.
• To develop a neural network architecture that learns from teacher transitions in real time, without central controllers or backpropagation.
• To extend the Church-Turing thesis to include automatic programming through a developmental, self-organizing system.
• To model how nature and nurture interact in lifelong cognitive development via a task-nonspecific developmental program (DP) and task-specific pattern-based learning.

Proposed method

• The model uses a Developmental Network (DN) that learns one transition at a time from a teacher Turing Machine, initializing neurons incrementally without error until full initialization.
• It replaces symbolic inputs with natural patterns (e.g., image patches) that emerge from sensory input, enabling grounding in real-world data.
• The system employs a self-wiring mechanism where synapses grow or shrink based on a novelty signal derived from the ratio of local error to average synaptic deviation, modeled via a synaptogenic factor.
• Motivation is implemented through two pathways: (a) pain/pleasure signals that modulate learning rates and motor output via neuromodulators like serotonin and dopamine; and (b) synaptic maintenance that drives curiosity and segmentation of objects/events.
• The control mechanism is an Attentive Agent FA that selects subsets of inputs (e.g., relevant words or objects) dynamically, enabling selective attention without symbolic parsing.
• The model is structured as a two-way bridge between sensory (X) and motor (Z) banks via a hidden representation (Y), with hierarchical subareas in X, Y, and Z that adapt postnatally.

Experimental results

Research questions

• RQ1 Can a neural network learn to auto-program for general purposes without symbolic programming or central control?
• RQ2 How can a machine learn a universal Turing Machine behavior incrementally and error-free from observed transitions?
• RQ3 What mechanisms allow a system to ground abstract computation in real-world sensory patterns rather than arbitrary symbols?
• RQ4 How can motivation and attention be modeled biologically to support lifelong, self-directed learning and abstraction?
• RQ5 How do nature (developmental program) and nurture (experience) interact in a self-organizing system to enable general-purpose programming?

Key findings

• The model achieves error-free learning during initial neuron initialization, after which it remains optimal at every time step under computational constraints, maximizing likelihood.
• The control of any Turing Machine, including universal ones, is formally proven to be an Attentive Agent FA, enabling dynamic input selection and attention.
• GENISAMA TMs can auto-program for general purposes, unlike symbolic TMs, grounded symbolic machines, or prior neural networks, as confirmed by Table V.
• The system naturally learns to represent and manipulate complex patterns (e.g., car signs, words) without pre-defined symbols, enabling adaptation to new environments without reprogramming.
• The model supports lifelong learning through a synaptogenic factor that adjusts synaptic strength based on error novelty, enabling object/event segmentation and curiosity-driven exploration.
• The developmental program (DP) is task-nonspecific and body-specific, while learned knowledge emerges from interactions between the environment and the DN, illustrating a clear separation of nature and nurture.