[Paper Review] Evolving Plasticity for Autonomous Learning under Changing Environmental Conditions

Anil Yaman, Giovanni Iacca+4 more|University of Twente Research Information|Apr 2, 2019

Neural Networks and Applications45 references18 citations

TL;DR

This paper proposes an evolutionary approach to discover interpretable, binary-encoded Hebbian plasticity rules that enable artificial neural networks to autonomously adapt during continuous online learning under changing environmental conditions. Using genetic algorithms to optimize simple, rule-based synaptic modifications, the method achieves performance comparable to offline optimization (74% of hill climbing performance) while providing insights into adaptive behaviors in foraging and prey-predator tasks.

ABSTRACT

A fundamental aspect of learning in biological neural networks is the plasticity property which allows them to modify their configurations during their lifetime. Hebbian learning is a biologically plausible mechanism for modeling the plasticity property in artificial neural networks (ANNs), based on the local interactions of neurons. However, the emergence of a coherent global learning behavior from local Hebbian plasticity rules is not very well understood. The goal of this work is to discover interpretable local Hebbian learning rules that can provide autonomous global learning. To achieve this, we use a discrete representation to encode the learning rules in a finite search space. These rules are then used to perform synaptic changes, based on the local interactions of the neurons. We employ genetic algorithms to optimize these rules to allow learning on two separate tasks (a foraging and a prey-predator scenario) in online lifetime learning settings. The resulting evolved rules converged into a set of well-defined interpretable types, that are thoroughly discussed. Notably, the performance of these rules, while adapting the ANNs during the learning tasks, is comparable to that of offline learning methods such as hill climbing.

Motivation & Objective

• To develop a method for evolving interpretable synaptic plasticity rules that enable autonomous learning in artificial neural networks under changing environmental conditions.
• To address the challenge of achieving coherent global learning behavior from local, biologically plausible Hebbian plasticity rules.
• To reduce the complexity of evolved plasticity rules while maintaining high performance in online, lifelong learning scenarios.
• To investigate how evolved plasticity rules adapt to different task dynamics, such as deterministic foraging versus stochastic prey-predator environments.
• To provide interpretability into the learning mechanisms of evolving neural networks by using a discrete, binary rule representation.

Proposed method

• Represent synaptic plasticity rules in a discrete, binary format encoding pairwise neuron activation interactions.
• Use genetic algorithms to evolve these binary rules over multiple generations to optimize performance on foraging and prey-predator tasks.
• Apply the evolved rules to perform online, lifetime synaptic weight updates based on local pre- and post-synaptic neuron activations.
• Define two seasonal environmental shifts in each task to test adaptation capabilities during dynamic conditions.
• Compare evolved plasticity rule performance against hand-coded rules and hill-climbing optimized networks to establish an upper performance bound.
• Visually analyze agent behaviors to interpret differences in evolved rule strategies, such as straight-line vs. diagonal movement patterns.

Experimental results

Research questions

• RQ1 Can interpretable, binary-encoded Hebbian plasticity rules be evolved to enable autonomous, lifelong learning in artificial neural networks under dynamic environmental conditions?
• RQ2 How do evolved plasticity rules differ in structure and behavior between deterministic (foraging) and stochastic (prey-predator) learning tasks?
• RQ3 To what extent can evolved plasticity rules achieve performance comparable to offline optimization methods like hill climbing in continuous learning settings?
• RQ4 What behavioral strategies emerge from evolved plasticity rules, and how do they relate to environmental dynamics such as reward structure and stochasticity?
• RQ5 Can the evolved rules avoid catastrophic forgetting while maintaining high performance across changing task conditions?

Key findings

• The evolved synaptic plasticity rules (ESP) achieved approximately 74% of the performance of hand-coded hill-climbing optimized networks in both foraging and prey-predator tasks, demonstrating strong competitive performance in online learning.
• In the foraging task, the best ESP rule performed synaptic changes only when the network produced undesired outputs (negative reinforcement), suggesting a strategy to avoid overwriting learned knowledge.
• In the prey-predator task, the best ESP rules exhibited higher learning rates (one order of magnitude higher than in foraging) due to the need for rapid adaptation to stochastic environmental dynamics.
• Two distinct behavioral strategies emerged from the top-performing ESP rules: one resembling the behavior of hand-coded agents (straight-line movement), and another favoring diagonal movement, which proved advantageous in evading randomly moving predators and preys.
• The evolved rules were interpretable and structurally distinct between tasks, indicating that the evolutionary process adapted rule complexity to task-specific demands.
• Visual inspection of agent behaviors confirmed that different ESP rules led to qualitatively different strategies, even when fitness scores were similar, highlighting the diversity of evolved solutions.