[Paper Review] Influence of the learning method in the performance of feedforward neural networks when the activity of neurons is modified

M. Konomi, G. M. Sacha|arXiv (Cornell University)|Apr 21, 2014

Neural Networks and Applications16 references3 citations

TL;DR

This paper investigates how different learning methods—back-propagation (BP) and an evolutionary algorithm (EA)—affect the robustness of feedforward neural networks when neuron activity is impaired. It demonstrates that EA-trained networks maintain significantly higher accuracy and faster convergence than BP networks, even when neurons are disabled post-training, suggesting EA is more resilient to neuron failure and better suited for modeling biologically plausible learning systems.

ABSTRACT

A method that allows us to give a different treatment to any neuron inside feedforward neural networks is presented. The algorithm has been implemented with two very different learning methods: a standard Back-propagation (BP) procedure and an evolutionary algorithm. First, we have demonstrated that the EA training method converges faster and gives more accurate results than BP. Then we have made a full analysis of the effects of turning off different combinations of neurons after the training phase. We demonstrate that EA is much more robust than BP for all the cases under study. Even in the case when two hidden neurons are lost, EA training is still able to give good average results. This difference implies that we must be very careful when pruning or redundancy effects are being studied since the network performance when losing neurons strongly depends on the training method. Moreover, the influence of the individual inputs will also depend on the training algorithm. Since EA keeps a good classification performance when units are lost, this method could be a good way to simulate biological learning systems since they must be robust against deficient neuron performance. Although biological systems are much more complex than the simulations shown in this article, we propose that a smart training strategy such as the one shown here could be considered as a first protection against the losing of a certain number of neurons.

Motivation & Objective

• To evaluate the impact of neuron activity modification on feedforward neural network performance.
• To compare the robustness of back-propagation (BP) and evolutionary algorithm (EA) training methods under neuron failure conditions.
• To investigate whether EA can better simulate biological learning systems due to inherent resilience against neuron loss.
• To analyze how input influence varies with training method when neurons are disabled.
• To determine if training method choice critically affects conclusions in pruning or redundancy studies.

Proposed method

• Implemented a neural network framework allowing selective deactivation of individual neurons post-training.
• Trained networks using two distinct learning methods: standard back-propagation (BP) and an evolutionary algorithm (EA).
• Applied EA with fitness-based selection, mutation, and crossover to optimize network weights and topology.
• Conducted systematic neuron deactivation experiments on both BP- and EA-trained networks across multiple configurations.
• Measured classification accuracy and convergence speed to compare performance under neuron failure.
• Analyzed input sensitivity and network robustness across different neuron combinations and training methods.

Experimental results

Research questions

• RQ1 How does the choice of learning method (BP vs. EA) affect network performance when neurons are deactivated after training?
• RQ2 Does EA training lead to faster convergence and higher accuracy compared to BP under the same conditions?
• RQ3 To what extent does EA maintain classification performance when hidden neurons are lost?
• RQ4 How does the influence of individual inputs change depending on the training algorithm used?
• RQ5 Can EA-trained networks better simulate biological systems' resilience to neuron dysfunction?

Key findings

• The evolutionary algorithm (EA) training method converged faster and achieved higher accuracy than back-propagation (BP) during the training phase.
• Even when two hidden neurons were deactivated, EA-trained networks maintained good average classification performance, demonstrating superior robustness.
• BP-trained networks showed a significant performance drop under neuron deactivation, especially with multiple neurons disabled.
• The influence of individual inputs on network output was more dependent on the training method, indicating that training algorithm choice affects input sensitivity.
• EA-trained networks exhibited greater resilience to neuron failure, suggesting that the training method critically influences conclusions in pruning and redundancy studies.
• The results imply that EA could serve as a more biologically plausible training strategy due to its inherent robustness against neuron loss.