[Paper Review] Insect cyborgs: Biological feature generators improve machine learning accuracy on limited data.

Charles B. Delahunt, J. Nathan Kutz|arXiv (Cornell University)|Aug 23, 2018

Neurobiology and Insect Physiology Research20 references4 citations

TL;DR

This paper proposes insect cyborgs—machine learning models enhanced by MothNet, a computational model of the moth olfactory system—as automatic feature generators to improve performance on limited-data tasks. By leveraging biological neural network principles like competitive inhibition and Hebbian plasticity, MothNet-generated features boosted standard ML classifiers' accuracy by 6% to 33% on vectorized MNIST, outperforming PCA, PLS, and pre-trained neural networks.

ABSTRACT

Machine learning (ML) classifiers always benefit from more informative input features. We seek to auto-generate stronger feature sets in order to address the difficulty that ML methods often experience given limited training data. A wide range of biological neural nets (BNNs) excel at fast learning, implying that they are adept at extracting informative features. We can thus look to BNNs for tools to improve ML performance in this low-data regime. The olfactory network learns new odors very rapidly, by means of three key elements: A competitive inhibition layer; a high-dimensional sparse plastic layer; and Hebbian updates of synaptic weights. In this work, we deployed MothNet, a computational model of the olfactory network, as an automatic feature generator: Attached as a front-end pre-processor, its Readout Neurons provided new features, derived from the original features, for use by standard ML classifiers. We found that these insect cyborgs, i.e. classifiers that are part-insect model and part-ML method, had significantly better performance than baseline ML methods alone on a vectorized MNIST dataset. The MothNet feature generator also substantially out-performed other feature generating methods such as PCA, PLS, and NNs, as well as pre-training to initialize NN weights. Cyborgs improved relative test set accuracy by an average of 6% to 33% depending on baseline ML accuracy, while relative reduction in test set error exceeded 50% for higher baseline accuracy ML models. These results indicate the potential value of BNN-inspired feature generators in the ML context.

Motivation & Objective

• Address the challenge of poor machine learning performance when training data is limited.
• Explore whether biological neural networks (BNNs), known for rapid learning, can inspire better feature extraction in ML.
• Develop a feature generation method that mimics the olfactory system’s efficiency in extracting informative features.
• Improve standard ML classifier performance on low-data regimes using biologically inspired preprocessing.

Proposed method

• Deploy MothNet, a computational model of the moth olfactory system, as a front-end pre-processor for machine learning pipelines.
• Use MothNet’s readout neurons to generate new, high-dimensional, sparse, and plastic features from original input data.
• Apply competitive inhibition and Hebbian synaptic weight updates within MothNet to enhance feature discrimination and adaptability.
• Feed MothNet-generated features into standard machine learning classifiers for downstream classification tasks.
• Train and evaluate the resulting insect cyborg models on the vectorized MNIST dataset to compare performance against baselines.
• Benchmark performance against PCA, PLS, neural networks, and pre-training methods using test set accuracy and error rates.

Experimental results

Research questions

• RQ1 Can a biologically inspired model of the insect olfactory system improve machine learning performance on limited training data?
• RQ2 How does MothNet-generated feature extraction compare to traditional dimensionality reduction techniques like PCA and PLS?
• RQ3 Does using MothNet as a feature generator outperform neural network pre-training in low-data regimes?
• RQ4 To what extent do insect cyborgs reduce test set error compared to standard ML models?
• RQ5 What is the impact of MothNet’s biological mechanisms—competitive inhibition, sparse plasticity, and Hebbian learning—on feature quality and classification accuracy?

Key findings

• Insect cyborgs, combining MothNet with standard ML classifiers, achieved a relative improvement in test set accuracy ranging from 6% to 33% compared to baseline models.
• The relative reduction in test set error exceeded 50% for machine learning models with higher baseline accuracy when enhanced by MothNet features.
• MothNet-generated features significantly outperformed those from PCA, PLS, and standard neural networks in low-data scenarios.
• The feature generation process using MothNet’s competitive inhibition and Hebbian plasticity produced more discriminative features than pre-training methods.
• The performance gains were most pronounced when baseline models had higher initial accuracy, indicating MothNet’s effectiveness in refining already strong models.
• The integration of MothNet as a pre-processor demonstrated consistent improvements across multiple evaluation metrics, validating its role as a robust feature generator.