[Paper Review] Can Single Neurons Solve MNIST? The Computational Power of Biological Dendritic Trees

Ilenna Jones, Konrad P. Körding|arXiv (Cornell University)|Sep 2, 2020

Neural dynamics and brain function45 references12 citations

TL;DR

This paper proposes a biologically inspired neuron model with nonlinear dendritic trees composed of thresholded linear units, demonstrating that such single neurons can solve complex image classification tasks like MNIST and CIFAR-10. By repeating inputs across multiple dendritic branches, the model achieves performance close to fully connected neural networks, revealing that dendritic nonlinearities and synaptic repetition significantly enhance computational power beyond traditional point-neuron models.

ABSTRACT

Physiological experiments have highlighted how the dendrites of biological neurons can nonlinearly process distributed synaptic inputs. This is in stark contrast to units in artificial neural networks that are generally linear apart from an output nonlinearity. If dendritic trees can be nonlinear, biological neurons may have far more computational power than their artificial counterparts. Here we use a simple model where the dendrite is implemented as a sequence of thresholded linear units. We find that such dendrites can readily solve machine learning problems, such as MNIST or CIFAR-10, and that they benefit from having the same input onto several branches of the dendritic tree. This dendrite model is a special case of sparse network. This work suggests that popular neuron models may severely underestimate the computational power enabled by the biological fact of nonlinear dendrites and multiple synapses per pair of neurons. The next generation of artificial neural networks may significantly benefit from these biologically inspired dendritic architectures.

Motivation & Objective

• To investigate whether single biological neurons with nonlinear dendritic trees can perform complex machine learning tasks like MNIST classification.
• To evaluate the impact of repeated synaptic inputs on the computational capacity of dendritic neuron models.
• To compare the performance of dendrite-structured models against linear classifiers and fully connected neural networks (FCNNs).
• To explore whether biologically plausible dendritic structures and synaptic repetition can enhance learning in artificial neural networks.
• To challenge the assumption that point-neuron models in artificial neural networks accurately reflect the computational power of real biological neurons.

Proposed method

• The model uses a binary tree structure where each node is a thresholded linear unit, simulating a dendritic compartment with nonlinear integration.
• Each input is distributed across multiple branches of the dendritic tree, with the number of repetitions controlled by a parameter k.
• The network is trained using standard deep learning optimization on 7 high-dimensional binary image classification tasks, including MNIST and EMNIST.
• Performance is evaluated by comparing the k-tree model to a linear classifier (lower bound) and a fully connected neural network (upper bound) with similar architecture.
• The model enforces a sparse, non-weight-shared structure to reflect biological constraints, with no convolution or shared weights.
• The input is vectorized and ordered to test sensitivity to input sequence, with random ordering shown to reduce performance.

Experimental results

Research questions

• RQ1 Can a single neuron with a nonlinear dendritic tree architecture solve the MNIST image classification task?
• RQ2 How does the repetition of synaptic inputs across multiple dendritic branches affect the computational performance of a single neuron model?
• RQ3 To what extent can a biologically constrained, sparse neural network model achieve performance comparable to a fully connected neural network?
• RQ4 Does the dendritic tree structure with thresholded linear units provide a significant performance advantage over linear point-neuron models?
• RQ5 How does input ordering influence the performance of the dendritic neuron model?

Key findings

• The k-tree model with k=32 achieved performance on MNIST that was nearly identical to a comparable fully connected neural network (FCNN).
• On FMNIST, EMNIST, and USPS tasks, the 32-tree model matched the performance of the FCNN, despite its highly constrained architecture.
• Performance increased consistently with k, from k=1 to k=32, across all tasks, indicating that repeated inputs enhance computational capacity.
• The model outperformed the linear classifier on all tasks, demonstrating that dendritic nonlinearities significantly improve performance.
• Randomizing the input order reduced model performance, indicating that input structure and sequence matter for the model's effectiveness.
• The results suggest that biological dendritic structures and repeated synaptic inputs are critical for enhancing single-neuron computation, challenging the limitations of point-neuron models.