[Paper Review] A Connectome Based Hexagonal Lattice Convolutional Network Model of the Drosophila Visual System

Fabian Tschopp, Michael B. Reiser+1 more|arXiv (Cornell University)|Jun 12, 2018

Neurobiology and Insect Physiology Research30 references18 citations

TL;DR

This paper proposes a hexagonal lattice convolutional network model of the Drosophila visual system based on electron microscopy connectome reconstructions. Using backpropagation through time to train the network on object tracking in natural videos, the connectome-initialized model spontaneously developed biologically accurate orientation and direction selectivity in T4 neurons—properties absent in randomly initialized networks—demonstrating that circuit function can be predicted from structural connectivity alone.

ABSTRACT

What can we learn from a connectome? We constructed a simplified model of the first two stages of the fly visual system, the lamina and medulla. The resulting hexagonal lattice convolutional network was trained using backpropagation through time to perform object tracking in natural scene videos. Networks initialized with weights from connectome reconstructions automatically discovered well-known orientation and direction selectivity properties in T4 neurons and their inputs, while networks initialized at random did not. Our work is the first demonstration, that knowledge of the connectome can enable in silico predictions of the functional properties of individual neurons in a circuit, leading to an understanding of circuit function from structure alone.

Motivation & Objective

• To test whether neural circuit function can be predicted from structural connectivity alone, using the Drosophila visual system as a model.
• To investigate whether connectome-derived weights enable the emergence of known functional properties like direction and orientation selectivity in T4 neurons.
• To compare connectome-initialized networks with randomly initialized ones to assess the role of structural constraints in shaping functional computation.
• To evaluate the robustness of connectome-based models under noise and incomplete connectivity.
• To explore the feasibility of deriving functional predictions from EM-based connectomes without prior functional assumptions.

Proposed method

• Constructed a hexagonal lattice convolutional network model of the lamina and medulla layers using published EM reconstructions of 43 neuron types.
• Mapped photoreceptor inputs (R1-R8) to their target columns in the lamina and medulla, enforcing spatial invariance via tiling of local connectome motifs.
• Initialized network weights using synapse counts from the connectome (Takemura et al., 2017) to reflect biological connectivity patterns.
• Trained the network via backpropagation through time on object tracking in natural scene videos, using a 3-stage fully connected decoder to predict 4 variables.
• Simplified complex cell types (e.g., Tm3, Lawf2) and omitted unconnected or poorly reconstructed cells (e.g., glia, Lawf1) due to incomplete data.
• Used RNA-seq data to assign excitatory/inhibitory properties to neurons and synapses, improving biophysical realism.

Experimental results

Research questions

• RQ1 Can a neural network model initialized with connectome-derived weights spontaneously develop known functional properties like direction and orientation selectivity in T4 neurons?
• RQ2 Does the connectome-based initialization lead to more biologically plausible and robust functional responses compared to random initialization?
• RQ3 To what extent does the accuracy of the connectome reconstruction influence the model’s ability to recover known physiological tuning properties?
• RQ4 How does the network’s functional performance change under noise or incomplete connectivity?
• RQ5 Can a proxy task like object tracking in natural videos lead to the emergence of complex neural computations without explicit functional supervision?

Key findings

• Networks initialized with connectome-derived weights successfully recapitulated the known direction selectivity of T4a–T4d neurons (preferred directions: 192°, 359°, 51°, 275°), while randomly initialized networks did not.
• The connectome-based model achieved high direction selectivity index (DSI) scores for T4a–T4d, with DSI values exceeding 0.7 in trained simulations, indicating strong tuning.
• Randomly initialized networks occasionally developed direction selectivity at non-physiological angles (e.g., T3 at 10°), but these responses were unstable and sensitive to noise.
• The connectome-initialized network remained robust under up to 40% multiplicative noise, maintaining correct T4 tuning, whereas random networks showed divergent and unreliable tuning.
• T5a neurons became direction selective (at 335° instead of expected 180°), suggesting that incomplete connectivity limits but does not prevent functional emergence.
• The model demonstrated that functional properties such as orientation and direction selectivity can emerge through training on a proxy task without explicit functional constraints, solely from structural connectivity.