[Paper Review] Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs

Jonas Kubilius, Martin Schrimpf+12 more|Lirias (KU Leuven)|Sep 13, 2019

Domain Adaptation and Few-Shot Learning108 citations

TL;DR

CORnet-S is a compact four-area, recurrent ANN that achieves top Brain-Score while maintaining strong ImageNet performance, highlighting recurrence as key to brain-likeness.

ABSTRACT

Deep convolutional artificial neural networks (ANNs) are the leading class of candidate models of the mechanisms of visual processing in the primate ventral stream. While initially inspired by brain anatomy, over the past years, these ANNs have evolved from a simple eight-layer architecture in AlexNet to extremely deep and branching architectures, demonstrating increasingly better object categorization performance, yet bringing into question how brain-like they still are. In particular, typical deep models from the machine learning community are often hard to map onto the brain's anatomy due to their vast number of layers and missing biologically-important connections, such as recurrence. Here we demonstrate that better anatomical alignment to the brain and high performance on machine learning as well as neuroscience measures do not have to be in contradiction. We developed CORnet-S, a shallow ANN with four anatomically mapped areas and recurrent connectivity, guided by Brain-Score, a new large-scale composite of neural and behavioral benchmarks for quantifying the functional fidelity of models of the primate ventral visual stream. Despite being significantly shallower than most models, CORnet-S is the top model on Brain-Score and outperforms similarly compact models on ImageNet. Moreover, our extensive analyses of CORnet-S circuitry variants reveal that recurrence is the main predictive factor of both Brain-Score and ImageNet top-1 performance. Finally, we report that the temporal evolution of the CORnet-S "IT" neural population resembles the actual monkey IT population dynamics. Taken together, these results establish CORnet-S, a compact, recurrent ANN, as the current best model of the primate ventral visual stream.

Motivation & Objective

• Motivate compact, brain-aligned architectures for ventral visual stream modeling.
• Develop CORnet-S with four anatomically mapped areas and recurrence to improve brain-likeness.
• Quantitatively compare CORnet-S to other models using Brain-Score and behavioral benchmarks.

Proposed method

• Design CORnet-S with four areas mapped to V1, V2, V4, and IT.
• Introduce recurrence within areas, with specific repetition counts per area.
• Train on ImageNet using SGD with momentum; batch size 256; 43 epochs.
• Use Brain-Score as a composite benchmark combining neural and behavioral data.
• Evaluate neural predictivity, behavioral predictivity, and object solution times (OST).
• Compare with a wide range of architectures including AlexNet, VGG, ResNet, Inception, NASNet, and BaseNets.

Experimental results

Research questions

• RQ1 How does a shallow, anatomically aligned recurrent ANN perform on brain-like benchmarks compared to deeper models?
• RQ2 What architectural elements (recurrence, bottleneck width, skip connections) most influence Brain-Score and ImageNet performance?
• RQ3 Can CORnet-S capture temporal IT dynamics and object solution times observed in primate IT?
• RQ4 How well do brain-score measures generalize across new neural/behavioral datasets and transfer tasks?

Key findings

• CORnet-S achieves the highest Brain-Score among tested models and outperforms similarly compact models on ImageNet.
• Recurrence is the main predictive factor for both Brain-Score and ImageNet top-1 performance.
• CORnet-S captures temporal IT dynamics, correlating with monkey IT object solution times, unlike purely feed-forward models.
• CORnet-S maintains strong ImageNet performance (73.1% top-1) with a depth of 15, the shallowest among top performers.
• Brain-Score generalizes across new subjects, new image sets, and CIFAR-100 transfer tests, with CORnet-S leading among shallow models.