[Paper Review] Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency

Robert Geirhos, Kristof Meding+1 more|arXiv (Cornell University)|Jun 30, 2020

Neural dynamics and brain function36 citations

TL;DR

Introduces trial-by-trial error consistency as a method to compare decision strategies between humans and CNNs; finds CNNs are highly consistent with each other, but humans and CNNs show only chance-level consistency, and CORnet-S behaves like a feedforward ResNet-50 rather than human data.

ABSTRACT

A central problem in cognitive science and behavioural neuroscience as well as in machine learning and artificial intelligence research is to ascertain whether two or more decision makers (be they brains or algorithms) use the same strategy. Accuracy alone cannot distinguish between strategies: two systems may achieve similar accuracy with very different strategies. The need to differentiate beyond accuracy is particularly pressing if two systems are near ceiling performance, like Convolutional Neural Networks (CNNs) and humans on visual object recognition. Here we introduce trial-by-trial error consistency, a quantitative analysis for measuring whether two decision making systems systematically make errors on the same inputs. Making consistent errors on a trial-by-trial basis is a necessary condition for similar processing strategies between decision makers. Our analysis is applicable to compare algorithms with algorithms, humans with humans, and algorithms with humans. When applying error consistency to object recognition we obtain three main findings: (1.) Irrespective of architecture, CNNs are remarkably consistent with one another. (2.) The consistency between CNNs and human observers, however, is little above what can be expected by chance alone -- indicating that humans and CNNs are likely implementing very different strategies. (3.) CORnet-S, a recurrent model termed the "current best model of the primate ventral visual stream", fails to capture essential characteristics of human behavioural data and behaves essentially like a standard purely feedforward ResNet-50 in our analysis. Taken together, error consistency analysis suggests that the strategies used by human and machine vision are still very different -- but we envision our general-purpose error consistency analysis to serve as a fruitful tool for quantifying future progress.

Motivation & Objective

• Motivate why accuracy alone is insufficient to infer shared strategies between humans and CNNs.
• Define and operationalize error consistency as a trial-by-trial measure of shared errors.
• Develop statistical bounds and confidence intervals for observed error overlap and Cohen’s kappa in this context.
• Apply the method to compare multiple CNNs with humans and with each other on visual object recognition tasks.
• Assess whether improvements in ImageNet accuracy correspond to more human-like error patterns.

Proposed method

• Define observed error overlap c_obs as the fraction of trials with identical correct/incorrect responses.
• Compute expected overlap c_exp under independent binomial decision makers using p_i and p_j (Eq. 1).
• Describe confidence intervals via simulation of independent observers and analytical bounds (Eq. 2–3; Section S.3 in appendix).
• Quantify error consistency with Cohen’s kappa: kappa = (c_obs − c_exp) / (1 − c_exp) (Eq. 4).
• Provide bounds on kappa as a function of c_exp (Eq. 5–6).
• Use and compare stimuli from cue-conflict, edge, silhouette, and ImageNet experiments; analyze CNNs (ImageNet-trained) and human observers.
• Include CORnet-S as a recurrent model and compare against ResNet-50 baselines and Brain-Score metrics.

Experimental results

Research questions

• RQ1 Do CNNs and humans produce consistent errors on the same stimuli beyond chance?
• RQ2 Does higher ImageNet accuracy imply more human-like error patterns in CNNs?
• RQ3 How does error consistency vary across architectures and between humans and different CNN families?
• RQ4 Does a recurrent model like CORnet-S offer higher error consistency with humans compared to purely feedforward models?
• RQ5 What are the limits and bounds of error consistency given baseline chance expectations?

Key findings

• CNNs are remarkably consistent with one another across architectures.
• Human-CNN error consistency is only slightly above chance, suggesting different strategies between humans and CNNs.
• CORnet-S shows little to no improvement over a ResNet-50 in error consistency with humans and behaves like a feedforward model in this analysis.
• CNN-CNN error consistency is typically high, sometimes exceeding human-human consistency.
• Even CORnet-S and ResNet-50 show high CNN-CNN consistency, indicating recurrent models may not yield distinct behavioral patterns in this measure.
• The analysis highlights the importance of evaluating behavioural metrics beyond neural predictivity or overall accuracy.