[Paper Review] ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases

Pierre Stock, Moustapha Cissé|arXiv (Cornell University)|Nov 30, 2017

Adversarial Robustness in Machine Learning36 references16 citations

TL;DR

This paper demonstrates that state-of-the-art ConvNets on ImageNet are significantly more accurate and robust than traditionally measured, as human evaluation reduces top-1 error from 22.69% to 9.47%. It introduces model criticism with adversarial examples to uncover hidden racial and semantic biases, showing explanations mitigate misclassification and reveal model flaws from an end-user perspective.

ABSTRACT

ConvNets and Imagenet have driven the recent success of deep learning for image classification. However, the marked slowdown in performance improvement combined with the lack of robustness of neural networks to adversarial examples and their tendency to exhibit undesirable biases question the reliability of these methods. This work investigates these questions from the perspective of the end-user by using human subject studies and explanations. The contribution of this study is threefold. We first experimentally demonstrate that the accuracy and robustness of ConvNets measured on Imagenet are vastly underestimated. Next, we show that explanations can mitigate the impact of misclassified adversarial examples from the perspective of the end-user. We finally introduce a novel tool for uncovering the undesirable biases learned by a model. These contributions also show that explanations are a valuable tool both for improving our understanding of ConvNets' predictions and for designing more reliable models.

Motivation & Objective

• To reassess the true performance of SOTA ConvNets on ImageNet beyond standard accuracy metrics.
• To investigate whether human judgment can reveal that model errors are less severe than traditionally assumed.
• To evaluate the role of explanations in improving end-user trust and robustness against adversarial examples.
• To detect and uncover undesirable biases—particularly racial and semantic biases—learned by models on ImageNet.
• To develop and validate a novel model criticism framework using adversarial examples for bias detection.

Proposed method

• Conducted human subject studies where five annotators judged whether misclassified ImageNet images were correctly classified by the model, using a consensus threshold of four out of five agreeing.
• Applied feature-based explanations via saliency maps and example-based explanations using prototypes and criticisms (model criticism) to interpret predictions.
• Used adversarial examples as a tool to generate criticisms that highlight counterintuitive or biased decision patterns in the model.
• Employed multiple explanation baselines: adversarial selection, MMD-critic, and probability-based selection for model criticism.
• Collected and analyzed image pairs from the internet with identical content except for skin color to test for racial bias in predictions.
• Trained a ResNet-101 on ImageNet and used its internal representations to extract prototypes and criticisms for bias analysis.

Experimental results

Research questions

• RQ1 To what extent is the true accuracy of SOTA ConvNets on ImageNet underestimated when evaluated by human consensus?
• RQ2 How do explanations affect end-user perception and trust in model predictions, especially for adversarial examples?
• RQ3 Can adversarial examples be effectively used to generate model criticisms that reveal hidden biases?
• RQ4 What kind of biases—particularly racial or semantic—do models learn on ImageNet, and how can they be detected?
• RQ5 How does model criticism compare to other explanation methods in uncovering model flaws and biases?

Key findings

• Human consensus reduced the top-1 error of a ResNet-101 on ImageNet from 22.69% to 9.47%, indicating that model errors are vastly underestimated.
• The top-5 error decreased from 6.44% to 1.94% under human evaluation, suggesting ImageNet is nearly solved when measured by human consensus.
• Providing explanations significantly mitigated the impact of adversarial examples on end-user perception, reducing misclassification confidence.
• Model criticism with adversarial examples effectively uncovered racial bias: 78% of prototypes for 'basketball' contained at least one Black person, while only 44% contained White people.
• For criticisms, 90% contained White people versus only 20% with Black people, indicating a strong bias toward Black players as prototypical for the class.
• Image pairs differing only in skin color showed that models classified images with Black players as 'basketball' 100% of the time, while similar images with lighter-skinned players were misclassified as other classes.