[Paper Review] On the Blindspots of Convolutional Networks

Elad Hoffer, Shai Fine+1 more|arXiv (Cornell University)|Feb 14, 2018

Adversarial Robustness in Machine Learning24 references5 citations

TL;DR

This paper reveals that convolutional neural networks (ConvNets) have structural blindspots—despite state-of-the-art performance, they fail to capture certain global, low-dimensional, or highly encoded signals that simpler models easily detect. By injecting truth-revealing signals (e.g., one-pixel patterns, length encoding) that are imperceptible to ConvNets but informative for classification, the authors show that traditional models like logistic regression and random forests outperform ConvNets when these signals are present, exposing limitations in feature selection and invariance mechanisms.

ABSTRACT

Deep convolutional network has been the state-of-the-art approach for a wide variety of tasks over the last few years. Its successes have, in many cases, turned it into the default model in quite a few domains. In this work, we will demonstrate that convolutional networks have limitations that may, in some cases, hinder it from learning properties of the data, which are easily recognizable by traditional, less demanding, models. To this end, we present a series of competitive analysis studies on image recognition and text analysis tasks, for which convolutional networks are known to provide state-of-the-art results. In our studies, we inject a truth-revealing signal, indiscernible for the network, thus hitting time and again the network's blind spots. The signal does not impair the network's existing performances, but it does provide an opportunity for a significant performance boost by models that can capture it. The various forms of the carefully designed signals shed a light on the strengths and weaknesses of convolutional network, which may provide insights for both theoreticians that study the power of deep architectures, and for practitioners that consider applying convolutional networks to the task at hand.

Motivation & Objective

• To identify and demonstrate structural limitations in convolutional neural networks (ConvNets) that prevent them from capturing certain data signals despite high performance on standard benchmarks.
• To investigate why ConvNets fail to utilize highly informative, non-local, or low-dimensional features that are easily captured by simpler models.
• To compare ConvNet performance against traditional models (e.g., logistic regression, random forests) when injected with truth-revealing signals that are imperceptible to ConvNets but boost performance in simpler models.
• To provide insights into the invariance and pooling mechanisms of ConvNets that may suppress important global or structured data properties.
• To guide practitioners in assessing when ConvNets may be inappropriate due to these blindspots, especially after data pre-processing or encoding.

Proposed method

• Injecting truth-revealing signals—such as one-pixel patterns, noise encoding, length encoding, and mnemonic encodings—into image and text datasets without altering original labels or network architecture.
• Using two distinct ConvNet architectures: one for image data (spatial convolution on pixel grids) and one for text data (temporal convolution on one-hot encoded sequences), both trained without modifications.
• Designing signals that are globally informative but structurally invisible to ConvNets, such as orthogonal noise in images or length-based encoding in text, which are unlikely in natural data but plausible in real-world artifacts.
• Comparing performance of ConvNets against traditional models (e.g., logistic regression, random forests, decision trees) on the same data, with and without injected signals, to isolate the impact of the hidden information.
• Analyzing the interplay between convolutional feature construction and pooling-based feature selection, particularly how invariance to shift and scale may suppress signals encoded in spatial or temporal position.
• Conducting ablation studies where only the truth-revealing signal remains, to test whether ConvNets can learn it when obscuring features are removed.

Experimental results

Research questions

• RQ1 Why do ConvNets fail to utilize certain global or low-dimensional signals that are easily captured by simpler models, even when those signals are highly predictive?
• RQ2 To what extent do structural invariances in ConvNets—such as translation and scale invariance—harm performance when the signal depends on spatial or temporal position?
• RQ3 Can simple, non-deep models outperform deep ConvNets when critical information is encoded in a way invisible to convolutional and pooling operations?
• RQ4 How do data pre-processing methods (e.g., one-hot encoding, spectrogram transformation) introduce or amplify ConvNet blindspots by distorting or hiding informative features?
• RQ5 What role does the interaction between feature construction (convolution) and selection (pooling) play in the network’s inability to detect certain informative patterns?

Key findings

• ConvNets failed to utilize one-pixel or pattern-based signals that were trivial for logistic regression and random forests to detect, even when the signals were perfectly predictive and imperceptible to the network.
• In text classification, length encoding signals (e.g., sentence length as a feature) were completely ignored by ConvNets but enabled simple models to surpass ConvNet performance.
• When all other features were removed and only the truth-revealing signal remained, ConvNets achieved perfect accuracy, indicating they can learn the signal when not obstructed by competing features.
• Shallow networks outperformed deep ConvNets in cases where the informative signal was low-dimensional or global, suggesting that deeper architectures may interfere with signal detection due to invariance mechanisms.
• The same blindspots emerged across different domains (images and text), indicating a fundamental structural limitation rather than a data-specific artifact.
• Traditional models like logistic regression and random forests achieved significantly better performance than ConvNets when the injected signals were present, despite being inferior in standard benchmarks.