[Paper Review] Analyzing Stability of Convolutional Neural Networks in the Frequency Domain

Elnaz Jahani Heravi, Hamed Habibi Aghdam+1 more|arXiv (Cornell University)|Nov 10, 2015

Adversarial Robustness in Machine Learning11 references3 citations

TL;DR

This paper proposes analyzing Convolutional Neural Networks (ConvNets) in the frequency domain using 4D visualization of filter Fourier transforms to explain their instability under imperceptible noise. It shows that filters with broad frequency responses amplify noise, while fine-tuning on noisy data produces filters with more concentrated spectra, significantly improving robustness to additive Gaussian noise.

ABSTRACT

Understanding the internal process of ConvNets is commonly done using visualization techniques. However, these techniques do not usually provide a tool for estimating the stability of a ConvNet against noise. In this paper, we show how to analyze a ConvNet in the frequency domain using a 4-dimensional visualization technique. Using the frequency domain analysis, we show the reason that a ConvNet might be sensitive to a very low magnitude additive noise. Our experiments on a few ConvNets trained on different datasets revealed that convolution kernels of a trained ConvNet usually pass most of the frequencies and they are not able to effectively eliminate the effect of high frequencies. Our next experiments shows that a convolution kernel which has a more concentrated frequency response could be more stable. Finally, we show that fine-tuning a ConvNet using a training set augmented with noisy images can produce more stable ConvNets.

Motivation & Objective

• To understand why ConvNets are sensitive to small, imperceptible noise in input images.
• To develop a frequency-domain analysis method for convolutional filters to diagnose instability causes.
• To evaluate whether training with noisy data improves ConvNet robustness against noise.
• To investigate the impact of loss functions, activation functions, and input size on stability.
• To determine if Gaussian smoothing enhances or degrades performance depending on noise levels.

Proposed method

• Visualize 3D convolutional filters as 4D frequency responses using their 2D Fourier transforms.
• Compute the mean power spectrum of first-layer filters across multiple ConvNets to compare frequency concentration.
• Train ConvNets on original and noise-augmented datasets to assess robustness improvements.
• Apply Gaussian smoothing to noisy images and evaluate performance across varying signal-to-noise ratios.
• Use 4D color-mapped visualizations to compare frequency responses of filters before and after fine-tuning.
• Correlate spectral concentration of filters with test accuracy on noisy inputs to quantify stability.

Experimental results

Research questions

• RQ1 Why are ConvNets sensitive to low-magnitude additive noise despite being imperceptible to humans?
• RQ2 How does the frequency response of convolutional filters relate to their robustness against noise?
• RQ3 Can fine-tuning on noisy training data produce ConvNets with more stable frequency responses?
• RQ4 Does Gaussian smoothing improve or degrade ConvNet performance depending on noise intensity?
• RQ5 Do specific loss functions or activation functions lead to more stable frequency responses in filters?

Key findings

• ConvNets trained on clean data exhibit broad frequency responses, allowing high-frequency noise to propagate through the network.
• After fine-tuning on noisy data, the mean spectrum of first-layer filters becomes more localized, indicating improved noise suppression capability.
• ConvNets trained with noisy data show a significant accuracy boost on noisy test sets, with accuracy dropping from 87.2% to 85.0% on CIFAR10 (hinge) at σ=40, compared to 83.7% for the softplus variant.
• Gaussian smoothing improves performance on high-SNR images only when noise is strong, but harms clean images due to mid-frequency attenuation.
• No single loss function or activation function consistently produces more stable filters; stability depends on filter spectral concentration.
• The frequency-domain analysis reveals that filter response broadening is a key cause of noise sensitivity, not just network depth or architecture.