[Paper Review] Blurs Behave Like Ensembles: Spatial Smoothings to Improve Accuracy, Uncertainty, and Robustness

Namuk Park, Songkuk Kim|arXiv (Cornell University)|May 26, 2021

Medical Imaging and Analysis4 citations

TL;DR

This paper proposes spatial smoothing—a technique that applies blur layers to convolutional neural network feature maps to emulate ensemble behavior. By stabilizing feature maps and smoothing the loss landscape, spatial smoothing significantly improves accuracy, uncertainty estimation, and robustness in both Bayesian and deterministic networks, enabling high performance with only a few ensemble members.

ABSTRACT

Neural network ensembles, such as Bayesian neural networks (BNNs), have shown success in the areas of uncertainty estimation and robustness. However, a crucial challenge prohibits their use in practice. BNNs require a large number of predictions to produce reliable results, leading to a significant increase in computational cost. To alleviate this issue, we propose spatial smoothing, a method that spatially ensembles neighboring feature map points of convolutional neural networks. By simply adding a few blur layers to the models, we empirically show that spatial smoothing improves accuracy, uncertainty estimation, and robustness of BNNs across a whole range of ensemble sizes. In particular, BNNs incorporating spatial smoothing achieve high predictive performance merely with a handful of ensembles. Moreover, this method also can be applied to canonical deterministic neural networks to improve the performances. A number of evidences suggest that the improvements can be attributed to the stabilized feature maps and the smoothing of the loss landscape. In addition, we provide a fundamental explanation for prior works - namely, global average pooling, pre-activation, and ReLU6 - by addressing them as special cases of spatial smoothing. These not only enhance accuracy, but also improve uncertainty estimation and robustness by making the loss landscape smoother in the same manner as spatial smoothing. The code is available at https://github.com/xxxnell/spatial-smoothing.

Motivation & Objective

• Address the high computational cost of Bayesian neural networks (BNNs), which require large ensemble sizes (e.g., 50) for reliable performance.
• Improve predictive performance, uncertainty estimation, and robustness in BNNs without increasing inference cost.
• Enable efficient deployment of BNNs in real-world applications by reducing the required number of ensemble members.
• Extend the benefits of ensemble-like behavior to deterministic neural networks through spatial smoothing.
• Provide a unified understanding of existing architectural components (e.g., global average pooling, ReLU6, pre-activation) as special cases of spatial smoothing.

Proposed method

• Introduce spatial smoothing as a method that applies spatial blurring to feature maps, effectively creating a local ensemble of neighboring feature points.
• Integrate blur layers into the network architecture to smooth feature representations and stabilize predictions.
• Combine spatial smoothing with probabilistic inference via Prob, which models Bernoulli distributions over feature map activations without assuming Gaussianity.
• Use the combination of Prob and Blur to approximate ensemble averaging without explicit sampling, reducing computational cost.
• Demonstrate that spatial smoothing can be applied to both Bayesian and deterministic networks, improving performance across multiple benchmarks.
• Reframe existing components—such as pre-activation, ReLU6, and global average pooling—as special cases of spatial smoothing, showing their inherent ensemble-like behavior.

Experimental results

Research questions

• RQ1 Can spatial smoothing reduce the number of ensemble members required for high predictive performance in Bayesian neural networks?
• RQ2 How does spatial smoothing improve uncertainty estimation and robustness in deep neural networks?
• RQ3 What is the role of feature map stabilization and loss landscape smoothing in the performance gains observed with spatial smoothing?
• RQ4 Can spatial smoothing be applied to deterministic neural networks to achieve similar improvements in accuracy and uncertainty?
• RQ5 Are established architectural components like pre-activation and global average pooling fundamentally equivalent to spatial smoothing?

Key findings

• Spatial smoothing enables Bayesian neural networks to achieve high predictive performance with as few as 4–8 ensemble members, drastically reducing computational cost compared to standard BNNs requiring 50+ members.
• The method improves uncertainty calibration and robustness on corrupted data (e.g., CIFAR-10-P), with relative confidence metrics showing reduced overconfidence on incorrect predictions.
• Spatial smoothing improves accuracy, uncertainty, and consistency on semantic segmentation tasks (e.g., CamVid dataset) for both deterministic and Bayesian U-Net models.
• Blurring alone degrades performance due to information loss, but when combined with Prob (which models Bernoulli distributions over activations), it significantly improves performance.
• Pre-activation, ReLU6, and global average pooling are shown to be special cases of spatial smoothing, explaining their success through the lens of local ensemble behavior.
• The combination of spatial smoothing and temporal smoothing (from prior work) yields the best predictive and computational performance, especially in data stream settings.