[Paper Review] Delving Deeper into Anti-aliasing in ConvNets

Xueyan Zou, Fanyi Xiao+2 more|arXiv (Cornell University)|Aug 21, 2020

Digital Media Forensic DetectionComputer Science32 references21 citations

TL;DR

This paper proposes a content-aware, adaptive low-pass filtering layer in ConvNets that predicts spatially and channel-wise varying filter weights to suppress aliasing during downsampling. By learning location- and channel-specific blurring, the method improves accuracy and shift consistency across ImageNet, COCO, and Cityscapes benchmarks, outperforming fixed-filter baselines with minimal computational overhead.

ABSTRACT

Aliasing refers to the phenomenon that high frequency signals degenerate into completely different ones after sampling. It arises as a problem in the context of deep learning as downsampling layers are widely adopted in deep architectures to reduce parameters and computation. The standard solution is to apply a low-pass filter (e.g., Gaussian blur) before downsampling. However, it can be suboptimal to apply the same filter across the entire content, as the frequency of feature maps can vary across both spatial locations and feature channels. To tackle this, we propose an adaptive content-aware low-pass filtering layer, which predicts separate filter weights for each spatial location and channel group of the input feature maps. We investigate the effectiveness and generalization of the proposed method across multiple tasks including ImageNet classification, COCO instance segmentation, and Cityscapes semantic segmentation. Qualitative and quantitative results demonstrate that our approach effectively adapts to the different feature frequencies to avoid aliasing while preserving useful information for recognition. Code is available at https://maureenzou.github.io/ddac/.

Motivation & Objective

• To address aliasing in ConvNets caused by downsampling, which leads to shift variance and performance degradation.
• To overcome the limitations of fixed low-pass filters (e.g., Gaussian blur) that uniformly blur all regions and channels, causing information loss.
• To develop an adaptive filtering mechanism that tailors blurring intensity to local content frequency and channel-specific feature patterns.
• To improve robustness to input shifts by introducing new evaluation metrics for shift consistency in segmentation tasks.
• To demonstrate generalization across image classification, semantic segmentation, instance segmentation, and domain generalization.

Proposed method

• Introduces a learnable, adaptive low-pass filtering layer that predicts filter weights per spatial location and channel group.
• Uses a lightweight MLP head to predict filter coefficients from input feature maps, enabling spatial and channel-wise adaptation.
• Applies the predicted filters as dynamic 2D convolutions with learnable kernels that vary across spatial positions and channels.
• Employs a differentiable filtering operation that integrates seamlessly into standard ConvNet architectures without architectural changes.
• Supports both spatial and channel-wise adaptation by grouping feature channels and predicting filters per group.
• Uses a differentiable approximation of the filtering process to enable end-to-end training with backpropagation.

Experimental results

Research questions

• RQ1 Can adaptive, content-aware filtering reduce aliasing in ConvNets more effectively than fixed low-pass filters?
• RQ2 How does spatial and channel-wise adaptive filtering affect shift consistency in semantic and instance segmentation?
• RQ3 Does the proposed method improve performance on image classification and segmentation benchmarks without increasing model capacity?
• RQ4 To what extent does the adaptive filtering preserve high-frequency features like edges while suppressing noise?
• RQ5 How does the number of channel groups influence performance and generalization in the adaptive filtering mechanism?

Key findings

• The proposed method achieves 68.0% top-1 accuracy on ImageNet with ResNet-18, outperforming ResNet (66.5%) and Gaussian blur (66.7%) baselines.
• The method improves shift consistency to 80.9%, significantly higher than ResNet (79.1%) and Gaussian blur (79.8%), demonstrating robustness to input shifts.
• Increasing the number of channel groups improves accuracy up to a saturation point at 8 groups, indicating diminishing returns beyond that.
• The method achieves better performance than simply increasing network capacity, gaining 0.3% accuracy with only 4.5% more parameters in ResNet-101.
• Qualitative results show superior edge preservation in Cityscapes semantic segmentation, with clearer road and pole contours compared to baselines.
• Visualizations confirm that the model learns to grow edges by applying stronger blurring along contours, preserving structural details after downsampling.