[Paper Review] Enhancing sensor resolution improves CNN accuracy given the same number of parameters or FLOPS

Ali Borji|arXiv (Cornell University)|Mar 9, 2021

Advanced Neural Network Applications2 references4 citations

TL;DR

This paper demonstrates that increasing input image resolution improves CNN accuracy without increasing model parameters or FLOPS, by systematically modifying network layers to maintain computational budget. The key contribution is proving that higher resolution alone enhances performance, independent of model capacity, through four resolution-aware architecture adaptation methods.

ABSTRACT

High image resolution is critical to obtain a good performance in many computer vision applications. Computational complexity of CNNs, however, grows significantly with the increase in input image size. Here, we show that it is almost always possible to modify a network such that it achieves higher accuracy at a higher input resolution while having the same number of parameters or/and FLOPS. The idea is similar to the EfficientNet paper but instead of optimizing network width, depth and resolution simultaneously, here we focus only on input resolution. This makes the search space much smaller which is more suitable for low computational budget regimes. More importantly, by controlling for the number of model parameters (and hence model capacity), we show that the additional benefit in accuracy is indeed due to the higher input resolution. Preliminary empirical investigation over MNIST, Fashion MNIST, and CIFAR10 datasets demonstrates the efficiency of the proposed approach.

Motivation & Objective

• To investigate whether improved CNN accuracy at higher resolution stems from resolution itself or increased model capacity.
• To develop methods that maintain the same number of parameters or FLOPS while increasing input resolution.
• To enable practical accuracy gains under fixed computational budgets by optimizing for resolution alone.
• To provide a scalable, low-budget alternative to full NAS or EfficientNet-style multi-dimensional search.

Proposed method

• Proposes four methods (I–IV) to reconfigure existing CNN layers to accept higher-resolution inputs while preserving parameter count or FLOPS.
• Method I uses strided convolutions and adaptive average pooling to maintain resolution and FLOPS.
• Method II adjusts kernel size and stride to preserve feature map size and FLOPS after convolutional layers.
• Method III modifies pooling and convolutional layers to maintain resolution and FLOPS across multiple layers.
• Method IV uses dilated convolutions and kernel size adjustments to increase receptive field and resolution without increasing FLOPS.
• Empirically evaluates methods on MNIST, Fashion-MNIST, and CIFAR-10 using grid search over resolution, kernel size, stride, and padding.

Experimental results

Research questions

• RQ1 Does higher input resolution improve CNN accuracy independently of model capacity, when parameters and FLOPS are held constant?
• RQ2 Can we systematically modify a CNN to accept higher-resolution inputs while preserving the same number of parameters or FLOPS?
• RQ3 Which architectural adaptation techniques are most effective for resolution enhancement under fixed computational budgets?
• RQ4 Is the performance gain from higher resolution due to finer-grained feature representation rather than increased model capacity?

Key findings

• Higher input resolution consistently improves CNN accuracy even when the number of parameters and FLOPS are held constant, proving resolution itself enhances performance.
• Approach IV, which uses dilated convolutions and kernel size adjustments, achieves the best accuracy gains, especially at lower resolutions where more room exists for resolution increase.
• Approach I, relying on average pooling, performs worse as resolution increases due to information loss from collapsing feature maps.
• Approaches II and III show marginal improvements, indicating that careful layer reconfiguration is essential for effective resolution scaling.
• The results confirm that fine-grained spatial information from higher resolution is exploitable by CNNs, even with identical model capacity.
• Across all datasets, it is often possible to find a higher-resolution model with the same FLOPS or parameters that outperforms the original.