[Paper Review] Computation-Performance Optimization of Convolutional Neural Networks with Redundant Kernel Removal

Chih‐Ting Liu, Yiheng Wu+2 more|arXiv (Cornell University)|May 30, 2017

Advanced Image Processing Techniques17 references3 citations

TL;DR

This paper proposes two computation-performance optimization (CPO) methods—Layer-Wise Polishment (LWP) and Gradient Optimization (GO)—to reduce redundant convolutional kernels in deep CNNs for efficient deployment on embedded systems. By pruning kernels based on sparsity and optimizing removal ratios under PSNR or computation constraints, the methods achieve up to 50% model compression with only 0.25 dB PSNR drop on super resolution tasks.

ABSTRACT

Deep Convolutional Neural Networks (CNNs) are widely employed in modern computer vision algorithms, where the input image is convolved iteratively by many kernels to extract the knowledge behind it. However, with the depth of convolutional layers getting deeper and deeper in recent years, the enormous computational complexity makes it difficult to be deployed on embedded systems with limited hardware resources. In this paper, we propose two computation-performance optimization methods to reduce the redundant convolution kernels of a CNN with performance and architecture constraints, and apply it to a network for super resolution (SR). Using PSNR drop compared to the original network as the performance criterion, our method can get the optimal PSNR under a certain computation budget constraint. On the other hand, our method is also capable of minimizing the computation required under a given PSNR drop.

Motivation & Objective

• Address the challenge of deploying deep CNNs on resource-constrained embedded systems due to high computational and memory demands.
• Reduce model complexity by removing redundant convolutional kernels without significant performance degradation.
• Enable flexible trade-offs between computational cost and model accuracy through adaptive kernel pruning.
• Develop a method applicable across diverse network architectures and hardware platforms, avoiding architecture-specific tuning.
• Achieve optimal performance under strict computation or accuracy constraints using sparsity-based pruning and regression-based optimization.

Proposed method

• Define kernel redundancy using layer-wise sparsity: kernels with weights below the mean absolute value in a layer are considered redundant.
• Apply Layer-Wise Polishment (LWP) by sorting kernels by sparsity and removing the most redundant ones up to a specified computation budget to minimize PSNR drop.
• Use a regression model trained on random pruning ratios to predict PSNR drop, enabling gradient-based optimization of removal ratios under a target PSNR constraint.
• Optimize the reducing factor vector $\mathbf{r}$ via gradient descent using mean-square error loss between predicted and target PSNR drop.
• Fine-tune the optimized $\mathbf{r}$ by incorporating architectural insights, such as prioritizing kernel retention in deeper layers.
• Validate the approach on a super resolution CNN, using PSNR drop as the primary performance metric.

Experimental results

Research questions

• RQ1 Can redundant convolutional kernels be effectively identified and removed to reduce computational cost in CNNs without significant performance loss?
• RQ2 How can kernel pruning be optimized across layers to minimize PSNR degradation under a fixed computation budget?
• RQ3 Can a learned regression model accurately predict PSNR drop for arbitrary pruning ratios, enabling automated optimization?
• RQ4 How does the distribution of pruning ratios across different network layers affect overall model performance?
• RQ5 To what extent can the proposed methods balance model size, inference speed, and accuracy on embedded platforms?

Key findings

• LWP achieves a 50% reduction in model size (from 20 to 10 layers) with only a 0.24 dB PSNR drop on Set5 ×2 and 0.26 dB on Set14 ×2.
• By adjusting pruning ratios across network segments—increasing pruning in early layers and reducing it in deeper layers—the PSNR drop was minimized to 0.24 dB.
• GO successfully optimized the reducing factor vector to achieve a PSNR drop of 0.29 dB on Set5 ×2 and 0.26 dB on Set14 ×2, closely matching the target.
• The regression model trained on random pruning ratios enabled gradient-based optimization, reducing the PSNR drop to 0.24 dB/0.25 dB after fine-tuning.
• The final optimized pruning policy retained 44–60 kernels per layer, with only one layer reduced to a single kernel, demonstrating effective sparsity-aware compression.
• The method is flexible and architecture-agnostic, enabling deployment across diverse embedded systems without low-level hardware tuning.