[Paper Review] EasyConvPooling: Random Pooling with Easy Convolution for Accelerating Training and Testing

Jian‐Zhong Sheng, Chuanbo Chen+2 more|arXiv (Cornell University)|Jun 5, 2018

Machine Learning and Data Classification24 references3 citations

TL;DR

This paper proposes EasyConvPooling (ECP), a novel technique that accelerates CNN training and inference by replacing conventional convolution and pooling with Easy Convolution and Random Pooling. ECP reduces computation by 75% by randomly selecting one pixel from every 2×2 region for convolution, achieving 1.45x speedup in training and 1.64x in testing with minimal accuracy drop.

ABSTRACT

Convolution operations dominate the overall execution time of Convolutional Neural Networks (CNNs). This paper proposes an easy yet efficient technique for both Convolutional Neural Network training and testing. The conventional convolution and pooling operations are replaced by Easy Convolution and Random Pooling (ECP). In ECP, we randomly select one pixel out of four and only conduct convolution operations of the selected pixel. As a result, only a quarter of the conventional convolution computations are needed. Experiments demonstrate that the proposed EasyConvPooling can achieve 1.45x speedup on training time and 1.64x on testing time. What's more, a speedup of 5.09x on pure Easy Convolution operations is obtained compared to conventional convolution operations.

Motivation & Objective

• To address the high computational cost of convolution operations in deep CNNs, which dominate training and inference time.
• To develop a universal, lightweight acceleration technique that works for both training and inference without architectural redesign.
• To minimize accuracy degradation while achieving significant speedup through randomized, sparse computation.
• To enable efficient deployment on resource-constrained platforms by reducing FLOPs and memory access in convolution layers.

Proposed method

• Introduces Random Pooling, where one pixel is randomly selected from each 2×2 spatial region to represent the entire block.
• Uses the index of the selected 'lucky' pixel to guide the subsequent Easy Convolution operation in the next layer.
• Applies Easy Convolution by computing only the convolution for the selected pixel, reducing computation to 25% of the original.
• Maintains feature map shape by aligning the convolution mode with the selected pixel's position (Mode K) to ensure spatial consistency.
• Employs a fixed index mapping from the pooling layer to the convolution layer to preserve spatial structure across layers.
• Supports both training and inference acceleration by replacing standard convolution and pooling with the ECP pipeline.

Experimental results

Research questions

• RQ1 Can random selection of a single pixel per 2×2 region preserve network performance while drastically reducing convolution computation?
• RQ2 How does the choice of random mode (Mode K) affect convergence and accuracy in the ECP framework?
• RQ3 What is the performance gain of ECP compared to conventional Max/ Average Pooling in terms of speedup and accuracy?
• RQ4 Can ECP be applied universally across different network depths and architectures with consistent speedup and minimal accuracy loss?
• RQ5 How does the combination of Random Pooling and Easy Convolution compare to using either technique alone in terms of efficiency and convergence?

Key findings

• ECP achieves a 1.45x speedup in training time and a 1.64x speedup in inference time compared to conventional CNNs with minimal accuracy degradation.
• The proposed method reduces convolution computation by 75%, resulting in a 5.09x speedup on pure Easy Convolution operations versus standard convolution.
• Random Pooling alone reduces training and testing time, but the combination with Easy Convolution yields significantly greater performance gains.
• ECP outperforms both Max Pooling and Average Pooling in terms of speedup, especially in deeper networks, with only a minor drop in accuracy compared to Max Pooling.
• The choice of Mode K (the selected pixel index) has little impact on convergence or final accuracy, indicating robustness to random selection.
• The method maintains good convergence behavior across one- and two-convolution-layer networks, demonstrating stability and generalization across architectures.