[Paper Review] Fully Learnable Group Convolution for Acceleration of Deep Neural Networks

Xijun Wang, Meina Kan+2 more|arXiv (Cornell University)|Mar 31, 2019

Advanced Neural Network Applications44 references4 citations

TL;DR

This paper proposes Fully Learnable Group Convolution (FLGC), a novel group convolution module that automatically learns optimal channel and filter groupings during end-to-end training, enabling significant acceleration without sacrificing accuracy. FLGC achieves up to 5× speedup over ResNet50 on CPU by dynamically learning group structures and supporting arbitrary group counts, outperforming both standard and learnable group convolution methods in accuracy and efficiency.

ABSTRACT

Benefitted from its great success on many tasks, deep learning is increasingly used on low-computational-cost devices, e.g. smartphone, embedded devices, etc. To reduce the high computational and memory cost, in this work, we propose a fully learnable group convolution module (FLGC for short) which is quite efficient and can be embedded into any deep neural networks for acceleration. Specifically, our proposed method automatically learns the group structure in the training stage in a fully end-to-end manner, leading to a better structure than the existing pre-defined, two-steps, or iterative strategies. Moreover, our method can be further combined with depthwise separable convolution, resulting in 5 times acceleration than the vanilla Resnet50 on single CPU. An additional advantage is that in our FLGC the number of groups can be set as any value, but not necessarily 2^k as in most existing methods, meaning better tradeoff between accuracy and speed. As evaluated in our experiments, our method achieves better performance than existing learnable group convolution and standard group convolution when using the same number of groups.

Motivation & Objective

• To address the high computational and memory cost of deep neural networks on resource-constrained devices such as smartphones and embedded systems.
• To overcome limitations of existing group convolution methods that rely on pre-defined, two-step, or iterative strategies for group structure selection.
• To develop a flexible, end-to-end trainable group convolution module that can be seamlessly integrated into any deep neural network architecture.
• To enable arbitrary group counts (not limited to powers of two) for better accuracy-speed trade-offs.
• To achieve significant inference speedup while maintaining or improving model accuracy through dynamic group structure learning.

Proposed method

• Proposes a fully learnable group convolution (FLGC) module that jointly learns the grouping of input channels and convolutional filters during backpropagation in an end-to-end manner.
• Introduces a differentiable grouping mechanism that allows the network to automatically determine optimal group assignments for input channels and filters based on overall network loss.
• Supports arbitrary group numbers (not restricted to 2^k), enabling more flexible and effective trade-offs between model accuracy and inference speed.
• Combines FLGC with depthwise separable convolution to further accelerate inference, achieving 5× speedup over vanilla ResNet50 on single CPU.
• Uses a standard residual block architecture with FLGC replacing 1×1 convolutions, and trains the model with a cosine learning rate schedule for 120 epochs.
• Employs the same hyperparameters as baseline models (e.g., CondenseNet) for fair comparison, ensuring that performance gains stem from the FLGC module itself.

Experimental results

Research questions

• RQ1 Can a group convolution module be trained end-to-end to automatically learn optimal channel and filter groupings without pre-defined or iterative strategies?
• RQ2 Does allowing arbitrary group counts (not limited to powers of two) lead to better accuracy-speed trade-offs than conventional group convolution methods?
• RQ3 Can FLGC be effectively combined with depthwise separable convolutions to achieve greater acceleration on CPU without accuracy degradation?
• RQ4 How does FLGC compare to state-of-the-art learnable group convolution (e.g., LGC in CondenseNet) and standard group convolution in terms of accuracy and computational efficiency?
• RQ5 Can FLGC be flexibly embedded into various network architectures (e.g., ResNet, MobileNetV2) to maintain performance while accelerating inference?

Key findings

• FLGC achieves 6.77% top-1 error on ImageNet with 44M FLOPs, outperforming both standard group convolution (29.0% error) and LGC (26.2% error) under the same architecture and group count.
• On CIFAR-10, ResNet50-FLGC2 achieves 6.77% error with 44M FLOPs, outperforming ResNet56-pruned [25] (6.94% error at 90M FLOPs) and ResNet56-pruned [14] (8.2% error at 62M FLOPs).
• MobileNetV2-FLGC with 8 groups achieves 6.91% top-1 error at 76M FLOPs, outperforming MobileNetV2-SGC (7.51% error) and approaching the performance of larger models like NASNet-A.
• FLGC enables up to 5× inference speedup over vanilla ResNet50 on single CPU when combined with depthwise separable convolutions.
• The method supports arbitrary group counts (e.g., G=2 to G=8), and performance remains stable or improves with increasing groups, demonstrating robustness and flexibility.
• FLGC achieves better accuracy than both standard group convolution and LGC from CondenseNet while maintaining similar or lower FLOP counts, proving its superiority in end-to-end learning of group structures.