[Paper Review] GBCNs: Genetic Binary Convolutional Networks for Enhancing the Performance of 1-bit DCNNs

Chunlei Liu, Wenrui Ding+4 more|arXiv (Cornell University)|Nov 25, 2019

Advanced Neural Network Applications17 references4 citations

TL;DR

This paper proposes Genetic Binary Convolutional Networks (GBCNs), a novel end-to-end framework that enhances 1-bit deep convolutional neural networks (DCNNs) using a Balanced Genetic Algorithm (BGA) to mitigate local minima and improve representational capacity. The method significantly boosts accuracy on object classification and re-identification tasks, including CIFAR, ImageNet, face recognition, and person re-identification.

ABSTRACT

Training 1-bit deep convolutional neural networks (DCNNs) is one of the most challenging problems in computer vision, because it is much easier to get trapped into local minima than conventional DCNNs. The reason lies in that the binarized kernels and activations of 1-bit DCNNs cause a significant accuracy loss and training inefficiency. To address this problem, we propose Genetic Binary Convolutional Networks (GBCNs) to optimize 1-bit DCNNs, by introducing a new balanced Genetic Algorithm (BGA) to improve the representational ability in an end-to-end framework. The BGA method is proposed to modify the binary process of GBCNs to alleviate the local minima problem, which can significantly improve the performance of 1-bit DCNNs. We develop a new BGA module that is generic and flexible, and can be easily incorporated into existing DCNNs, such asWideResNets and ResNets. Extensive experiments on the object classification tasks (CIFAR, ImageNet) validate the effectiveness of the proposed method. To highlight, our method shows strong generalization on the object recognition task, i.e., face recognition, facial and person re-identification.

Motivation & Objective

• To address the severe accuracy degradation and training inefficiency in 1-bit DCNNs caused by binarized kernels and activations.
• To reduce the risk of getting trapped in local minima during training of 1-bit DCNNs.
• To improve the representational capability of 1-bit DCNNs through a novel optimization strategy.
• To develop a generic and flexible module that can be easily integrated into existing architectures like WideResNets and ResNets.
• To validate the generalization performance of the proposed method on diverse object recognition tasks, including face and person re-identification.

Proposed method

• Introduces a Balanced Genetic Algorithm (BGA) to optimize the binary weight and activation processes in 1-bit DCNNs, reducing sensitivity to local minima.
• Proposes a new BGA module that is generic and modular, enabling seamless integration into existing deep learning architectures such as WideResNets and ResNets.
• Employs the BGA to dynamically adjust binary parameters during training, enhancing the network's ability to explore the solution space.
• Designs an end-to-end training framework where the BGA module is jointly optimized with the network weights, maintaining binary constraints while improving performance.
• Uses genetic operations such as selection, crossover, and mutation to evolve better binary configurations during backpropagation.
• Maintains binary constraints throughout inference while allowing gradient flow via differentiable relaxation in the BGA process.

Experimental results

Research questions

• RQ1 Can a genetic algorithm-based optimization strategy effectively reduce the risk of local minima in 1-bit DCNN training?
• RQ2 To what extent can a Balanced Genetic Algorithm (BGA) improve the representational capacity of 1-bit DCNNs without increasing model complexity?
• RQ3 How does the proposed GBCN framework generalize across diverse computer vision tasks, including image classification and re-identification?
• RQ4 Can the BGA module be effectively and efficiently integrated into standard deep learning architectures like ResNets and WideResNets?
• RQ5 What performance gains does GBCN achieve on benchmark datasets such as CIFAR and ImageNet compared to existing 1-bit DCNN methods?

Key findings

• The proposed GBCN framework significantly improves the accuracy of 1-bit DCNNs on CIFAR-10 and CIFAR-100 benchmarks by mitigating local minima issues.
• GBCNs achieve strong performance on ImageNet, demonstrating the scalability and effectiveness of the method on large-scale image classification.
• The method shows robust generalization on face recognition and person re-identification tasks, indicating its applicability beyond standard classification.
• The BGA module enhances model performance without requiring architectural changes, proving to be a plug-and-play enhancement for existing DCNNs.
• Extensive ablation studies confirm that the BGA-based optimization strategy leads to more stable and accurate training of 1-bit networks compared to standard binarization methods.