[论文解读] A New Compensatory Genetic Algorithm-Based Method for Effective Compressed Multi-function Convolutional Neural Network Model Selection with Multi-Objective Optimization

Luna M. Zhang|arXiv (Cornell University)|Jun 8, 2019

Brain Tumor Detection and Classification参考文献 11被引用 5

一句话总结

本文提出一种新型补偿遗传算法（GA），用于自动选择压缩的多任务卷积神经网络（CMCNNs），以在高分类准确率、低模型尺寸、快速推理和能效方面实现平衡。通过优化基于Inception-V4架构的多样化激活函数与剪枝层，该方法生成的CMCNN在F1得分、内存使用（最低达454KB）和推理速度方面优于非压缩的MCNN，在保持最佳F1得分100%的同时，模型尺寸缩小了44%。

ABSTRACT

In recent years, there have been many popular Convolutional Neural Networks (CNNs), such as Google's Inception-V4, that have performed very well for various image classification problems. These commonly used CNN models usually use the same activation function, such as RELU, for all neurons in the convolutional layers; they are "Single-function CNNs." However, SCNNs may not always be optimal. Thus, a "Multi-function CNN" (MCNN), which uses different activation functions for different neurons, has been shown to outperform a SCNN. Also, CNNs typically have very large architectures that use a lot of memory and need a lot of data in order to be trained well. As a result, they tend to have very high training and prediction times too. An important research problem is how to automatically and efficiently find the best CNN with both high classification performance and compact architecture with high training and prediction speeds, small power usage, and small memory size for any image classification problem. It is very useful to intelligently find an effective, fast, energy-efficient, and memory-efficient "Compressed Multi-function CNN" (CMCNN) from a large number of candidate MCNNs. A new compensatory algorithm using a new genetic algorithm (GA) is created to find the best CMCNN with an ideal compensation between performance and architecture size. The optimal CMCNN has the best performance and the smallest architecture size. Simulations using the CIFAR10 dataset showed that the new compensatory algorithm could find CMCNNs that could outperform non-compressed MCNNs in terms of classification performance (F1-score), speed, power usage, and memory usage. Other effective, fast, power-efficient, and memory-efficient CMCNNs based on popular CNN architectures will be developed for image classification problems in important real-world applications, such as brain informatics and biomedical imaging.

研究动机与目标

• 为解决在移动设备和生物医学设备等资源受限环境中的紧凑高性能CNN的寻找挑战。
• 通过在各层中实现多任务激活函数配置，克服单功能CNN（如仅使用ReLU）的局限性。
• 开发一种自动化的多目标模型选择框架，同时优化准确率、模型尺寸、推理速度和能效。
• 在不牺牲分类性能的前提下，降低深度CNN的计算与内存开销。

提出的方法

• 设计一种新型补偿遗传算法（GA），用于探索具有不同卷积层数量和混合激活函数的CMCNN架构搜索空间。
• 适应性函数结合多个目标——测试F1得分、模型尺寸、推理时间与功耗，采用加权非线性评分以平衡权衡。
• 通过在超参数（包括层数和激活函数分配）上应用选择、交叉与变异操作，演化CMCNN种群。
• 以改进的Inception-V4架构为基础，系统性地减少Inception模块（A、B、C），生成压缩变体。
• 采用多目标优化策略，确保保留Pareto最优解，优先选择高性能且尺寸最小的模型。
• 框架支持用户调节适应性函数中的权重，以优先考虑特定目标，如模型尺寸或准确率。

实验结果

研究问题

• RQ1 多目标遗传算法能否有效优化压缩的多任务CNN，以实现高准确率与低资源消耗？
• RQ2 CMCNN在F1得分、内存使用和推理速度方面与非压缩MCNN相比表现如何？
• RQ3 所提出的补偿GA是否优于随机或非自适应模型选择，在识别最优CMCNN架构方面表现更优？
• RQ4 在各层中使用多样化激活函数时，模型尺寸缩减对分类性能有何影响？
• RQ5 适应性函数是否可调节以优先考虑模型紧凑性，而不会显著降低F1得分？

主要发现

• 由新型GA生成的CMCNN模型（CM_GA^4）模型尺寸为454KB，比非压缩MCNN（815KB）小44%，同时保持或超过其F1得分。
• 在60组测试案例中的58组，新型GA选择的CMCNN（CM_GA^k）在F1得分上优于其非优化对应模型（CM^k），其中CM_GA^8与CM^8表现一致。
• 在45组中的43组，使用新型GA的CMCNN在F1得分更高且内存使用更低，优于非压缩MCNN（M_GA^10），展现出更优的效率。
• 表现最佳的CMCNN（CM_GA^4）具有最短的执行时间和最小的内存占用，表明其具备高能效，适用于边缘设备。
• 新型GA显著提升了模型选择效率，基准测试中所有高性能模型均由补偿GA生成，而非随机搜索。
• 结果表明，模型尺寸可能主导适应性函数，因为第二小的模型始终获得最高F1得分，表明尺寸与准确率之间存在强烈权衡。