[论文解读] Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search

Xiangxiang Chu, Bo Zhang+2 more|arXiv (Cornell University)|Jan 4, 2019

Advanced Vision and Imaging参考文献 39被引用 38

一句话总结

MoreMNAS 将多目标进化搜索与增强控制相结合，设计移动神经网络，使用更少 FLOPS 即可实现具有竞争力的 SR 模型。

ABSTRACT

Fabricating neural models for a wide range of mobile devices demands for a specific design of networks due to highly constrained resources. Both evolution algorithms (EA) and reinforced learning methods (RL) have been dedicated to solve neural architecture search problems. However, these combinations usually concentrate on a single objective such as the error rate of image classification. They also fail to harness the very benefits from both sides. In this paper, we present a new multi-objective oriented algorithm called MoreMNAS (Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search) by leveraging good virtues from both EA and RL. In particular, we incorporate a variant of multi-objective genetic algorithm NSGA-II, in which the search space is composed of various cells so that crossovers and mutations can be performed at the cell level. Moreover, reinforced control is mixed with a natural mutating process to regulate arbitrary mutation, maintaining a delicate balance between exploration and exploitation. Therefore, not only does our method prevent the searched models from degrading during the evolution process, but it also makes better use of learned knowledge. Our experiments conducted in Super-resolution domain (SR) deliver rivalling models compared to some state-of-the-art methods with fewer FLOPS.

研究动机与目标

• 在高度受限的移动设备上推进神经架构搜索，除了准确性之外还包含多目标。
• 提出一种混合算法，利用进化算子与强化学习提升搜索效率和模型质量。
• 实现跨单元的变异和交叉，以在保持已学习知识的同时探索多样化的架构。
• 在探索与利用之间取得平衡，防止进化过程中的性能下降。
• 在资源高效的超分辨率领域中展示有效性。

提出的方法

• 使用一种变体的 NSGA-II，其中搜索空间由能够进行单元级交叉和变异的单元组成。
• 结合带有自然变异过程的强化控制来调节变异。
• 将进化搜索与强化信号结合，引导选择和变异朝向多目标结果。
• 将该方法应用于移动神经网络架构，重点是在保持质量的同时降低 FLOPS。
• 在超分辨率任务上进行评估，以在效率和性能方面与最先进方法进行比较。

实验结果

研究问题

• RQ1 相较于现有方法，MoreMNAS 能否在移动网络上以更低的 FLOPS 实现具有竞争力的准确性或感知质量？
• RQ2 将强化控制与多目标进化搜索结合是否可以改善收敛性并保留有用的架构？
• RQ3 单元级遗传算子如何影响移动 NAS 的架构空间探索？
• RQ4 所提出方法对知识重用以及防止进化过程中的退化有何影响？

主要发现

• MoreMNAS 在超分辨率领域以更少的 FLOPS 提供媲美现有最先进方法的模型。
• 该方法利用单元级交叉/变异与强化控制来平衡探索与利用。
• 该方法有助于防止进化过程中被搜索模型的退化，并更好地利用已学习的知识。