[논문 리뷰] Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search

Xiangxiang Chu, Bo Zhang외 2명|arXiv (Cornell University)|2019. 01. 04.

Advanced Vision and Imaging참고 문헌 39인용 수 38

한 줄 요약

MoreMNAS는 다목적 진화 탐색과 강화된 제어를 결합하여 모바일 신경망을 설계하고, 더 적은 FLOPS로 경쟁력 있는 SR 모델을 달성한다.

ABSTRACT

Fabricating neural models for a wide range of mobile devices demands for a specific design of networks due to highly constrained resources. Both evolution algorithms (EA) and reinforced learning methods (RL) have been dedicated to solve neural architecture search problems. However, these combinations usually concentrate on a single objective such as the error rate of image classification. They also fail to harness the very benefits from both sides. In this paper, we present a new multi-objective oriented algorithm called MoreMNAS (Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search) by leveraging good virtues from both EA and RL. In particular, we incorporate a variant of multi-objective genetic algorithm NSGA-II, in which the search space is composed of various cells so that crossovers and mutations can be performed at the cell level. Moreover, reinforced control is mixed with a natural mutating process to regulate arbitrary mutation, maintaining a delicate balance between exploration and exploitation. Therefore, not only does our method prevent the searched models from degrading during the evolution process, but it also makes better use of learned knowledge. Our experiments conducted in Super-resolution domain (SR) deliver rivalling models compared to some state-of-the-art methods with fewer FLOPS.

연구 동기 및 목표

• highly constrained mobile devices with multiple objectives beyond accuracy.
• Propose a hybrid algorithm that leverages evolutionary operators and reinforced learning to improve search efficiency and model quality.
• Enable cross-cell level mutations and crossovers to explore diverse architectures while preserving learned knowledge.
• Balance exploration and exploitation to prevent performance degradation during evolution.
• Demonstrate effectiveness in a super-resolution domain with resource-efficient models.

제안 방법

• Use a variant of NSGA-II where the search space is composed of cells enabling cell-level crossover and mutation.
• Incorporate reinforced control with a natural mutating process to regulate mutations.
• Combine evolutionary search with reinforcement signals to guide selection and mutation toward multi-objective outcomes.
• Apply the method to mobile neural architectures with a focus on reducing FLOPS while maintaining quality.
• Evaluate on a super-resolution task to compare against state-of-the-art methods in terms of efficiency and performance.

실험 결과

연구 질문

• RQ1 Can MoreMNAS achieve competitive accuracy or perceptual quality with lower FLOPS for mobile networks compared to existing methods?
• RQ2 Does integrating reinforced control with multi-objective evolutionary search improve convergence and preservation of useful architectures?
• RQ3 How do cell-level genetic operators affect exploration of architecture space in mobile NAS?
• RQ4 What is the impact of the proposed approach on knowledge reuse and preventing degradation during evolution?

주요 결과

• MoreMNAS delivers rivalling models to state-of-the-art methods in the super-resolution domain with fewer FLOPS.
• The approach leverages cell-level crossover/mutation and reinforced control to balance exploration and exploitation.
• The method helps prevent degradation of searched models during evolution and makes better use of learned knowledge.