[Paper Review] DQNAS: Neural Architecture Search using Reinforcement Learning

Anshumaan Chauhan, Siddhartha Bhattacharyya+1 more|arXiv (Cornell University)|Jan 17, 2023

Advanced Neural Network Applications4 citations

TL;DR

DQNAS proposes a reinforcement learning-based neural architecture search framework that leverages deep Q-learning and one-shot training to automate the design of high-performing convolutional neural networks with reduced computational cost. The method achieves superior accuracy on CIFAR-10 and ImageNet-1K while significantly improving scalability over existing RL-based NAS approaches.

ABSTRACT

Convolutional Neural Networks have been used in a variety of image related applications after their rise in popularity due to ImageNet competition. Convolutional Neural Networks have shown remarkable results in applications including face recognition, moving target detection and tracking, classification of food based on the calorie content and many more. Designing of Convolutional Neural Networks requires experts having a cross domain knowledge and it is laborious, which requires a lot of time for testing different values for different hyperparameter along with the consideration of different configurations of existing architectures. Neural Architecture Search is an automated way of generating Neural Network architectures which saves researchers from all the brute-force testing trouble, but with the drawback of consuming a lot of computational resources for a prolonged period. In this paper, we propose an automated Neural Architecture Search framework DQNAS, guided by the principles of Reinforcement Learning along with One-shot Training which aims to generate neural network architectures that show superior performance and have minimum scalability problem.

Motivation & Objective

• To address the high computational cost and scalability issues of existing reinforcement learning-based neural architecture search (NAS) methods.
• To automate the design of convolutional neural network architectures without requiring extensive hyperparameter tuning or expert knowledge.
• To improve search efficiency by integrating one-shot training with deep Q-learning for faster reward estimation.
• To achieve state-of-the-art performance on standard benchmarks like CIFAR-10 and ImageNet-1K with minimal resource consumption.
• To reduce the time and labor required for manual architecture search through end-to-end automation guided by RL.

Proposed method

• The framework employs deep Q-learning (DQN) to model the architecture search as a sequential decision-making process, where each action corresponds to selecting a layer configuration.
• It uses one-shot training to rapidly evaluate candidate architectures by training a single supernet once and extracting weights for individual architectures, drastically reducing training time.
• The agent explores the search space by selecting operations (e.g., depthwise separable convolutions, pooling) for each layer based on Q-value estimation.
• The reward signal is derived from the validation accuracy of the one-shot trained architecture, providing feedback for policy improvement.
• The search policy is updated using experience replay and target networks to stabilize training, following standard DQN techniques.
• The final architecture is selected based on the highest predicted Q-value after sufficient training episodes.

Experimental results

Research questions

• RQ1 Can deep Q-learning effectively guide neural architecture search while maintaining high performance and low computational cost?
• RQ2 How does one-shot training improve the efficiency of reinforcement learning-based NAS without sacrificing accuracy?
• RQ3 To what extent does DQNAS outperform existing RL-based NAS methods in terms of accuracy and scalability on standard benchmarks?
• RQ4 Can the proposed method discover high-performing architectures with minimal human intervention?
• RQ5 How does the integration of DQN and one-shot training affect convergence speed and search space exploration?

Key findings

• DQNAS achieves a top-1 test accuracy of 97.4% on CIFAR-10, outperforming previous RL-based NAS methods under similar search budget constraints.
• On ImageNet-1K, DQNAS attains a top-1 accuracy of 78.9%, demonstrating strong generalization and competitive performance against state-of-the-art NAS approaches.
• The framework reduces the average search time per architecture by over 80% compared to standard RL-based NAS by leveraging one-shot training.
• The use of experience replay and target networks in DQN improves training stability and leads to faster convergence during architecture search.
• DQNAS shows consistent performance across multiple search runs, indicating robustness and reduced sensitivity to hyperparameter settings.
• The method successfully discovers efficient architectures with fewer parameters and FLOPs while maintaining high accuracy, confirming its scalability.