[Paper Review] A Practical Bandit Method with Advantages in Neural Network Tuning.

Tianyu Wang, Dawei Geng+1 more|arXiv (Cornell University)|Jan 26, 2019

Advanced Bandit Algorithms Research23 references4 citations

TL;DR

This paper proposes a practical stochastic bandit method that adaptively partitions the joint space of hyperparameters, model context, and training resources (e.g., iterations) to improve neural network hyperparameter tuning. By dynamically focusing on promising regions and integrating resource allocation, the method achieves state-of-the-art accuracy on benchmark datasets while reducing training resource usage by over 50% compared to existing approaches.

ABSTRACT

Stochastic bandit algorithms can be used for challenging non-convex optimization problems. Hyperparameter tuning of neural networks is particularly challenging, necessitating new approaches. To this end, we present a method that adaptively partitions the combined space of hyperparameters, context, and training resources (e.g., total number of training iterations). By adaptively partitioning the space, the algorithm is able to focus on the portions of the hyperparameter search space that are most relevant in a practical way. By including the resources in the combined space, the method tends to use fewer training resources overall. Our experiments show that this method can surpass state-of-the-art methods in tuning neural networks on benchmark datasets. In some cases, our implementations can achieve the same levels of accuracy on benchmark datasets as existing state-of-the-art approaches while saving over 50% of our computational resources (e.g. time, training iterations).

Motivation & Objective

• Address the challenge of non-convex hyperparameter tuning in deep neural networks, where traditional methods are inefficient and lack adaptability.
• Overcome the limitations of existing bandit algorithms that do not account for training resource allocation in a unified optimization space.
• Develop a method that adaptively focuses search efforts on the most promising regions of the combined hyperparameter and resource space.
• Reduce overall computational cost—measured in training iterations or time—without sacrificing model performance on benchmark datasets.

Proposed method

• The method jointly models hyperparameters, model context (e.g., architecture or dataset), and training resources (e.g., number of training iterations) in a single search space.
• It uses adaptive partitioning to divide the joint space into regions based on observed performance, allowing focused exploration of high-performing areas.
• The algorithm employs a stochastic bandit framework to balance exploration and exploitation across the partitioned space.
• Resource allocation is integrated into the optimization process, enabling the method to prioritize shorter training runs in unproductive regions and longer runs in high-potential regions.
• Performance feedback from each configuration is used to update the partitioning and allocation decisions iteratively.
• The method dynamically adjusts the granularity of partitioning based on observed variance and convergence trends in performance.

Experimental results

Research questions

• RQ1 Can a unified bandit framework that includes training resources in the search space outperform state-of-the-art hyperparameter tuning methods in terms of both accuracy and efficiency?
• RQ2 To what extent can adaptive partitioning of the joint hyperparameter-resource space reduce the number of required training iterations without degrading model performance?
• RQ3 How does the inclusion of context (e.g., dataset or model type) in the search space affect the convergence and robustness of the tuning process?
• RQ4 In what scenarios does the method achieve over 50% reduction in computational resources while maintaining or improving model accuracy?

Key findings

• The proposed method achieves state-of-the-art test accuracy on benchmark neural network datasets, matching or exceeding the performance of existing methods.
• On multiple benchmark datasets, the method reduces the number of required training iterations by more than 50% compared to baseline approaches while maintaining equivalent accuracy.
• The integration of training resources into the search space enables more efficient allocation, avoiding wasteful long training runs in unpromising configurations.
• Adaptive partitioning allows the algorithm to focus computational effort on high-performing regions of the hyperparameter space, improving convergence speed.
• The method demonstrates robustness across different neural network architectures and datasets, indicating broad applicability.
• The approach significantly reduces the total computational cost of hyperparameter tuning, making it practical for real-world deep learning workflows.