[Paper Review] Enhancing Machine Learning Model Performance with Hyper Parameter Optimization: A Comparative Study

Caner Erden, Halil İbrahim Demir+1 more|arXiv (Cornell University)|Feb 14, 2023

Machine Learning and Data Classification6 citations

TL;DR

This study compares classical hyperparameter optimization (HPO) methods—grid search, random search, Bayesian optimization—and meta-heuristic algorithms—genetic algorithm (GA), particle swarm optimization (PSO)—for improving machine learning model performance. Using a Python-based implementation on a classification dataset, PSO outperformed all other methods in test accuracy (92.62%), F1-score (0.92), and training accuracy (99.33%), while maintaining reasonable computational time, demonstrating its superiority in balancing performance and efficiency.

ABSTRACT

One of the most critical issues in machine learning is the selection of appropriate hyper parameters for training models. Machine learning models may be able to reach the best training performance and may increase the ability to generalize using hyper parameter optimization (HPO) techniques. HPO is a popular topic that artificial intelligence studies have focused on recently and has attracted increasing interest. While the traditional methods developed for HPO include exhaustive search, grid search, random search, and Bayesian optimization; meta-heuristic algorithms are also employed as more advanced methods. Meta-heuristic algorithms search for the solution space where the solutions converge to the best combination to solve a specific problem. These algorithms test various scenarios and evaluate the results to select the best-performing combinations. In this study, classical methods, such as grid, random search and Bayesian optimization, and population-based algorithms, such as genetic algorithms and particle swarm optimization, are discussed in terms of the HPO. The use of related search algorithms is explained together with Python programming codes developed on packages such as Scikit-learn, Sklearn Genetic, and Optuna. The performance of the search algorithms is compared on a sample data set, and according to the results, the particle swarm optimization algorithm has outperformed the other algorithms.

Motivation & Objective

• To evaluate and compare the performance of classical and meta-heuristic hyperparameter optimization (HPO) techniques in improving machine learning model accuracy.
• To assess the trade-off between optimization performance and computational efficiency (CPU time) across different HPO algorithms.
• To identify the most effective HPO method for enhancing model generalization and predictive performance on a real-world classification dataset.
• To provide practical implementation code using Python libraries such as Scikit-learn, Optuna, and sklearn-genetic for reproducible HPO workflows.

Proposed method

• The study employs five HPO techniques: grid search, random search, Bayesian optimization, genetic algorithm (GA), and particle swarm optimization (PSO) on a fixed dataset.
• Hyperparameter search spaces are defined for key model parameters: n_estimators (50–400), max_features (4–10), max_depth (4–10), and criterion (‘gini’, ‘entropy’).
• Performance is evaluated using precision, recall, and F1-score on both training and test sets, with 15 independent runs to ensure robustness.
• The PSO algorithm uses a population size of 10, 40 generations, and cognitive/social parameters φ₁ = φ₂ = 0.5.
• The GA uses a population size of 10, 40 generations, crossover probability of 0.9, and mutation probability of 0.05.
• All algorithms are implemented in Python using Jupyter notebooks with libraries including Scikit-learn, Optuna, skopt, and sklearn-genetic, executed on a laptop with 12GB RAM and i7 processor.

Experimental results

Research questions

• RQ1 Which hyperparameter optimization method—classical or meta-heuristic—yields the highest test accuracy on a given classification dataset?
• RQ2 How do the computational times of different HPO algorithms compare, and which offers the best balance between speed and performance?
• RQ3 Does particle swarm optimization (PSO) achieve superior generalization and F1-score compared to Bayesian optimization, genetic algorithms, and traditional search methods?
• RQ4 To what extent do interconnected hyperparameters (e.g., learning rate and optimizer choice) affect the optimization outcome in HPO?

Key findings

• Particle swarm optimization (PSO) achieved the highest test accuracy of 92.62%, significantly outperforming Bayesian optimization (86.11%), GA (86.09%), random search (84.25%), and grid search (82.97%).
• PSO achieved a perfect training accuracy of 99.33%, indicating strong learning capacity and minimal underfitting.
• PSO recorded the best F1-score of 0.94 for the 'DOWN' class and 0.92 for the 'UP' class, demonstrating superior class-wise generalization.
• PSO had a reasonable CPU time of 1,286.4 seconds, outperforming GA (6,639.02 seconds) and Bayesian optimization (5,850.78 seconds), while being faster than random search (2,567.56 seconds).
• Grid search had the lowest CPU time (714.89 seconds), but also the worst test accuracy, confirming that speed does not guarantee performance.
• The 'DOWN' class consistently showed higher precision and recall than the 'UP' class across all algorithms, with PSO achieving the highest values in all metrics.