[Paper Review] A Review of 315 Benchmark and Test Functions for Machine Learning Optimization Algorithms and Metaheuristics with Mathematical and Visual Descriptions

M.Z. Naser, ‬‬‬Mohammad Khaled al-Bashiti+8 more|arXiv (Cornell University)|Jun 13, 2024

Machine Learning and Data Classification4 citations

TL;DR

This paper presents a comprehensive review of 315 benchmark and test functions used in machine learning optimization and metaheuristic algorithms, offering detailed mathematical formulations, visual descriptions, and domain-specific suitability. It identifies the 25 most frequently used functions, proposes two new high-dimensional, dynamic, and challenging test functions, and highlights gaps in current benchmarking practices to guide future algorithm development.

ABSTRACT

In the rapidly evolving optimization and metaheuristics domains, the efficacy of algorithms is crucially determined by the benchmark (test) functions. While several functions have been developed and derived over the past decades, little information is available on the mathematical and visual description, range of suitability, and applications of many such functions. To bridge this knowledge gap, this review provides an exhaustive survey of more than 300 benchmark functions used in the evaluation of optimization and metaheuristics algorithms. This review first catalogs benchmark and test functions based on their characteristics, complexity, properties, visuals, and domain implications to offer a wide view that aids in selecting appropriate benchmarks for various algorithmic challenges. This review also lists the 25 most commonly used functions in the open literature and proposes two new, highly dimensional, dynamic and challenging functions that could be used for testing new algorithms. Finally, this review identifies gaps in current benchmarking practices and suggests directions for future research.

Motivation & Objective

• To address the lack of accessible, detailed information on the mathematical and visual properties of widely used benchmark functions in optimization and metaheuristics.
• To systematically categorize 315 benchmark functions based on complexity, characteristics, and application domains for improved algorithm selection.
• To identify the 25 most commonly used benchmark functions in the open literature to establish a standard reference for researchers.
• To propose two novel, high-dimensional, dynamic, and challenging test functions for evaluating next-generation optimization algorithms.
• To identify limitations in current benchmarking practices and suggest future research directions for more robust algorithm evaluation.

Proposed method

• A systematic survey and classification of 315 benchmark and test functions based on their mathematical formulations, visual features, complexity, and domain relevance.
• Categorization of functions by properties such as unimodal, multimodal, separable, non-separable, scalable, and fixed-dimensional characteristics.
• Visual representation of each function using 2D and 3D plots to illustrate topography and search space complexity.
• Analysis of the frequency of use of benchmark functions across the open literature to identify the 25 most prevalent functions.
• Design and formalization of two new benchmark functions with high dimensionality, dynamic shifts, and increased complexity to challenge modern optimization algorithms.
• Identification of gaps in current benchmarking practices, including lack of standardization and insufficient representation of real-world problem characteristics.

Experimental results

Research questions

• RQ1 Which benchmark functions are most frequently used in the literature, and what properties make them dominant in algorithm evaluation?
• RQ2 How do the mathematical and visual characteristics of benchmark functions influence the performance evaluation of optimization algorithms?
• RQ3 What are the limitations of current benchmarking practices in evaluating metaheuristics and machine learning optimization algorithms?
• RQ4 How can new benchmark functions be designed to better reflect the complexity of real-world optimization problems?
• RQ5 What criteria should guide the selection of appropriate benchmark functions for specific algorithmic challenges?

Key findings

• The 25 most commonly used benchmark functions in the literature were identified, providing a standardized reference for algorithm evaluation.
• Many benchmark functions lack detailed mathematical and visual descriptions, creating a significant knowledge gap in algorithm development and comparison.
• The review reveals that existing benchmarks often fail to represent high-dimensional, dynamic, and complex real-world problems adequately.
• Two new benchmark functions were proposed—designed to be highly dimensional, dynamic, and challenging—to improve the rigor of algorithm testing.
• The study highlights the urgent need for standardized, diverse, and realistic benchmark suites to ensure fair and meaningful evaluation of optimization algorithms.
• Visual and mathematical descriptions of 315 benchmark functions were compiled, offering a comprehensive resource for researchers in metaheuristics and machine learning optimization.