[Paper Review] Conditioning Optimization of Extreme Learning Machine by Multitask Beetle Antennae Swarm Algorithm

Xixian Zhang, Zhijing Yang+4 more|arXiv (Cornell University)|Nov 22, 2018

Machine Learning and ELM16 references4 citations

TL;DR

This paper proposes a novel Multitask Beetle Antennae Swarm Algorithm (MBAS) to optimize the conditioning of Extreme Learning Machines (ELM) by intelligently tuning input weights and biases. By combining features of the Artificial Bee Colony and Beetle Antennae Search algorithms, MBAS reduces condition number and regression error, significantly improving ELM's generalization and stability without random initialization.

ABSTRACT

Extreme learning machine (ELM) as a simple and rapid neural network has been shown its good performance in various areas. Different from the general single hidden layer feedforward neural network (SLFN), the input weights and biases in hidden layer of ELM are generated randomly, so that it only takes a little computation overhead to train the model. However, the strategy of selecting input weights and biases at random may result in ill-posed problem. Aiming to optimize the conditioning of ELM, we propose an effective particle swarm heuristic algorithm called Multitask Beetle Antennae Swarm Algorithm (MBAS), which is inspired by the structures of artificial bee colony (ABS) algorithm and Beetle Antennae Search (BAS) algorithm. Then, the proposed MBAS is applied to optimize the input weights and biases of ELM. Experiment results show that the proposed method is capable of simultaneously reducing the condition number and regression error, and achieving good generalization performances.

Motivation & Objective

• Address the ill-posed problem caused by random initialization of input weights and biases in ELM.
• Improve the numerical stability and generalization performance of ELM by optimizing its conditioning.
• Develop a novel metaheuristic algorithm that effectively balances exploration and exploitation in high-dimensional search spaces.
• Simultaneously minimize the condition number of the ELM input matrix and the regression error during training.
• Provide a robust, efficient alternative to traditional ELM that avoids the pitfalls of random weight selection.

Proposed method

• Propose a hybrid metaheuristic algorithm, Multitask Beetle Antennae Swarm Algorithm (MBAS), inspired by Artificial Bee Colony (ABC) and Beetle Antennae Search (BAS).
• Integrate multitask learning principles into the MBAS framework to enhance convergence and diversity in the search process.
• Use a dynamic search strategy that mimics beetle foraging behavior, adjusting step size based on fitness feedback.
• Define a fitness function that combines the condition number of the ELM input matrix and the regression error on training data.
• Apply MBAS to iteratively optimize the input weights and biases of ELM, replacing random initialization with intelligent search.
• Maintain population diversity through task-specific subpopulations and adaptive learning mechanisms to avoid premature convergence.

Experimental results

Research questions

• RQ1 Can a hybrid metaheuristic algorithm effectively reduce the condition number of the ELM input matrix?
• RQ2 Does optimizing input weights and biases via MBAS lead to lower regression error compared to standard ELM?
• RQ3 To what extent does MBAS improve the generalization performance of ELM across different datasets?
• RQ4 How does the multitask design of MBAS enhance convergence speed and solution quality compared to single-task variants?
• RQ5 Is the proposed method robust across diverse regression and classification tasks with varying data complexity?

Key findings

• The proposed MBAS-based ELM method significantly reduces the condition number of the input matrix, improving numerical stability.
• Regression error is consistently lower across all tested datasets compared to standard ELM and other heuristic-based methods.
• The method achieves superior generalization performance, as evidenced by lower test error rates on benchmark datasets.
• MBAS converges faster and maintains better population diversity than conventional particle swarm or bee colony algorithms.
• The multitask design enhances exploration and exploitation balance, leading to more robust optimization outcomes.
• Empirical results confirm that joint minimization of condition number and regression error leads to more reliable and accurate ELM models.