[Paper Review] Particle Swarm Optimization: Development of a General-Purpose Optimizer

Mauro S. Innocente, Johann Sienz|arXiv (Cornell University)|Jan 25, 2021

Metaheuristic Optimization Algorithms Research3 citations

TL;DR

This paper presents a general-purpose particle swarm optimization (PSO) framework that enhances the standard PSO algorithm through systematic parameter tuning, reliable stopping criteria, and effective constraint-handling techniques. The approach achieves high efficiency and ease of implementation, requiring minimal code and few parameters, while maintaining strong performance across diverse optimization problems, including constrained and non-smooth functions.

ABSTRACT

Traditional methods present a very restrictive range of applications, mainly limited by the features of the function to be optimized and of the constraint functions. In contrast, evolutionary algorithms present almost no restriction to the features of these functions, although the most appropriate constraint-handling technique is still an open question. The particle swarm optimization (PSO) method is sometimes viewed as another evolutionary algorithm because of their many similarities, despite not being inspired by the same metaphor. Namely, they evolve a population of individuals taking into consideration previous experiences and using stochastic operators to introduce new responses. The advantages of evolutionary algorithms with respect to traditional methods have been greatly discussed in the literature for decades. While all such advantages are valid when comparing the PSO paradigm to traditional methods, its main advantages with respect to evolutionary algorithms consist of its noticeably lower computational cost and easier implementation. In fact, the plain version can be programmed in a few lines of code, involving no operator design and few parameters to be tuned. This paper deals with three important aspects of the method: the influence of the parameters' tuning on the behaviour of the system; the design of stopping criteria so that the reliability of the solution found can be somehow estimated and computational cost can be saved; and the development of appropriate techniques to handle constraints, given that the original method is designed for unconstrained optimization problems.

Motivation & Objective

• To develop a robust, general-purpose PSO optimizer suitable for a wide range of optimization problems.
• To investigate the influence of PSO parameters on algorithm behavior and convergence characteristics.
• To design reliable stopping criteria that balance solution quality and computational cost.
• To propose effective constraint-handling techniques for extending PSO to constrained optimization problems.
• To demonstrate the method's efficiency, simplicity, and reliability through empirical evaluation.

Proposed method

• The paper employs a standard PSO algorithm with velocity and position updates governed by cognitive and social components.
• Parameter tuning is systematically evaluated to identify optimal values for acceleration coefficients and inertia weight.
• Stopping criteria are designed based on convergence metrics, such as stagnation in global best position and velocity thresholding.
• A penalty-based constraint-handling technique is introduced, integrating feasibility and objective function improvement.
• The algorithm is implemented in a minimal codebase, emphasizing simplicity and low computational overhead.
• The framework is validated on benchmark problems, including constrained and non-differentiable functions.

Experimental results

Research questions

• RQ1 How do the key PSO parameters (e.g., acceleration coefficients and inertia weight) influence convergence and solution quality?
• RQ2 What stopping criteria ensure reliable solution estimation while minimizing unnecessary computation?
• RQ3 How can constraints be effectively handled in PSO without compromising convergence or performance?
• RQ4 To what extent does the proposed PSO variant maintain low computational cost and ease of implementation?
• RQ5 Can the method reliably solve a broad class of optimization problems, including non-smooth and constrained functions?

Key findings

• Optimal parameter settings significantly improve convergence speed and solution accuracy, with inertia weight values around 0.7 and acceleration coefficients near 1.5–2.0.
• The proposed stopping criteria effectively reduce computational cost by detecting premature convergence and stagnation.
• The penalty-based constraint-handling method successfully guides the swarm toward feasible solutions without requiring complex repair mechanisms.
• The PSO variant maintains a minimal implementation (few lines of code) with only a few tunable parameters, enhancing usability.
• The method demonstrates robust performance across diverse test problems, including non-convex and constrained functions.
• The approach achieves competitive results compared to other evolutionary algorithms, with notably lower computational overhead.