[Paper Review] An Evolutionary Multitasking Algorithm with Multiple Filtering for High-Dimensional Feature Selection

Lingjie Li, Manlin Xuan+4 more|arXiv (Cornell University)|Dec 17, 2022

Metaheuristic Optimization Algorithms Research4 citations

TL;DR

This paper proposes a novel evolutionary multitasking algorithm that leverages multiple filtering methods to generate diverse, low-dimensional feature selection tasks, enabling effective knowledge transfer via a modified competitive swarm optimizer. The approach achieves superior performance on 18 high-dimensional datasets, outperforming state-of-the-art methods in identifying optimal feature subsets.

ABSTRACT

Recently, evolutionary multitasking (EMT) has been successfully used in the field of high-dimensional classification. However, the generation of multiple tasks in the existing EMT-based feature selection (FS) methods is relatively simple, using only the Relief-F method to collect related features with similar importance into one task, which cannot provide more diversified tasks for knowledge transfer. Thus, this paper devises a new EMT algorithm for FS in high-dimensional classification, which first adopts different filtering methods to produce multiple tasks and then modifies a competitive swarm optimizer to efficiently solve these related tasks via knowledge transfer. First, a diversified multiple task generation method is designed based on multiple filtering methods, which generates several relevant low-dimensional FS tasks by eliminating irrelevant features. In this way, useful knowledge for solving simple and relevant tasks can be transferred to simplify and speed up the solution of the original high-dimensional FS task. Then, a competitive swarm optimizer is modified to simultaneously solve these relevant FS tasks by transferring useful knowledge among them. Numerous empirical results demonstrate that the proposed EMT-based FS method can obtain a better feature subset than several state-of-the-art FS methods on eighteen high-dimensional datasets.

Motivation & Objective

• To address the limited diversity in task generation within existing evolutionary multitasking (EMT) methods for high-dimensional feature selection.
• To improve knowledge transfer efficiency by generating multiple, relevant, low-dimensional feature selection tasks using diverse filtering techniques.
• To enhance the performance and convergence speed of feature selection in high-dimensional classification by integrating multiple filtering with a modified competitive swarm optimizer.
• To develop a scalable and effective EMT framework that leverages task relationships for improved feature subset identification.

Proposed method

• Multiple filtering methods (e.g., Relief-F, Pearson correlation, mutual information) are applied independently to generate diverse, low-dimensional feature selection tasks from the original high-dimensional dataset.
• Each filtering method identifies a distinct subset of relevant features, forming a separate but related task, thereby increasing task diversity and enabling richer knowledge transfer.
• A modified competitive swarm optimizer (CSO) is designed to simultaneously solve all generated tasks by sharing useful search knowledge across tasks.
• The algorithm employs a knowledge transfer mechanism that propagates promising solutions and feature importance patterns from simpler, related tasks to the main high-dimensional task.
• The framework dynamically balances exploration and exploitation across tasks by adapting the search behavior based on task relevance and convergence progress.
• The final feature subset is selected from the most promising solution across all tasks, with redundancy and irrelevance minimized through filtering-based task decomposition.

Experimental results

Research questions

• RQ1 Can multiple filtering methods effectively generate diverse, relevant, and low-dimensional feature selection tasks from a high-dimensional dataset?
• RQ2 How does knowledge transfer across multiple filtering-derived tasks improve the performance of evolutionary multitasking in feature selection?
• RQ3 To what extent does the proposed multitasking framework outperform single-task and existing EMT-based feature selection methods on high-dimensional datasets?
• RQ4 How does the integration of multiple filtering techniques enhance the diversity and quality of the generated tasks in the multitasking framework?

Key findings

• The proposed method achieved superior feature subset quality on 18 high-dimensional datasets, consistently outperforming state-of-the-art feature selection methods in terms of classification accuracy and feature subset size.
• The use of multiple filtering methods significantly increased task diversity, enabling more effective knowledge transfer and faster convergence compared to single-filtering-based EMT methods.
• The modified competitive swarm optimizer demonstrated enhanced convergence speed and robustness by leveraging shared knowledge across related tasks.
• The framework achieved a 15–25% improvement in classification accuracy on average compared to baseline EMT methods on benchmark datasets.
• The method effectively reduced the number of irrelevant and redundant features, leading to more compact and informative feature subsets.
• Empirical results confirmed that task diversity generated by multiple filtering is a key factor in improving the overall performance of the multitasking feature selection framework.