[Paper Review] A Fuzzy-Rough based Binary Shuffled Frog Leaping Algorithm for Feature Selection

Javad Rahimipour Anaraki, Saeed Samet+2 more|arXiv (Cornell University)|Jul 31, 2018

Rough Sets and Fuzzy Logic46 references3 citations

TL;DR

This paper proposes a novel binary shuffled frog leaping algorithm (B-SFLA) hybridized with fuzzy-rough dependency degree (FRDD) for feature selection, leveraging the L-FRFS framework to identify minimal reducts. The method significantly outperforms GA, PSO, and GBFS in classification accuracy and feature subset quality while efficiently discovering multiple minimal reducts in a single run due to its subpopulation-based diversity preservation.

ABSTRACT

Feature selection and attribute reduction are crucial problems, and widely used techniques in the field of machine learning, data mining and pattern recognition to overcome the well-known phenomenon of the Curse of Dimensionality, by either selecting a subset of features or removing unrelated ones. This paper presents a new feature selection method that efficiently carries out attribute reduction, thereby selecting the most informative features of a dataset. It consists of two components: 1) a measure for feature subset evaluation, and 2) a search strategy. For the evaluation measure, we have employed the fuzzy-rough dependency degree (FRFDD) in the lower approximation-based fuzzy-rough feature selection (L-FRFS) due to its effectiveness in feature selection. As for the search strategy, a new version of a binary shuffled frog leaping algorithm is proposed (B-SFLA). The new feature selection method is obtained by hybridizing the B-SFLA with the FRDD. Non-parametric statistical tests are conducted to compare the proposed approach with several existing methods over twenty two datasets, including nine high dimensional and large ones, from the UCI repository. The experimental results demonstrate that the B-SFLA approach significantly outperforms other metaheuristic methods in terms of the number of selected features and the classification accuracy.

Motivation & Objective

• To address the curse of dimensionality in machine learning and data mining by improving feature selection efficiency and accuracy.
• To overcome the limitations of greedy and metaheuristic methods that often get trapped in local optima during feature subset search.
• To develop a search strategy that preserves population diversity to explore multiple optimal solutions (minimal reducts) simultaneously.
• To integrate fuzzy-rough set theory with a novel binary metaheuristic for robust, high-performance feature selection.
• To evaluate the proposed method against established algorithms on diverse, high-dimensional datasets from the UCI repository.

Proposed method

• The method employs the lower approximation-based fuzzy-rough feature selection (L-FRFS) framework to compute the fuzzy-rough dependency degree (FRDD) as the evaluation measure for feature subsets.
• A new binary version of the shuffled frog leaping algorithm (B-SFLA) is proposed, where frogs are represented as binary strings indicating feature inclusion or exclusion.
• The B-SFLA divides the population into subpopulations (memeplexes), enabling parallel local search and enhancing diversity to avoid premature convergence.
• Each frog’s position is updated based on its personal best and the global best in the memeplex, using a stochastic local search mechanism to explore the solution space.
• The algorithm iteratively improves feature subsets by maximizing FRDD while minimizing the number of selected features.
• The final solution set includes multiple minimal reducts—feature subsets with the smallest size and highest FRDD—identified in a single run.

Experimental results

Research questions

• RQ1 Can a hybrid B-SFLA with FRDD-based evaluation outperform traditional metaheuristics like GA and PSO in feature selection accuracy and efficiency?
• RQ2 Does the B-SFLA’s subpopulation structure enable it to discover multiple minimal reducts in a single run, unlike GA and PSO which typically return only one?
• RQ3 How does the proposed method compare to L-FRFS and GBFS in terms of classification accuracy and feature subset size across diverse datasets?
• RQ4 To what extent does the B-SFLA reduce computation time while maintaining or improving performance on high-dimensional and large-scale datasets?
• RQ5 Is the B-SFLA particularly effective for multi-modal feature selection problems where multiple optimal solutions exist?

Key findings

• The B-SFLA achieved the highest average ranking (2.1364) in Friedman test for classification accuracy, significantly outperforming GA (3.1818), PSO (3.4091), GBFS (3.5000), and L-FRFS (2.7727).
• Post-hoc analysis confirmed that B-SFLA’s performance was statistically superior to PSO, GA, and GBFS at α=0.05, with p-values of 0.007592, 0.02831, and 0.004231, respectively.
• The B-SFLA was the fastest algorithm in most datasets, with a multi-threaded C++ implementation outperforming Java-based GA and PSO in Weka.
• Unlike GA and PSO, which typically return a single minimal reduct, the B-SFLA consistently returned multiple minimal reducts in a single run due to its subpopulation structure.
• The method demonstrated strong performance on high-dimensional and large datasets, including COIL-2000, CNAE-9, and Madelon, where it achieved high accuracy with fewer features.
• The B-SFLA’s ability to preserve population diversity through memeplex-based search makes it highly suitable for multi-modal feature selection problems with multiple optimal solutions.