[Paper Review] Unsupervised Feature Selection Based on Space Filling Concept

Mohamed Laib, Mikhaïl Kanevski|arXiv (Cornell University)|Jun 27, 2017

Metaheuristic Optimization Algorithms Research13 references3 citations

TL;DR

This paper proposes UfsCov, a novel unsupervised feature selection algorithm based on the space-filling coverage measure to minimize redundancy in high-dimensional data. By iteratively selecting features that maximize spatial coverage using a sequential forward selection approach, the method achieves optimal feature subsets with minimal parameters, demonstrating strong performance on simulated, environmental, and hyperspectral datasets, particularly improving classification accuracy with reduced feature sets.

ABSTRACT

The paper deals with the adaptation of a new measure for the unsupervised feature selection problems. The proposed measure is based on space filling concept and is called the coverage measure. This measure was used for judging the quality of an experimental space filling design. In the present work, the coverage measure is adapted for selecting the smallest informative subset of variables by reducing redundancy in data. This paper proposes a simple analogy to apply this measure. It is implemented in a filter algorithm for unsupervised feature selection problems. The proposed filter algorithm is robust with high dimensional data and can be implemented without extra parameters. Further, it is tested with simulated data and real world case studies including environmental data and hyperspectral image. Finally, the results are evaluated by using random forest algorithm.

Motivation & Objective

• To address the curse of dimensionality in high-dimensional data by reducing redundancy without requiring labeled output.
• To adapt the space-filling coverage measure—originally used in experimental design—into a filter-based unsupervised feature selection framework.
• To develop a robust, parameter-free algorithm that selects the smallest informative feature subset by maximizing data space coverage.
• To evaluate the method’s performance on simulated data, environmental datasets (e.g., permafrost), and hyperspectral images (e.g., Indian Pines).
• To validate the selected features using a consistent random forest evaluation protocol across multiple datasets.

Proposed method

• The coverage measure quantifies how uniformly data points fill the feature space by comparing their distribution to a regular grid; lower values indicate better space filling.
• The UfsCov algorithm uses sequential forward selection (SFS) to iteratively add features that most improve the coverage measure.
• At each step, the algorithm computes the coverage value for all candidate feature subsets and selects the one with the minimum coverage, indicating maximal uniformity and information content.
• The method is implemented as a filter-based approach, independent of any learning model, making it computationally efficient and scalable.
• The algorithm is tested with noise injection and data shuffling to assess robustness.
• Results are evaluated using random forest classification, with accuracy and Kappa coefficient reported across multiple runs.

Experimental results

Research questions

• RQ1 Can the space-filling coverage measure be effectively adapted for unsupervised feature selection in high-dimensional data?
• RQ2 How does the UfsCov algorithm compare to baseline methods in reducing redundancy while preserving informative features?
• RQ3 Does the selected feature subset improve downstream classification performance, particularly with complex real-world data like hyperspectral images and environmental monitoring datasets?
• RQ4 How robust is the UfsCov algorithm under data perturbations such as noise and shuffling?
• RQ5 Can the coverage measure serve as a reliable, parameter-free criterion for identifying the optimal feature subset in unsupervised settings?

Key findings

• On the Permafrost dataset, UfsCov selected 17 features out of 26, achieving a classification accuracy of 0.9339 (Kappa = 0.867), slightly outperforming the full feature set (accuracy 0.9301, Kappa 0.848).
• For the Indian Pines hyperspectral image, the algorithm identified 69 out of 200 bands as optimal, reducing dimensionality while maintaining strong performance.
• The random forest evaluation showed that the selected features from UfsCov consistently matched or slightly improved classification accuracy compared to using all features.
• The coverage measure reached its minimum at 17 features in the Permafrost case and 69 features in the Indian Pines case, indicating optimal space filling at these subset sizes.
• The algorithm demonstrated robustness under noise and data shuffling, maintaining consistent performance across multiple runs.
• The UfsCov algorithm is available in R via the SFtools library on CRAN, enabling broad reproducibility and integration into data science pipelines.