[Paper Review] A New Heuristic for Feature Selection by Consistent Biclustering

Antonio Mucherino, Sonia Cafieri|arXiv (Cornell University)|Mar 17, 2010

Rough Sets and Fuzzy Logic12 references3 citations

TL;DR

This paper proposes a novel bilevel optimization heuristic for feature selection via consistent biclustering, reformulating the NP-hard 0–1 linear fractional problem to improve solution quality. The method outperforms prior heuristics by selecting more features while maintaining low classification error on leukemia and synthetic datasets, achieving up to 7,081 selected features with only 2 misclassifications on validation data.

ABSTRACT

Given a set of data, biclustering aims at finding simultaneous partitions in biclusters of its samples and of the features which are used for representing the samples. Consistent biclusterings allow to obtain correct classifications of the samples from the known classification of the features, and vice versa, and they are very useful for performing supervised classifications. The problem of finding consistent biclusterings can be seen as a feature selection problem, where the features that are not relevant for classification purposes are removed from the set of data, while the total number of features is maximized in order to preserve information. This feature selection problem can be formulated as a linear fractional 0-1 optimization problem. We propose a reformulation of this problem as a bilevel optimization problem, and we present a heuristic algorithm for an efficient solution of the reformulated problem. Computational experiments show that the presented algorithm is able to find better solutions with respect to the ones obtained by employing previously presented heuristic algorithms.

Motivation & Objective

• To address the NP-hard feature selection problem in consistent biclustering, where irrelevant features degrade classification accuracy.
• To reformulate the 0–1 linear fractional optimization problem as a bilevel program to enable more effective heuristic solution strategies.
• To maximize the number of selected features while preserving classification accuracy, ensuring consistency between sample and feature clusterings.
• To validate the proposed heuristic on real-world gene expression data, particularly acute leukemia datasets, with a separate validation set for performance assessment.

Proposed method

• Reformulate the original 0–1 linear fractional feature selection problem as a bilevel optimization problem, with a linear program in the inner level.
• Use a deterministic algorithm to solve the continuous relaxation of the inner problem exactly at each iteration of the outer-level heuristic.
• Apply a greedy strategy to iteratively select features that improve the objective function under consistency constraints.
• Introduce two types of relaxed consistency: α-consistent and β-consistent biclustering, allowing trade-offs between feature count and classification accuracy.
• Use a validation set to evaluate classification performance, measuring misclassification errors on unseen samples.
• Implement a heuristic that balances feature selection size and classification quality by tuning α and β parameters.

Experimental results

Research questions

• RQ1 Can a bilevel optimization reformulation improve the quality of feature selection in consistent biclustering compared to existing heuristics?
• RQ2 How does the proposed heuristic perform in terms of feature count and classification accuracy on real leukemia gene expression data?
• RQ3 What is the impact of varying α and β parameters on the number of selected features and the number of misclassifications in validation sets?
• RQ4 Does the proposed method maintain consistency between sample and feature clusterings while maximizing the number of relevant features?
• RQ5 Can the heuristic efficiently handle large-scale data sets where deterministic methods are infeasible?

Key findings

• The proposed heuristic selected 7,081 features with only 2 misclassifications on the leukemia validation set when α = 0 or β = 1.00, outperforming the prior heuristic which selected 7,024 features with the same error rate.
• For α ≥ 50, the method achieved just one misclassification on the validation set, demonstrating improved robustness to noise.
• The algorithm consistently selected more features than the prior heuristic across most α and β values, except at α = 70, where it selected 7,046 features versus 6,989 in the prior method.
• The number of selected features decreased with increasing α or β values, reflecting the trade-off between consistency strength and feature count.
• Computational cost increased with higher α or β, requiring several minutes of CPU time for some experiments, indicating scalability challenges at high consistency thresholds.
• On synthetic data, the method successfully identified 7,450 features in a 7,500-feature dataset under α-consistency, demonstrating high feature retention under ideal conditions.