[论文解读] Selecting Efficient Features via a Hyper-Heuristic Approach

Mitra Montazeri, Mahdieh Soleymani Baghshah+1 more|arXiv (Cornell University)|Jan 20, 2016

Metaheuristic Optimization Algorithms Research参考文献 30被引用 4

一句话总结

本文提出一种超启发式方法，通过遗传算法作为监督器，从16种启发式局部搜索算子中动态选择，以实现探索与开发之间的平衡，从而实现高效的特征选择。在UCI数据集上的评估表明，该方法在准确率和收敛速度方面优于现有技术，能够根据历史表现智能地调整搜索策略。

ABSTRACT

By Emerging huge databases and the need to efficient learning algorithms on these datasets, new problems have appeared and some methods have been proposed to solve these problems by selecting efficient features. Feature selection is a problem of finding efficient features among all features in which the final feature set can improve accuracy and reduce complexity. One way to solve this problem is to evaluate all possible feature subsets. However, evaluating all possible feature subsets is an exhaustive search and thus it has high computational complexity. Until now many heuristic algorithms have been studied for solving this problem. Hyper-heuristic is a new heuristic approach which can search the solution space effectively by applying local searches appropriately. Each local search is a neighborhood searching algorithm. Since each region of the solution space can have its own characteristics, it should be chosen an appropriate local search and apply it to current solution. This task is tackled to a supervisor. The supervisor chooses a local search based on the functional history of local searches. By doing this task, it can trade of between exploitation and exploration. Since the existing heuristic cannot trade of between exploration and exploitation appropriately, the solution space has not been searched appropriately in these methods and thus they have low convergence rate. For the first time, in this paper use a hyper-heuristic approach to find an efficient feature subset. In the proposed method, genetic algorithm is used as a supervisor and 16 heuristic algorithms are used as local searches. Empirical study of the proposed method on several commonly used data sets from UCI data sets indicates that it outperforms recent existing methods in the literature for feature selection.

研究动机与目标

• 解决大规模数据集中完整特征子集评估带来的高计算复杂度问题。
• 克服固定启发式方法在探索与开发之间平衡不足的局限性。
• 基于历史表现，开发一种动态选择局部搜索算子的机制。
• 通过自适应选择搜索策略，提升特征选择中的收敛速度与分类准确率。

提出的方法

• 设计了一种超启发式框架，其中遗传算法作为监督器，从16种不同的局部搜索启发式中进行选择。
• 每种局部搜索代表一种针对特征子集空间不同区域的邻域探索策略。
• 监督器根据其历史表现记录，在每次迭代中选择最有效的局部搜索。
• 通过分类准确率评估候选特征子集的适应度，引导搜索向最优子集逼近。
• 通过动态切换启发式算子，系统在探索（多样化搜索）与开发（精细化搜索）之间保持平衡。
• 在多个UCI基准数据集上进行了实验评估，以验证其相对于最先进方法的性能表现。

实验结果

研究问题

• RQ1 超启发式框架是否能通过动态自适应局部搜索策略，提升特征子集选择的性能？
• RQ2 遗传算法监督器的引入在特征选择中如何改善探索与开发之间的权衡？
• RQ3 在准确率与收敛速度方面，所提出方法相较于现有基于启发式的特征选择技术，优势有多大？
• RQ4 动态选择启发式算子是否能加快在多样化数据集上的收敛速度？
• RQ5 该方法是否能在具有不同特征数量与样本规模的UCI基准数据集上实现良好泛化能力？

主要发现

• 所提出的超启发式方法在多个UCI数据集上实现了比现有特征选择方法更高的分类准确率。
• 由于通过动态启发式选择有效平衡了探索与开发，该方法表现出更快的收敛速度。
• 遗传算法监督器成功识别并应用了在搜索过程不同阶段最合适的局部搜索算子。
• 使用16种多样化的启发式算子使系统能够适应特征子集搜索空间的不同拓扑结构。
• 实证结果证实，该方法在准确率与收敛速度方面均优于近期最先进的方法。