[Paper Review] AutoFS: Automated Feature Selection via Diversity-aware Interactive Reinforcement Learning

Wei Fan, Kunpeng Liu+4 more|arXiv (Cornell University)|Aug 27, 2020

Evolutionary Algorithms and Applications26 references4 citations

TL;DR

This paper proposes AutoFS, an interactive reinforcement learning framework that balances effectiveness and efficiency in automated feature selection by integrating diverse, skilled trainers—K-Best and Decision Tree-based—into a multi-agent reinforcement learning setup. By identifying assertive and hesitant agents and applying a hybrid teaching strategy, AutoFS accelerates convergence and achieves superior performance over both traditional and reinforced baselines on real-world datasets.

ABSTRACT

In this paper, we study the problem of balancing effectiveness and efficiency in automated feature selection. Feature selection is a fundamental intelligence for machine learning and predictive analysis. After exploring many feature selection methods, we observe a computational dilemma: 1) traditional feature selection methods (e.g., mRMR) are mostly efficient, but difficult to identify the best subset; 2) the emerging reinforced feature selection methods automatically navigate feature space to explore the best subset, but are usually inefficient. Are automation and efficiency always apart from each other? Can we bridge the gap between effectiveness and efficiency under automation? Motivated by such a computational dilemma, this study is to develop a novel feature space navigation method. To that end, we propose an Interactive Reinforced Feature Selection (IRFS) framework that guides agents by not just self-exploration experience, but also diverse external skilled trainers to accelerate learning for feature exploration. Specifically, we formulate the feature selection problem into an interactive reinforcement learning framework. In this framework, we first model two trainers skilled at different searching strategies: (1) KBest based trainer; (2) Decision Tree based trainer. We then develop two strategies: (1) to identify assertive and hesitant agents to diversify agent training, and (2) to enable the two trainers to take the teaching role in different stages to fuse the experiences of the trainers and diversify teaching process. Such a hybrid teaching strategy can help agents to learn broader knowledge, and, thereafter, be more effective. Finally, we present extensive experiments on real-world datasets to demonstrate the improved performances of our method: more efficient than existing reinforced selection and more effective than classic selection.

Motivation & Objective

• To address the computational dilemma in automated feature selection: balancing effectiveness (finding optimal subsets) and efficiency (fast convergence).
• To improve sample efficiency in reinforcement learning-based feature selection by incorporating external, expert knowledge from skilled trainers.
• To develop a hybrid teaching strategy that diversifies training by selecting different agents (assertive vs. hesitant) and leveraging complementary strengths of multiple trainers.
• To demonstrate that combining self-exploration with interactive, diverse teacher guidance leads to faster convergence and better feature subset quality.

Proposed method

• Formulates feature selection as an interactive reinforcement learning problem with multiple agents, each representing a feature selector that takes select/deselect actions.
• Introduces two external trainers: a K-Best based trainer and a Decision Tree based trainer, each providing strategy-specific advice to agents.
• Identifies assertive agents (confident in decisions) and hesitant agents (uncertain), and selectively advises only the latter to improve learning efficiency.
• Employs a hybrid teaching strategy that dynamically assigns teaching roles to the two trainers across different learning stages to fuse diverse knowledge and enhance exploration.
• Uses experience replay with memory units storing state-action-reward transitions, and trains policy networks via Adam optimizer with ε-greedy exploration.
• Fixes the downstream task as a decision tree classifier with default scikit-learn settings for fair comparison across methods.

Experimental results

Research questions

• RQ1 Can external, skilled trainers improve the sample efficiency of reinforcement learning-based feature selection without sacrificing effectiveness?
• RQ2 How does diversifying the teaching process—through agent selection (assertive vs. hesitant) and multiple trainers—affect learning performance?
• RQ3 Does a hybrid teaching strategy that combines K-Best and Decision Tree-based guidance outperform single-trainer or baseline reinforcement learning methods?
• RQ4 To what extent can interactive reinforcement learning reduce the number of exploration steps required to find high-performing feature subsets?

Key findings

• AutoFS with the hybrid teaching strategy achieved the highest overall best accuracy across four real-world datasets, outperforming both traditional feature selection and basic reinforced feature selection (MARLFS).
• The K-Best based trainer and Decision Tree based trainer individually improved convergence speed over MARLFS, with both reaching higher accuracy than MARLFS within 1,500 exploration steps.
• The hybrid teaching strategy led to faster convergence and better final performance than using either trainer alone, demonstrating the benefit of knowledge fusion from complementary strategies.
• The method significantly reduced the number of steps required to identify high-performing feature subsets by leveraging targeted advice to hesitant agents, improving learning efficiency.