[Paper Review] Feature selection as Monte-Carlo Search in Growing Single Rooted Directed Acyclic Graph by Best Leaf Identification

Aurélien Pélissier, Atsuyoshi Nakamura+1 more|arXiv (Cornell University)|Nov 19, 2018

Artificial Intelligence in Games4 citations

TL;DR

This paper proposes a novel Monte Carlo Tree Search (MCTS) framework for feature selection by modeling the feature space as a growing single-rooted directed acyclic graph (SR-DAG), where best arm identification (BAI) and best leaf identification (BLI) are solved using confidence-bound-based exploration. The method achieves (ε, δ)-correctness with theoretical sample complexity guarantees and demonstrates competitive performance on benchmark datasets, outperforming FUSE in certain settings due to improved backpropagation and expansion policies.

ABSTRACT

Monte Carlo tree search (MCTS) has received considerable interest due to its spectacular success in the difficult problem of computer Go and also proved beneficial in a range of other domains. A major issue that has received little attention in the MCTS literature is the fact that, in most games, different actions can lead to the same state, that may lead to a high degree of redundancy in tree representation and unnecessary additional computational cost. We extend MCTS to single rooted directed acyclic graph (SR-DAG), and consider the Best Arm Identification (BAI) and the Best Leaf Identification (BLI) problem of an expanding SR-DAG of arbitrary depth. We propose algorithms that are (epsilon, delta)-correct in the fixed confidence setting, and prove an asymptotic upper bounds of sample complexity for our BAI algorithm. As a major application for our BLI algorithm, a novel approach for Feature Selection is proposed by representing the feature set space as a SR-DAG and repeatedly evaluating feature subsets until a candidate for the best leaf is returned, a proof of concept is shown on benchmark data sets.

Motivation & Objective

• To address redundancy in MCTS due to transpositions by representing game/state spaces as single-rooted directed acyclic graphs (SR-DAGs) instead of trees.
• To formalize feature selection as a Best Leaf Identification (BLI) problem in an expanding SR-DAG.
• To develop a theoretically grounded MCTS algorithm with (ε, δ)-correctness for BAI and BLI in growing SR-DAGs.
• To improve sample efficiency and reduce computational redundancy in MCTS by sharing knowledge across equivalent states via DAG structure.
• To provide a proof of concept for feature selection using the BLI-MCDS algorithm on synthetic and real-world datasets.

Proposed method

• Models the feature set space as a growing single-rooted directed acyclic graph (SR-DAG), where nodes represent feature subsets and edges represent feature addition.
• Applies a confidence-bound-based selection policy using Upper Confidence Bound (UCB) principles, but with descendant-based confidence intervals rather than parent-based ones.
• Introduces the LUCB-exMCDS algorithm for Best Arm Identification (BAI) in expanding SR-DAGs, ensuring (ε, δ)-correctness with theoretical sample complexity bounds.
• Employs an expansion policy (BAIAdd) that grows the DAG based on leaf evaluation counts, favoring promising subtrees.
• Uses an 'update-all' backpropagation policy that propagates confidence intervals to all ancestors, enhancing knowledge sharing across paths.
• Employs RAVE scores and exploration functions (β₁ theoretical, β₂ empirical) to guide node selection and expansion in the DAG.

Experimental results

Research questions

• RQ1 Can Monte Carlo Tree Search be effectively extended to single-rooted directed acyclic graphs (SR-DAGs) to reduce redundancy from transpositions in game and optimization problems?
• RQ2 Does modeling feature selection as a Best Leaf Identification (BLI) problem in an expanding SR-DAG lead to improved sample efficiency and accuracy compared to tree-based MCTS?
• RQ3 Can the proposed LUCB-exMCDS algorithm achieve (ε, δ)-correctness in BAI with provable sample complexity bounds in growing SR-DAGs?
• RQ4 How does the update-all backpropagation policy compare to update-descent in terms of convergence and performance in BLI tasks?
• RQ5 To what extent does the DAG-based MCTS framework outperform UCT-based FUSE in feature selection on benchmark datasets?

Key findings

• The LUCB-exMCDS algorithm achieves (ε, δ)-correctness for Best Arm Identification (BAI) in growing SR-DAGs with an asymptotic upper bound on sample complexity.
• The BLI-MCDS algorithm successfully identifies the best leaf in the feature DAG, with improved backpropagation via 'update-all' policy compared to FUSE's 'update-descent'.
• On the linear artificial dataset with 30 features, BLI-MCDS outperformed FUSE in finding the optimal leaf, while FUSE failed to converge on the same task.
• On the Madelon dataset (500 features), FUSE slightly outperformed BLI-MCDS due to higher leaf exploration, indicating sensitivity to expansion policy and branching factor.
• The use of RAVE scores and β₂ exploration function improved practical performance by reducing dependency on full DAG updates after each expansion.
• The framework demonstrates that DAG representation in MCTS reduces redundancy and improves knowledge sharing, especially in problems with high transposition frequency like feature selection.