[Paper Review] A Nested Genetic Algorithm for Explaining Classification Data Sets with Decision Rules

Paul-Amaury Matt, Rosina Ziegler+3 more|arXiv (Cornell University)|Aug 23, 2022

Rough Sets and Fuzzy Logic4 citations

TL;DR

This paper proposes a nested genetic algorithm, termed the Quadratic Genetic Algorithm (QGA), to automatically extract concise, accurate, and high-coverage decision rule sets from classification data sets. By combining a heuristic initialization (RF+HC) with two inner genetic algorithms that enhance solution feasibility and fitness, QGA achieves up to 98.6% coverage with minimal rule complexity and errors, outperforming baseline methods on 10 public data sets.

ABSTRACT

Our goal in this paper is to automatically extract a set of decision rules (rule set) that best explains a classification data set. First, a large set of decision rules is extracted from a set of decision trees trained on the data set. The rule set should be concise, accurate, have a maximum coverage and minimum number of inconsistencies. This problem can be formalized as a modified version of the weighted budgeted maximum coverage problem, known to be NP-hard. To solve the combinatorial optimization problem efficiently, we introduce a nested genetic algorithm which we then use to derive explanations for ten public data sets.

Motivation & Objective

• To automatically extract a compact, accurate, and high-coverage set of decision rules that explain a classification data set.
• To address the NP-hard nature of selecting optimal rule sets by formulating it as a modified weighted budgeted maximum coverage problem.
• To overcome limitations of post-hoc explainability methods by generating inherently interpretable (white-box) rule-based models.
• To improve upon existing rule extraction techniques by integrating heuristic initialization with multi-level genetic optimization for better coverage and feasibility.

Proposed method

• The method begins with rule extraction from decision trees trained on the data, forming an initial pool of candidate rules.
• A heuristic approach (RF+HC) is used to generate an initial population of rule sets with high coverage and low complexity.
• A nested genetic algorithm architecture is employed, with two inner genetic algorithms: one to sample feasible solutions to inequality constraints (Ax ≤ b), and another to generate beneficial mutations.
• The algorithm optimizes a fitness function balancing coverage, accuracy, rule count, and inconsistency, using a quadratic unconstrained binary optimization (QUBO) formulation.
• The inner genetic algorithms iteratively refine solutions, improving coverage while maintaining feasibility and reducing computational time.
• The final rule set is selected based on the highest fitness score across 30 runs, ensuring robustness and generalization.

Experimental results

Research questions

• RQ1 Can a nested genetic algorithm effectively solve the NP-hard problem of selecting an optimal rule set for classification data sets?
• RQ2 How does the integration of inner genetic algorithms improve solution feasibility and coverage compared to standard evolutionary approaches?
• RQ3 To what extent can the proposed method achieve higher coverage with lower rule complexity and error count than baseline rule extraction techniques?
• RQ4 Does the use of a heuristic-initialized population combined with multi-level genetic optimization lead to more scalable and robust rule set generation?

Key findings

• The QGA achieved a maximum coverage of 98.6% on the Tic-tac-toe data set, significantly outperforming RF+HC (96.3%) and IRFRE (14.4%) in coverage.
• On the Blood Transfusion data set, QGA reached 99.1% accuracy with 96.9/100 coverage and only 149.4/150 errors, outperforming RF+HC (97.9% accuracy, 96.4/100 coverage).
• The QGA reduced computation time compared to IRFRE and IRFRE RF+HC, achieving high coverage in under 120 seconds on most data sets.
• The nested genetic algorithm successfully overcame the issue of infeasible solutions in standard genetic algorithms, enabling consistent fitness improvement across generations.
• On the Mammographic Mass data set, QGA achieved 97.3% accuracy with 84.6/90 coverage and 148.7/150 errors, outperforming IRFRE (81.9% accuracy, 46.6/90 coverage).
• The fitness evolution curves show consistent improvement over generations, with minimum, average, and maximum fitness scores converging toward 100% in all data sets.