[Paper Review] REx: An Efficient Rule Generator

S. M. Kamruzzaman|arXiv (Cornell University)|Sep 25, 2010

Data Mining Algorithms and Applications12 references3 citations

TL;DR

REx is a recursive, rule-based algorithm that extracts concise, interpretable classification rules from trained artificial neural networks (ANNs) by leveraging first-order data information to identify minimal sufficient conditions for class discrimination. It generates perfect, order-insensitive rules without relying on weights, achieving high accuracy and generalization on benchmark datasets like breast cancer and iris with error rates matching the original data's inconsistency level.

ABSTRACT

This paper describes an efficient algorithm REx for generating symbolic rules from artificial neural network (ANN). Classification rules are sought in many areas from automatic knowledge acquisition to data mining and ANN rule extraction. This is because classification rules possess some attractive features. They are explicit, understandable and verifiable by domain experts, and can be modified, extended and passed on as modular knowledge. REx exploits the first order information in the data and finds shortest sufficient conditions for a rule of a class that can differentiate it from patterns of other classes. It can generate concise and perfect rules in the sense that the error rate of the rules is not worse than the inconsistency rate found in the original data. An important feature of rule extraction algorithm, REx, is its recursive nature. They are concise, comprehensible, order insensitive and do not involve any weight values. Extensive experimental studies on several benchmark classification problems, such as breast cancer, iris, season, and golf-playing, demonstrate the effectiveness of the proposed approach with good generalization ability.

Motivation & Objective

• To develop an efficient, recursive rule extraction method from trained artificial neural networks that produces interpretable and verifiable classification rules.
• To generate concise rules that are order-insensitive and do not depend on network weight values.
• To ensure extracted rules achieve error rates no worse than the inconsistency rate present in the original training data.
• To improve interpretability and modularity of ANN-derived knowledge for domain experts.
• To demonstrate strong generalization performance across diverse benchmark classification problems.

Proposed method

• REx recursively explores the input space using first-order information to identify minimal sufficient conditions that distinguish a target class from others.
• It constructs rules by identifying the shortest set of feature conditions that correctly classify instances of a given class.
• The algorithm is order-insensitive, meaning rule quality does not depend on the sequence of feature evaluation.
• It avoids reliance on weight values, focusing instead on logical and structural patterns in the data.
• The rule generation process is guided by the consistency of the data, ensuring extracted rules are perfect (i.e., error rate matches data inconsistency).
• The method is applied iteratively to each class, generating a set of non-reduundant, comprehensible rules.

Experimental results

Research questions

• RQ1 Can a rule extraction algorithm generate concise, interpretable rules from trained ANNs without relying on weight values?
• RQ2 How effectively can a recursive, first-order approach identify minimal sufficient conditions for class discrimination?
• RQ3 To what extent does the extracted rule set maintain the error rate of the original data, ensuring perfect rule generation?
• RQ4 How does the rule set perform in terms of generalization across diverse benchmark classification problems?
• RQ5 Can the rule extraction process be made order-insensitive while preserving accuracy and interpretability?

Key findings

• REx successfully generated concise and perfect classification rules across multiple benchmark datasets, including breast cancer, iris, season, and golf-playing.
• The error rate of the extracted rules matched the inconsistency rate of the original training data, confirming rule perfection.
• The algorithm demonstrated strong generalization ability on unseen data, indicating robustness.
• Rules were found to be comprehensible, modular, and verifiable by domain experts due to their explicit and logical structure.
• The recursive nature of REx enabled efficient exploration of the feature space without dependency on weight values.
• The method proved effective and scalable on real-world classification problems, with minimal computational overhead.