[Paper Review] X-TREPAN: a multi class regression and adapted extraction of comprehensible decision tree in artificial neural networks

Awudu Karim, Shangbo Zhou|arXiv (Cornell University)|Aug 30, 2015

Machine Learning and Data Classification15 references5 citations

TL;DR

X-TREPAN proposes a novel method for extracting comprehensible decision trees from trained artificial neural networks, extending the TREPAN algorithm to handle multi-class regression tasks. By adapting Single-test TREPAN and C4.5 for neural network interpretation, the approach achieves improved model interpretability without sacrificing classification accuracy, validated through empirical evaluation and statistical testing on real-world datasets.

ABSTRACT

In this work, the TREPAN algorithm is enhanced and extended for extracting decision trees from neural networks. We empirically evaluated the performance of the algorithm on a set of databases from real world events. This benchmark enhancement was achieved by adapting Single-test TREPAN and C4.5 decision tree induction algorithms to analyze the datasets. The models are then compared with X-TREPAN for comprehensibility and classification accuracy. Furthermore, we validate the experimentations by applying statistical methods. Finally, the modified algorithm is extended to work with multi-class regression problems and the ability to comprehend generalized feed forward networks is achieved.

Motivation & Objective

• To improve the interpretability of trained artificial neural networks by extracting comprehensible decision trees.
• To extend the TREPAN algorithm for multi-class regression problems.
• To maintain high classification accuracy while enhancing model explainability.
• To validate the method using statistical analysis and benchmarking against existing decision tree and neural network models.
• To enable the interpretation of generalized feedforward neural networks through adapted decision tree extraction.

Proposed method

• Adaptation of the Single-test TREPAN algorithm to extract decision trees from trained neural networks.
• Integration of C4.5 decision tree induction techniques to improve tree comprehensibility and performance.
• Modification of the extraction process to support multi-class regression tasks.
• Application of statistical methods to validate experimental results and model robustness.
• Use of empirical evaluation on real-world datasets to compare X-TREPAN with baseline models.
• Design of a framework that maps neural network activations to interpretable decision rules.

Experimental results

Research questions

• RQ1 Can the TREPAN algorithm be effectively extended to support multi-class regression in artificial neural networks?
• RQ2 How does the comprehensibility of decision trees extracted via X-TREPAN compare to baseline methods in real-world datasets?
• RQ3 What is the trade-off between classification accuracy and interpretability in the proposed approach?
• RQ4 To what extent does X-TREPAN preserve the predictive performance of the original neural network?
• RQ5 How do statistical validation methods confirm the reliability of the extracted decision trees?

Key findings

• X-TREPAN successfully extends the TREPAN algorithm to support multi-class regression problems in artificial neural networks.
• The extracted decision trees maintain high classification accuracy while significantly improving model interpretability.
• Empirical evaluation on real-world datasets confirms the effectiveness of the adapted Single-test TREPAN and C4.5 methods.
• Statistical validation supports the reliability and robustness of the experimental results.
• The method enables the comprehension of generalized feedforward neural networks through structured, human-readable decision rules.
• The approach demonstrates a balanced trade-off between predictive performance and model transparency.