[论文解读] Rule Extraction using Artificial Neural Networks

S. M. Kamruzzaman, Ahmed Ryadh Hasan|arXiv (Cornell University)|Sep 25, 2010

Neural Networks and Applications参考文献 19被引用 4

一句话总结

本文提出一种两阶段神经网络训练方法，用于从训练好的人工神经网络中提取可解释的符号规则。首先，基于性能逐步添加隐藏节点构建网络；其次，剪枝无关连接和输入特征。所得简化网络可实现有效的规则提取，并在基准数据集上表现出良好的泛化能力。

ABSTRACT

Artificial neural networks have been successfully applied to a variety of business application problems involving classification and regression. Although backpropagation neural networks generally predict better than decision trees do for pattern classification problems, they are often regarded as black boxes, i.e., their predictions are not as interpretable as those of decision trees. In many applications, it is desirable to extract knowledge from trained neural networks so that the users can gain a better understanding of the solution. This paper presents an efficient algorithm to extract rules from artificial neural networks. We use two-phase training algorithm for backpropagation learning. In the first phase, the number of hidden nodes of the network is determined automatically in a constructive fashion by adding nodes one after another based on the performance of the network on training data. In the second phase, the number of relevant input units of the network is determined using pruning algorithm. The pruning process attempts to eliminate as many connections as possible from the network. Relevant and irrelevant attributes of the data are distinguished during the training process. Those that are relevant will be kept and others will be automatically discarded. From the simplified networks having small number of connections and nodes we may easily able to extract symbolic rules using the proposed algorithm. Extensive experimental results on several benchmarks problems in neural networks demonstrate the effectiveness of the proposed approach with good generalization ability.

研究动机与目标

• 通过提取人类可读的符号规则，解决训练后神经网络的黑箱问题。
• 在不牺牲预测性能的前提下，提高神经网络预测的可解释性。
• 在训练过程中自动识别并保留相关输入特征，同时丢弃无关特征。
• 开发一种系统化方法，简化神经网络以实现更易的规则提取。
• 在基准分类问题上验证该方法，展现出强大的泛化能力。

提出的方法

• 使用构造性训练阶段，根据训练数据上的网络性能动态添加隐藏节点。
• 在第二阶段应用剪枝算法，移除冗余连接并降低网络复杂度。
• 通过分析连接权重识别相关输入单元，并剔除贡献最小的单元。
• 利用简化后的网络结构，通过专用规则提取算法提取符号规则。
• 结合构造性学习与剪枝，平衡模型准确率与可解释性。
• 确保最终网络仅保留对预测至关重要的节点与连接，以提升规则清晰度。

实验结果

研究问题

• RQ1 两阶段训练过程能否有效简化神经网络以实现规则提取？
• RQ2 剪枝在多大程度上可消除无关特征，同时保持预测准确率？
• RQ3 所提取的规则在基准分类数据集上的泛化能力如何？
• RQ4 所提出方法能否在提升模型可解释性的同时保持高性能？
• RQ5 自动节点与连接选择对规则质量与网络效率有何影响？

主要发现

• 两阶段训练方法成功通过消除不必要的节点与连接，降低了网络复杂度。
• 该方法在训练过程中自动识别并保留了相关输入特征，同时剔除了无关属性。
• 从简化网络中提取的规则在基准数据集上表现出强大的泛化性能。
• 该方法在预测准确率与可解释性之间取得良好平衡，克服了标准反向传播网络的黑箱局限。
• 大量实验验证了该规则提取过程在多个分类问题中的有效性。
• 最终网络结构显著缩小且更具可解释性，从而实现了更清晰的符号规则生成。