[论文解读] A Polynomial Neural Network with Controllable Precision and Human-Readable Topology for Prediction and System Identification.

Gang Liu, Jing Wang|arXiv (Cornell University)|Apr 8, 2020

Neural Networks and Applications参考文献 20被引用 9

一句话总结

该论文提出了一种可控且可解释的多项式神经网络（CR-PNN），其结构类似于泰勒展开，能够通过网络深度精确控制近似精度，并具备透明的内部机制。CR-PNN的训练速度比标准人工神经网络（ANNs）快五倍，推理速度快十倍，在合成数据和真实世界数据上均表现出更优的泛化能力和全局优化性能。

ABSTRACT

Although artificial neural networks (ANNs) are successful, there is still a concern among many over their black box nature. Why do they work? Could we design a transparent network? This paper presents a controllable and readable polynomial neural network (CR-PNN) for approximation, prediction, and system identification. CR-PNN is simple enough to be described as one small formula, so that we can control the approximation precision and explain the internal structure of the network. CR-PNN, in fact, essentially is the fascinating Taylor expansion in the form of network. The number of layers represents precision. Derivatives in Taylor expansion are exactly imitated by error back-propagation algorithm. Firstly, we demonstrated that CR-PNN shows excellent analysis performance to the black box system through ten synthetic data with noise. Also, the results were compared with synthetic data to substantiate its search easily towards the global optimum. Secondly, it was verified, by ten real-world applications, that CR-PNN brought better generalization capability relative to the typical ANNs that approximate depended on the nonlinear activation function. Finally, 200,000 repeated experiments, with 4898 samples, demonstrated that CR-PNN is five times more efficient than typical ANN for one epoch and ten times more efficient than typical ANN for one forward-propagation. In short, compared with the traditional neural networks, the novelties and advantages of CR-PNN include readability of the internal structure, easy to find global optimal solution, lower computational complexity, and likely better robustness to real-world approximation. (We're strong believers in Open Source, and provide CR-PNN code for others. GitHub: this https URL)

研究动机与目标

• 为解决传统人工神经网络（ANNs）的黑箱特性，设计一种具有可解释、人类可读拓扑结构的模型。
• 通过网络深度实现对近似精度的精确控制，模拟泰勒级数展开的机制。
• 在真实世界预测与系统辨识任务中，提升泛化能力和鲁棒性。
• 相比标准ANNs，降低计算复杂度，加速训练与推理过程。
• 通过避免非线性激活函数网络中常见的局部极小值，促进全局优化。

提出的方法

• CR-PNN被设计为多项式网络，其中每一层对应泰勒展开中的一个高阶项，从而实现系统性的精度控制。
• 网络采用误差反向传播计算导数，直接模拟泰勒级数中的导数计算过程。
• 架构在数学上紧凑，可表示为单一公式，确保透明性与可解释性。
• 网络深度决定近似精度，每一层增加一个更高阶项。
• 模型避免使用非线性激活函数，转而依赖多项式基函数，以提升稳定性和可解释性。
• 通过在多项式系数上进行梯度下降优化训练，实现高效收敛。

实验结果

研究问题

• RQ1 能否设计一种具有透明、人类可读结构的神经网络，使其内部功能完全可解释？
• RQ2 基于泰勒展开的多项式网络在真实世界应用中是否比标准ANNs具有更好的泛化性能？
• RQ3 能否通过深度系统性地控制神经网络的近似精度，类似于泰勒级数的阶数？
• RQ4 CR-PNN在训练和推理方面是否比标准ANNs更具计算效率？
• RQ5 由于其简化且类似凸函数的结构，CR-PNN能否可靠地收敛至全局最优解？

主要发现

• 在十个真实世界应用中，CR-PNN表现出优于标准ANNs的泛化性能。
• 在200,000次重复实验中，使用4,898个样本，CR-PNN在前向传播阶段比标准ANNs快十倍。
• CR-PNN每轮训练的速度比标准ANNs快五倍，表明其计算复杂度更低。
• 该模型在十个含噪声的合成数据集上表现优异，证实了其鲁棒性与高精度近似能力。
• CR-PNN的结构支持轻松实现全局优化，避免了非线性激活函数网络中常见的局部极小值问题。
• 网络采用透明的、基于公式的结构设计，可完全解释其内部功能与近似机制。