[Paper Review] fKAN: Fractional Kolmogorov-Arnold Networks with trainable Jacobi basis functions

Alireza Afzal Aghaei|arXiv (Cornell University)|Jun 11, 2024

Mathematical Analysis and Transform Methods4 citations

TL;DR

This paper introduces fKAN, a novel neural network architecture that replaces traditional activation functions with trainable fractional-order Jacobi basis functions within a Kolmogorov-Arnold Network (KAN) framework. By leveraging the mathematical properties of fractional Jacobi polynomials—such as simple derivatives and non-polynomial behavior—the model achieves superior performance in deep learning and physics-informed learning tasks, including regression, image denoising, and solving ordinary, partial, and fractional delay differential equations, with faster training and higher accuracy than standard KANs and other activation functions.

ABSTRACT

Recent advancements in neural network design have given rise to the development of Kolmogorov-Arnold Networks (KANs), which enhance speed, interpretability, and precision. This paper presents the Fractional Kolmogorov-Arnold Network (fKAN), a novel neural network architecture that incorporates the distinctive attributes of KANs with a trainable adaptive fractional-orthogonal Jacobi function as its basis function. By leveraging the unique mathematical properties of fractional Jacobi functions, including simple derivative formulas, non-polynomial behavior, and activity for both positive and negative input values, this approach ensures efficient learning and enhanced accuracy. The proposed architecture is evaluated across a range of tasks in deep learning and physics-informed deep learning. Precision is tested on synthetic regression data, image classification, image denoising, and sentiment analysis. Additionally, the performance is measured on various differential equations, including ordinary, partial, and fractional delay differential equations. The results demonstrate that integrating fractional Jacobi functions into KANs significantly improves training speed and performance across diverse fields and applications.

Motivation & Objective

• To develop a new neural network architecture that improves interpretability, accuracy, and training speed by integrating fractional-order orthogonal Jacobi polynomials as learnable activation functions.
• To address limitations of fixed activation functions (e.g., ReLU’s dying neuron problem) and traditional KANs by introducing a more flexible, adaptive basis function with non-polynomial behavior and simple derivatives.
• To evaluate the proposed fKAN model across diverse deep learning and physics-informed learning tasks, including regression, image classification, denoising, and solving differential equations.
• To demonstrate that fractional-order Jacobi functions can serve as effective, trainable basis functions that outperform standard KANs and other activation functions in both accuracy and convergence speed.
• To explore the feasibility and performance of using fractional derivatives in neural network layers for solving complex differential equations, including fractional delay differential equations.

Proposed method

• The fKAN architecture replaces the standard spline-based 1D functions in KANs with trainable fractional-order Jacobi basis functions, enabling adaptive, non-polynomial activation learning.
• The basis functions are derived from orthogonal Jacobi polynomials of fractional order, which offer closed-form derivative expressions and maintain activity for both positive and negative inputs.
• The network uses a functional approximation framework based on the Kolmogorov-Arnold representation theorem, where each edge applies a learned 1D function instead of a linear weight.
• The model is trained using a loss function that includes both data loss and residual loss in physics-informed learning, particularly for differential equations.
• For solving fractional differential equations, the Caputo fractional derivative is implemented via a lower-triangular operational matrix of differentiation, enabling end-to-end training with L-BFGS optimization.
• The architecture employs a multi-layer design with fJNB (fractional Jacobi Neural Basis) layers, followed by fully connected layers and a final regression head, with input dimensionality adapted to problem-specific variables (e.g., time and space in PDEs).

Figure 2: The architecture of the fractional Jacobi neural block proposed in formula ( 1 ), featuring trainable parameters $\alpha$ , $\beta$ , and $\gamma$ . This block serves as an adaptive activation function, enabling the network to learn optimal values for these parameters during training. The

Experimental results

Research questions

• RQ1 Can fractional-order Jacobi polynomials serve as effective, trainable basis functions in KANs to improve model performance across diverse deep learning tasks?
• RQ2 How does the integration of fractional Jacobi functions affect training speed, generalization, and interpretability compared to standard KANs and fixed activation functions?
• RQ3 To what extent can fKAN accurately solve complex differential equations, including ordinary, partial, and fractional delay differential equations, in physics-informed deep learning settings?
• RQ4 What are the quantitative advantages of fKAN in terms of test accuracy and convergence speed when applied to regression, image classification, and denoising tasks?
• RQ5 How do the mathematical properties of fractional Jacobi functions—such as non-polynomial behavior and simple derivatives—contribute to improved optimization dynamics in neural networks?

Key findings

• fKAN achieves significantly faster training convergence and higher test accuracy than standard KANs and other activation functions across synthetic regression, image classification, image denoising, and sentiment analysis tasks.
• The model demonstrates high accuracy in solving the Burgers' PDE, with predicted solutions closely matching the exact analytical solution for both parameter sets tested (e.g., $m_0=1, m_1=0.01, m_2=0.1$ and $m_0=0.1, m_1=0.0001, m_2=0.5$), as shown in Figure 10.
• For the fractional delay differential equation, fKAN successfully approximates the solution with low residual error, although fluctuations in residuals were observed due to the complexity of multi-fJNB architectures.
• The use of fractional-order Jacobi basis functions enables effective learning of non-polynomial, non-linear patterns, particularly in problems involving fractional derivatives.
• Despite higher time complexity and reduced interpretability compared to standard KANs due to the global nature of the basis functions, fKAN outperforms baseline models in all evaluated tasks.
• The model’s performance is robust across different network depths and widths, with consistent improvements in loss and accuracy when using trainable fractional Jacobi functions over fixed or standard KAN basis functions.

(a) Utilizing fJNB in a sequential architecture, where the value of $q$ may vary for each block. This flexibility allows for the incorporation of different fractional Jacobi basis functions with varying degrees of complexity across successive blocks in the network. By adapting the value of $q$ , the