[Paper Review] Kolmogorov-Arnold Networks (KAN) for Time Series Classification and Robust Analysis

Chang Dong, Li Zheng+1 more|arXiv (Cornell University)|Aug 14, 2024

Neural Networks and Applications4 citations

TL;DR

This paper evaluates Kolmogorov-Arnold Networks (KAN) for time series classification across 128 UCR datasets, demonstrating KAN achieves performance comparable to MLPs while exhibiting superior adversarial robustness due to lower Lipschitz constants. The study reveals that the base function dominates model output, and counterintuitively, higher grid sizes in KAN improve robustness despite increasing Lipschitz constants.

ABSTRACT

Kolmogorov-Arnold Networks (KAN) has recently attracted significant attention as a promising alternative to traditional Multi-Layer Perceptrons (MLP). Despite their theoretical appeal, KAN require validation on large-scale benchmark datasets. Time series data, which has become increasingly prevalent in recent years, especially univariate time series are naturally suited for validating KAN. Therefore, we conducted a fair comparison among KAN, MLP, and mixed structures. The results indicate that KAN can achieve performance comparable to, or even slightly better than, MLP across 128 time series datasets. We also performed an ablation study on KAN, revealing that the output is primarily determined by the base component instead of b-spline function. Furthermore, we assessed the robustness of these models and found that KAN and the hybrid structure MLP\_KAN exhibit significant robustness advantages, attributed to their lower Lipschitz constants. This suggests that KAN and KAN layers hold strong potential to be robust models or to improve the adversarial robustness of other models.

Motivation & Objective

• To evaluate the performance of Kolmogorov-Arnold Networks (KAN) in time series classification (TSC) on large-scale benchmark datasets.
• To compare KAN with traditional MLPs and hybrid architectures (MLP_KAN, KAN_MLP) under identical configurations to ensure fair evaluation.
• To investigate the contribution of the base function and B-spline components in KAN through ablation studies.
• To assess the adversarial robustness of KAN and related models under PGD untargeted attacks, analyzing the role of Lipschitz constants.
• To explain the counterintuitive observation that higher grid sizes in KAN improve robustness despite increasing Lipschitz constants.

Proposed method

• KAN is applied to 128 univariate time series datasets from the UCR archive, using a fair comparison setup with MLP, MLP_KAN, KAN_MLP, and MLP_L models.
• The KAN architecture uses learnable activation functions (B-splines) on edges, with the base function and spline components explicitly analyzed via ablation.
• Adversarial robustness is evaluated using PGD untargeted attacks with $ε$ ranging from 0.05 to 0.5, measuring attack success rate (ASR) across all datasets.
• Lipschitz constants are computed and compared across models to correlate robustness with model smoothness and gradient sensitivity.
• The ablation study isolates the base function and B-spline components to assess their individual contributions to model output and optimization dynamics.
• A hypothesis is proposed to explain the paradoxical robustness of high-grid-size KANs: despite higher Lipschitz constants, the dominant base function and gradient misdirection in spline components may reduce effective attack success.

Figure 1 : A three-layer KAN structure with the architecture [3-5-3-1].

Experimental results

Research questions

• RQ1 Can KAN achieve performance comparable to MLP in time series classification across a large-scale benchmark of 128 datasets?
• RQ2 What is the relative contribution of the base function versus the B-spline function in determining KAN output?
• RQ3 Why does KAN with larger grid sizes exhibit greater adversarial robustness despite having higher Lipschitz constants?
• RQ4 How does the Lipschitz constant of KAN compare to MLP and hybrid models, and to what extent does it explain robustness differences?
• RQ5 Why is optimization more difficult for KAN with large grid sizes, and how does this affect model performance?

Key findings

• KAN achieves classification performance comparable to MLP across all 128 UCR time series datasets, with no significant performance gap observed.
• The base function contributes more significantly to the final output than the B-spline function, as shown by ablation studies where removing the base function drastically alters output distributions.
• KAN exhibits superior adversarial robustness compared to MLP, with attack success rates (ASR) below 75% on 50% of datasets at $ε = 0.5$, outperforming MLP and KAN_MLP.
• The MLP_KAN hybrid model achieves the best robustness among all models tested, with ASR below 75% on 50% of datasets at $ε = 0.5$.
• Despite having higher Lipschitz constants, KAN models with larger grid sizes show greater robustness, suggesting that gradient misdirection in spline components may reduce effective attack success.
• The B-spline function with large grid size is difficult to optimize, as evidenced by a narrow, zero-centered output distribution and lower test accuracy, indicating optimization challenges at high grid counts.

(a) Test Accuracy of five models