[Paper Review] Are KANs Effective for Multivariate Time Series Forecasting?

Xiao Han, Xinfeng Zhang+3 more|arXiv (Cornell University)|Aug 21, 2024

Stock Market Forecasting Methods7 citations

TL;DR

This paper introduces Reversible Mixture of KAN Experts (RMoK), a single-layer KAN-based model for multivariate time series forecasting, and shows it often achieves state-of-the-art results across seven real-world datasets while examining performance, integration, speed, and interpretability of KAN-based approaches.

ABSTRACT

Multivariate time series forecasting is a crucial task that predicts the future states based on historical inputs. Related techniques have been developing in parallel with the machine learning community, from early statistical learning methods to current deep learning methods. Despite their significant advancements, existing methods continue to struggle with the challenge of inadequate interpretability. The rise of the Kolmogorov-Arnold Network (KAN) provides a new perspective to solve this challenge, but current work has not yet concluded whether KAN is effective in time series forecasting tasks. In this paper, we aim to evaluate the effectiveness of KANs in time-series forecasting from the perspectives of performance, integrability, efficiency, and interpretability. To this end, we propose the Multi-layer Mixture-of-KAN network (MMK), which achieves excellent performance while retaining KAN's ability to be transformed into a combination of symbolic functions. The core module of MMK is the mixture-of-KAN layer, which uses a mixture-of-experts structure to assign variables to best-matched KAN experts. Then, we explore some useful experimental strategies to deal with the issues in the training stage. Finally, we compare MMK and various baselines on seven datasets. Extensive experimental and visualization results demonstrate that KANs are effective in multivariate time series forecasting. Code is available at: https://github.com/2448845600/EasyTSF.

Motivation & Objective

• Motivate the use of Kolmogorov-Arnold Networks (KAN) for time series forecasting to address representational and interpretability gaps in deep models.
• Propose a simple, interpretable single-layer KAN-based model (RMoK) with a mixture of KAN experts for TSF.
• Empirically compare RMoK against Transformer, CNN, and Linear baselines on seven real-world datasets.
• Examine how KAN variants interact with temporal features, dataset periodicity, and model efficiency.
• Investigate interpretability of RMoK via gating weights visualization and example analyses.

Proposed method

• Define MoK (mixture of KAN experts) layer where each expert is a KAN variant and a gating network assigns variables to experts.
• Use RevIN+ normalization, a single MoK layer for prediction, and RevIN− to denormalize outputs.
• Employ a load-balancing loss to prevent a single expert from dominating (L_load-balancing = CV(loads)^2) and combine it with MSE loss for training.
• Compare KAN-based models (including RMoK variants) with linear, CNN, and Transformer baselines on seven real-world TSF datasets.
• Integrate MoK into Transformer-based models as a plug-in to assess performance gains (e.g., replacing linear projections in attention modules).
• Visualize gating weights to analyze the relationship between temporal feature weights and data periodicity.

Figure 1 : The computational process of Linear and KAN layers under a certain input and output dimension.

Experimental results

Research questions

• RQ1 Do KAN-based models, particularly RMoK, achieve competitive or state-of-the-art performance in multivariate time series forecasting across diverse real-world datasets?
• RQ2 How do KAN variants interact with dataset size (number of variables) and temporal characteristics (periodicity, trend) in TSF?
• RQ3 Can KAN-based components be effectively integrated into Transformer-based models to improve performance?
• RQ4 What does RMoK reveal about interpretability via gating weights and the alignment of experts with temporal features?

Key findings

• RMoK achieves best results in most cases across seven real-world datasets and is competitive with or outperforms state-of-the-art baselines.
• On datasets with fewer variables, RMoK variants often excel, while on higher-variable datasets, the base MoK (RMoK-B) performs strongly, indicating the mixture-of-experts approach handles variable counts well.
• KAN-based models generally outperform Linear baselines, suggesting KAN's representation aligns well with time-series periodicity and trend.
• Integrating MoK into Transformer-based models (iTransformer with MoK) yields improved performance on several datasets, demonstrating effective plug-in capability.
• RMoK enables interpretability by depicting variable-to-expert assignments through gating weights, and analysis shows learned weights reflect data periodicity (e.g., daily cycles).
• KAN variants can closely approximate univariate forecasting behavior and convert multivariate forecasting into a set of weighted univariate tasks, aiding explainability.

Figure 2 : The structure of RMoK and MoK layer.