[论文解读] TKAN: Temporal Kolmogorov-Arnold Networks

Rémi Genet, Hugo Inzirillo|arXiv (Cornell University)|May 12, 2024

Time Series Analysis and Forecasting被引用 23

一句话总结

TKAN 将 递归 Kolmogorov-Arnold Networks 与 LSTM 风格的记忆门结合起来，以实现多步时间序列预测，在比特币市场数据上显示出鲁棒性及在长时域上优于 GRU/LSTM 的性能

ABSTRACT

Recurrent Neural Networks (RNNs) have revolutionized many areas of machine learning, particularly in natural language and data sequence processing. Long Short-Term Memory (LSTM) has demonstrated its ability to capture long-term dependencies in sequential data. Inspired by the Kolmogorov-Arnold Networks (KANs) a promising alternatives to Multi-Layer Perceptrons (MLPs), we proposed a new neural networks architecture inspired by KAN and the LSTM, the Temporal Kolomogorov-Arnold Networks (TKANs). TKANs combined the strenght of both networks, it is composed of Recurring Kolmogorov-Arnold Networks (RKANs) Layers embedding memory management. This innovation enables us to perform multi-step time series forecasting with enhanced accuracy and efficiency. By addressing the limitations of traditional models in handling complex sequential patterns, the TKAN architecture offers significant potential for advancements in fields requiring more than one step ahead forecasting.

研究动机与目标

• Motivate and extend Kolmogorov-Arnold Networks (KANs) to temporal data for multi-step forecasting.
• Integrate memory and gating mechanisms to manage temporal dependencies.
• Evaluate TKAN against GRU and LSTM on real-world financial time series.

提出的方法

• Propose TKAN as an architecture that embeds memory via RKAN layers and an LSTM-like gating mechanism.
• Define memory-enhanced transformations where phi_{l,j,i} are time-dependent and incorporate memory h_{l,i}(t).
• Use a two-layer TKAN with 5 B-spline activations to model temporal dependencies.
• Train using Adam with early stopping and learning-rate reduction on plateau; evaluate with MSE loss and R^2 metric.
• Benchmark against GRU, LSTM, and a last-value baseline on BTC/USDT hourly data from Binance (2020-2022).
• Report and analyze training/validation loss to assess stability and generalization.

实验结果

研究问题

• RQ1 Can TKAN improve multi-step ahead forecasting for temporal data compared to traditional RNNs (GRU/LSTM)?
• RQ2 Does integrating RKAN memory with gated mechanisms provide more stable training and better long-horizon accuracy?
• RQ3 How does TKAN perform on real financial time series, particularly for BTC notional values, across multiple forecast horizons?

主要发现

• TKAN maintains positive R^2 at longer horizons where GRU/LSTM become negative, e.g., at 12 steps (0.105111) and 15 steps (0.086077).
• TKAN shows lower performance variance across five runs than LSTM/GRU, indicating greater training stability.
• At 1-step ahead, GRU slightly outperforms TKAN and LSTM, but TKAN excels in mid-to-long horizons.
• All models outperform the naive last-value benchmark across horizons; TKAN remains more robust as horizon increases.
• TKAN with 5 B-spline activations demonstrates more stable convergence between training and validation loss than GRU/LSTM.