[论文解读] Temporal Fusion Transformers for Streamflow Prediction: Value of Combining Attention with Recurrence

Sinan Rasiya Koya, Tirthankar Roy|arXiv (Cornell University)|May 21, 2023

Hydrology and Watershed Management Studies被引用 7

一句话总结

该论文评估将注意力与递归相结合的 Temporal Fusion Transformer 在流量预测中的表现，在 2,610 个全球分布的集水区优于 LSTM 和 Transformer 模型。

ABSTRACT

Over the past few decades, the hydrology community has witnessed notable advancements in streamflow prediction, particularly with the introduction of cutting-edge machine-learning algorithms. Recurrent neural networks, especially Long Short-Term Memory (LSTM) networks, have become popular due to their capacity to create precise forecasts and realistically mimic the system dynamics. Attention-based models, such as Transformers, can learn from the entire data sequence concurrently, a feature that LSTM does not have. This work tests the hypothesis that combining recurrence with attention can improve streamflow prediction. We set up the Temporal Fusion Transformer (TFT) architecture, a model that combines both of these aspects and has never been applied in hydrology before. We compare the performance of LSTM, Transformers, and TFT over 2,610 globally distributed catchments from the recently available Caravan dataset. Our results demonstrate that TFT indeed exceeds the performance benchmark set by the LSTM and Transformers for streamflow prediction. Additionally, being an explainable AI method, TFT helps in gaining insights into the streamflow generation processes.

研究动机与目标

• 通过先进的机器学习架构来捕捉长程依赖性和系统动力学，以提升径流预测的动机。
• 在全球分布的数据集上评估 LSTM、Transformer 和 Temporal Fusion Transformer (TFT) 的性能。
• 评估 TFT 作为可解释人工智能方法的可解释性，以深入了解径流生成过程。

提出的方法

• 配置并比较三种模型（LSTM、Transformer、TFT）用于径流预测。
• 以 Caravan 数据集的 2,610 个全球分布覆盖作为评估基准。
• 应用 TFT 将递归与注意力机制结合以增强预测。
• 将模型性能与基线递归与基于注意力的方法进行对比。
• 利用 TFT 的可解释性特征来解释径流生成动力学。

实验结果

研究问题

• RQ1 TFT 相对于 LSTM 和 Transformer 基线是否能改善径流预测？
• RQ2 在水文领域，将递归与注意力相结合如何影响预测精度和可靠性？
• RQ3 TFT 是否能提供关于驱动径流生成过程的可解释洞察？

主要发现

• TFT 在径流预测方面超越了 LSTM 和 Transformer 的性能基线。
• 研究表明将递归与注意力结合在水文预测任务中可带来精度提升。
• TFT 提供了可解释的 AI 能力，可洞察径流生成过程。
• 在 Caravan 数据集的 2,610 个全球分布集水区进行了评估。
• 结果支持在水文预测模型中结合注意力与递归的价值。