[论文解读] seq2graph: Discovering Dynamic Dependencies from Multivariate Time Series with Multi-level Attention

Xuan-Hong Dang, Syed Yousaf Shah+1 more|arXiv (Cornell University)|Dec 7, 2018

Time Series Analysis and Forecasting参考文献 29被引用 7

一句话总结

该论文提出 seq2graph，一种深度学习模型，通过多级注意力机制和双用途门控循环单元（GRUs），在多变量时间序列中发现动态的、非线性的时序依赖关系和跨序列依赖关系。通过端到端联合训练滞后依赖检测与跨序列依赖图学习，该模型在合成数据集和真实世界数据集（包括云监控和制造传感器数据）上实现了准确、可解释的依赖图以及卓越的预测性能。

ABSTRACT

Discovering temporal lagged and inter-dependencies in multivariate time series data is an important task. However, in many real-world applications, such as commercial cloud management, manufacturing predictive maintenance, and portfolios performance analysis, such dependencies can be non-linear and time-variant, which makes it more challenging to extract such dependencies through traditional methods such as Granger causality or clustering. In this work, we present a novel deep learning model that uses multiple layers of customized gated recurrent units (GRUs) for discovering both time lagged behaviors as well as inter-timeseries dependencies in the form of directed weighted graphs. We introduce a key component of Dual-purpose recurrent neural network that decodes information in the temporal domain to discover lagged dependencies within each time series, and encodes them into a set of vectors which, collected from all component time series, form the informative inputs to discover inter-dependencies. Though the discovery of two types of dependencies are separated at different hierarchical levels, they are tightly connected and jointly trained in an end-to-end manner. With this joint training, learning of one type of dependency immediately impacts the learning of the other one, leading to overall accurate dependencies discovery. We empirically test our model on synthetic time series data in which the exact form of (non-linear) dependencies is known. We also evaluate its performance on two real-world applications, (i) performance monitoring data from a commercial cloud provider, which exhibit highly dynamic, non-linear, and volatile behavior and, (ii) sensor data from a manufacturing plant. We further show how our approach is able to capture these dependency behaviors via intuitive and interpretable dependency graphs and use them to generate highly accurate forecasts.

研究动机与目标

• 解决传统方法（如格兰杰因果关系）无法捕捉的时间可变、非线性依赖关系在多变量时间序列中的发现挑战。
• 开发一种深度学习模型，能够在推理过程中统一学习单个时间序列内的滞后依赖关系和跨序列依赖关系。
• 提供可解释的、动态的依赖图，以反映云服务和制造工厂等真实系统中关系的演化。
• 通过统一的、端到端可训练架构，利用发现的依赖关系提升预测精度。

提出的方法

• 该模型使用双用途 GRU 组件，解码时序信息以提取每个时间序列内的滞后依赖关系。
• 所有时间序列的隐藏表示被聚合，形成更高层级上用于跨依赖关系发现的输入向量。
• 多级注意力机制使模型能够同时关注相关的历史时间点和跨序列关系。
• 该架构采用多层定制化 GRUs，以捕捉序列数据中的长期非线性依赖关系。
• 整个网络采用端到端训练，实现滞后依赖与跨序列依赖学习的联合优化。
• 输出为有向加权依赖图，可视化时间序列之间的动态关系。

实验结果

研究问题

• RQ1 深度学习模型能否发现传统方法（如格兰杰因果关系）所遗漏的、多变量时间序列中的非线性、时变依赖关系？
• RQ2 如何在统一框架中联合学习单个时间序列内的滞后依赖关系与跨序列依赖关系？
• RQ3 端到端的联合训练在波动性较强的现实时间序列中，对依赖关系发现与预测精度的提升程度如何？
• RQ4 该模型能否生成可解释的依赖图，以反映系统行为的动态变化，例如在云服务监控中？

主要发现

• 在云服务数据中，seq2graph 识别出 pkg-rx 是 cpu-util 和 mem-util 的强预测因子，格兰杰因果关系检验的 p 值极低（p < 1e-120），结果可靠。
• 在预测精度方面，seq2graph 显著优于 RNN-vanilla 和 RNN-residual，尤其在云数据集中的关键时间序列（如 mem-util 和 cpu-util）上表现更优。
• seq2graph 在波动时期（如时间点 208 到 214 之间）成功捕捉到依赖关系的动态变化，而静态模型则未能识别。
• 消融实验表明，若省略滞后依赖或跨依赖组件，性能均会下降，证实了联合学习的必要性。
• 该模型揭示了与电源相关的指标（如 PowerSPHTI、VoltHTI）具有延迟影响，其影响可延伸至最新时间点之后。
• seq2graph 生成的依赖图直观且可解释，为复杂系统中的系统行为提供了深入洞察，有助于早期发现因果关系。