[论文解读] Sparse Graphical Linear Dynamical Systems

Émilie Chouzenoux, V́ıctor Elvira|arXiv (Cornell University)|Jul 6, 2023

Gaussian Processes and Bayesian InferenceComputer Science参考文献 111被引用 3

一句话总结

该论文提出DGLASSO，一种新颖的线性高斯状态空间模型联合图模型框架，整合了静态图Lasso与动态Granger依赖关系。通过块交替主要化-最小化算法，DGLASSO在合成数据与真实气象数据上实现了优越的参数估计与稀疏性恢复，其准确度与收敛性优于现有方法。

ABSTRACT

Time-series datasets are central in machine learning with applications in numerous fields of science and engineering, such as biomedicine, Earth observation, and network analysis. Extensive research exists on state-space models (SSMs), which are powerful mathematical tools that allow for probabilistic and interpretable learning on time series. Learning the model parameters in SSMs is arguably one of the most complicated tasks, and the inclusion of prior knowledge is known to both ease the interpretation but also to complicate the inferential tasks. Very recent works have attempted to incorporate a graphical perspective on some of those model parameters, but they present notable limitations that this work addresses. More generally, existing graphical modeling tools are designed to incorporate either static information, focusing on statistical dependencies among independent random variables (e.g., graphical Lasso approach), or dynamic information, emphasizing causal relationships among time series samples (e.g., graphical Granger approaches). However, there are no joint approaches combining static and dynamic graphical modeling within the context of SSMs. This work proposes a novel approach to fill this gap by introducing a joint graphical modeling framework that bridges the graphical Lasso model and a causal-based graphical approach for the linear-Gaussian SSM. We present DGLASSO (Dynamic Graphical Lasso), a new inference method within this framework that implements an efficient block alternating majorization-minimization algorithm. The algorithm's convergence is established by departing from modern tools from nonlinear analysis. Experimental validation on various synthetic data showcases the effectiveness of the proposed model and inference algorithm.

研究动机与目标

• 为现有方法在状态空间模型（SSMs）中分别处理静态与动态依赖关系的缺陷提供解决方案。
• 开发一种统一框架，将线性高斯SSMs中的静态图Lasso与动态Granger建模相结合。
• 在同时包含统计依赖与因果依赖的前提下，实现对模型参数（特别是转移矩阵A）的稀疏估计。
• 设计一种高效的推理算法，具备强收敛性保证，适用于复杂SSM的参数估计。
• 在合成数据与真实世界天气变异数据集上验证该方法，证明其性能优于当前最先进方法。

提出的方法

• 提出一种联合图模型框架，将线性高斯SSMs中的静态（图Lasso）与动态（Granger基）依赖关系相结合。
• 引入DGLASSO，一种基于块交替主要化-最小化算法的推理方法，用于在稀疏性诱导惩罚下优化似然函数。
• 采用邻近算法处理优化中的非光滑稀疏性诱导项，实现高效计算。
• 采用类似EM的结构，其中E步通过卡尔曼滤波器与RTS平滑器计算后验均值与协方差。
• 应用主要化技术构建一个上界原目标函数的代理函数，确保单调收敛。
• 利用非线性分析工具（包括Kurdyka–Łojasiewicz不等式）建立收敛性，确保在弱条件下实现全局收敛。

Figure 4 : Evolution of RMSE, F1, cNMSE and loss scores on the estimation of ${\mathbf{A}}$ (left) and ${\mathbf{P}}$ (right) by DGLASSO, as a function of hyperparameters $(\lambda_{A},\lambda_{P})$ , for dataset A (averaged on 10 runs). As a comparison, the averaged RMSE scores for $(\lambda_{A},\l

实验结果

研究问题

• RQ1 能否构建一个统一框架，联合建模线性高斯状态空间模型中的静态与动态依赖关系？
• RQ2 如何在保持可解释性与估计准确性的前提下，有效对转移矩阵A施加稀疏性？
• RQ3 所提出的DGLASSO算法是否在参数估计与图恢复方面优于现有方法（如GLASSO、rGLASSO、MMLE、GRAPHEM）？
• RQ4 所提出的块交替主要化-最小化算法在高维SSM中的收敛行为如何？
• RQ5 该方法在具有复杂时间依赖与跨变量依赖关系的真实时间序列数据上的表现如何？

主要发现

• 在所有稀疏度水平下，DGLASSO对转移矩阵A的估计RMSE最低，在s_A = 27时RMSE = 0.0605，显著优于MMLE（0.0761）与GRAPHEM（0.0446）。
• 在s_A = 10时，DGLASSO在图恢复任务中AUC达0.9814，显著优于rGLASSO（0.7494）与GLASSO（0.6667）。
• 在状态分布估计方面，DGLASSO在s_A = 10时cNMSE为9.3856×10⁻⁷，优于MMLE（2.3831×10⁻⁶）与GRAPHEM（8.9556×10⁻⁵）。
• 在预测分布似然（对数似然）方面，DGLASSO在s_A = 10时达到12,290.79，显著低于GRAPHEM的29,034.72，表明其预测精度更优。
• DGLASSO在AUC、F1与RMSE各项指标上，于所有稀疏度水平下均保持稳定收敛，并持续优于MMLE、GLASSO、rGLASSO与GRAPHEM。
• 该算法展现出优异的收敛特性，经现代非线性分析工具（包括Kurdyka–Łojasiewicz不等式）验证。

Figure 5 : Box plots for the RMSE and F1 scores when running MLEM (left) and DGLASSO (right) on 50 randomly generated LG-SSM time series, for dataset A (top) and dataset D (bottom). DGLASSO outperforms MLEM on all runs, in terms of accuracy metrics (the higher, the better), while its RMSE scores are