[论文解读] Learning Product Graphs Underlying Smooth Graph Signals

Muhammad Asad Lodhi, Waheed U. Bajwa|arXiv (Cornell University)|Feb 26, 2020

Advanced Graph Neural Networks参考文献 41被引用 5

一句话总结

本文提出了一种新颖的方法，通过线性规划与交替优化，从平滑图信号中学习结构化乘积图——Kronecker图、笛卡尔图和强积图。该方法通过利用低秩分解，降低了样本复杂度与计算复杂度，在最先进的图学习方法中实现了更优的性能与局部收敛性保证。

ABSTRACT

Real-world data is often times associated with irregular structures that can analytically be represented as graphs. Having access to this graph, which is sometimes trivially evident from domain knowledge, provides a better representation of the data and facilitates various information processing tasks. However, in cases where the underlying graph is unavailable, it needs to be learned from the data itself for data representation, data processing and inference purposes. Existing literature on learning graphs from data has mostly considered arbitrary graphs, whereas the graphs generating real-world data tend to have additional structure that can be incorporated in the graph learning procedure. Structure-aware graph learning methods require learning fewer parameters and have the potential to reduce computational, memory and sample complexities. In light of this, the focus of this paper is to devise a method to learn structured graphs from data that are given in the form of product graphs. Product graphs arise naturally in many real-world datasets and provide an efficient and compact representation of large-scale graphs through several smaller factor graphs. To this end, first the graph learning problem is posed as a linear program, which (on average) outperforms the state-of-the-art graph learning algorithms. This formulation is of independent interest itself as it shows that graph learning is possible through a simple linear program. Afterwards, an alternating minimization-based algorithm aimed at learning various types of product graphs is proposed, and local convergence guarantees to the true solution are established for this algorithm. Finally the performance gains, reduced sample complexity, and inference capabilities of the proposed algorithm over existing methods are also validated through numerical simulations on synthetic and real datasets.

研究动机与目标

• 为解决在底层图未知时，从平滑图信号中学习大规模、结构化图的挑战。
• 通过利用现实世界图中的固有乘积结构，降低样本、内存与计算复杂度。
• 开发一种可扩展的图学习框架，显式整合Kronecker、笛卡尔与强积结构。
• 为所提出的交替优化算法建立局部收敛性保证。

提出的方法

• 将图学习问题公式化为一个线性规划，其平均性能优于当前最先进的方法。
• 提出一种交替优化算法，用于学习乘积结构图的因子图，利用跨模式的可分性。
• 使用基于张量的表示方法，将图拉普拉斯矩阵表达为较小因子拉普拉斯矩阵的Kronecker积。
• 通过将目标函数分解为模式相关的子问题，推导出每个因子图的闭式更新。
• 证明每个因子子问题具有强凸性，从而依据块坐标下降理论，实现线性收敛速率。
• 采用基于迹的优化与矩阵浓度不等式，界定估计误差并确保鲁棒性。

实验结果

研究问题

• RQ1 图学习能否被有效重述为线性规划，以提升性能与可扩展性？
• RQ2 如何利用乘积图结构（Kronecker、笛卡尔、强积）来减少图学习中的参数数量？
• RQ3 针对学习乘积图应用的交替优化算法，可建立何种收敛性保证？
• RQ4 与任意图学习相比，引入乘积结构在多大程度上降低了样本复杂度？
• RQ5 所提出的方法在合成数据与真实世界数据集上的重构精度与推理能力表现如何？

主要发现

• 所提出的图学习线性规划公式化方法在合成数据上实现了优于当前最先进方法的平均性能。
• 由于因子子问题的强凸性，交替优化算法以线性速率局部收敛至临界点。
• 通过利用低维因子图而非直接学习完整图，该方法显著降低了样本复杂度。
• 数值模拟显示，该方法在合成与真实世界数据集上均表现出更优的推理能力与重构精度。
• 利用矩阵浓度不等式，以高概率建立了误差界，验证了对数据采样变异性的鲁棒性。
• 该方法的通用性超越了笛卡尔积图，支持Kronecker与强积结构，而先前工作不具备此能力。