[Paper Review] Graph Neural Lasso for Dynamic Network Regression

Yixin Chen, Lin Meng+1 more|arXiv (Cornell University)|Jul 25, 2019

Machine Learning and ELM40 references4 citations

TL;DR

This paper proposes Graph Neural Lasso (Gnl), a novel graph neural network that jointly models dynamic relationships and temporal trends in inter-connected sequence data by extending gated diffusive units with an attention mechanism and L1-regularization. Gnl learns dynamic graph structures end-to-end and achieves state-of-the-art performance on stock and traffic prediction tasks.

ABSTRACT

The regression of multiple inter-connected sequence data is a problem in various disciplines. Formally, we name the regression problem of multiple inter-connected data entities as the "dynamic network regression" in this paper. Within the problem of stock forecasting or traffic speed prediction, we need to consider both the trends of the entities and the relationships among the entities. A majority of existing approaches can't capture that information together. Some of the approaches are proposed to deal with the sequence data, like LSTM. The others use the prior knowledge in a network to get a fixed graph structure and do prediction on some unknown entities, like GCN. To overcome the limitations in those methods, we propose a novel graph neural network, namely Graph Neural Lasso (GNL), to deal with the dynamic network problem. GNL extends the GDU (gated diffusive unit) as the base neuron to capture the information behind the sequence. Rather than using a fixed graph structure, GNL can learn the dynamic graph structure automatically. By adding the attention mechanism in GNL, we can learn the dynamic relations among entities within each network snapshot. Combining these two parts, GNL is able to model the dynamic network problem well. Experimental results provided on two networked sequence datasets, i.e., Nasdaq-100 and METR-LA, show that GNL can address the network regression problem very well and is also very competitive among the existing approaches.

Motivation & Objective

• Address the challenge of modeling both temporal trends and evolving inter-entity relationships in dynamic networks.
• Overcome limitations of fixed-structure GNNs and static graph assumptions in existing methods for sequence regression.
• Enable end-to-end learning of dynamic graph structures without requiring prior knowledge of network topology.
• Improve regression performance and generalization in networked time-series data through integrated attention and L1-regularization.
• Provide a scalable, deep learning framework for dynamic network regression applicable to finance, traffic, and neuroscience.

Proposed method

• Uses Gated Diffusive Units (Gdu) as the core recurrent cell to model temporal dynamics and latent trends in node sequences.
• Introduces an attention mechanism to dynamically aggregate information from related nodes, capturing time-varying relationships.
• Learns the graph structure end-to-end during training, eliminating the need for predefined or static network topologies.
• Applies L1-norm regularization to model parameters to reduce overfitting and accelerate convergence.
• Employs multiple optimization strategies—F-norm, piecewise derivative, proximal gradient, and accelerated proximal gradient—for training stability and efficiency.
• Combines recurrent modeling with attention-based message passing to jointly capture node-level dynamics and network-level evolution.

Experimental results

Research questions

• RQ1 Can a graph neural network effectively model both temporal trends and evolving inter-node relationships in dynamic networks?
• RQ2 Does end-to-end learning of dynamic graph structures outperform methods relying on fixed or pre-specified topologies?
• RQ3 How does the integration of attention and L1-regularization improve generalization and convergence in dynamic network regression?
• RQ4 To what extent does Gnl outperform existing baselines on real-world dynamic network datasets like Nasdaq-100 and METR-LA?
• RQ5 How do different optimization techniques affect the training dynamics and final performance of the Gnl model?

Key findings

• Gnl achieves state-of-the-art performance on the Nasdaq-100 dataset, with a 1-day MAE of 0.1321 and R2 of 0.9556 using the PW gradient method.
• On the METR-LA traffic dataset, Gnl achieves a 15-minute MAE of 2.7928 and R2 of 0.8869, demonstrating strong predictive accuracy for urban traffic speed.
• The model with L1-regularization (Gnl-PW) converges faster and generalizes better than the non-regularized variant, confirming the regularization's effectiveness.
• The attention mechanism successfully captures time-varying correlations, as shown in heatmaps that reflect known traffic patterns (e.g., rush hour flows) and stock co-movement trends.
• Training loss curves for all four optimization methods (F, PW, PG, APG) show stable convergence, indicating robust training dynamics.
• Ablation studies confirm that the combination of Gdu, attention, and L1-regularization leads to superior performance compared to ablated variants.