[Paper Review] Graph-based Time Series Clustering for End-to-End Hierarchical Forecasting

Andrea Cini, Danilo P. Mandic+1 more|arXiv (Cornell University)|May 30, 2023

Time Series Analysis and Forecasting4 citations

TL;DR

This paper proposes Hierarchical Graph Predictor (HiGP), a graph-based deep learning framework that unifies relational and hierarchical inductive biases for end-to-end time series forecasting. By modeling hierarchical structures as pyramidal graphs and using trainable graph pooling, HiGP learns cluster assignments directly from data while enforcing prediction coherency through a differentiable reconciliation layer, achieving state-of-the-art forecasting accuracy on benchmark datasets.

ABSTRACT

Relationships among time series can be exploited as inductive biases in learning effective forecasting models. In hierarchical time series, relationships among subsets of sequences induce hard constraints (hierarchical inductive biases) on the predicted values. In this paper, we propose a graph-based methodology to unify relational and hierarchical inductive biases in the context of deep learning for time series forecasting. In particular, we model both types of relationships as dependencies in a pyramidal graph structure, with each pyramidal layer corresponding to a level of the hierarchy. By exploiting modern - trainable - graph pooling operators we show that the hierarchical structure, if not available as a prior, can be learned directly from data, thus obtaining cluster assignments aligned with the forecasting objective. A differentiable reconciliation stage is incorporated into the processing architecture, allowing hierarchical constraints to act both as an architectural bias as well as a regularization element for predictions. Simulation results on representative datasets show that the proposed method compares favorably against the state of the art.

Motivation & Objective

• To unify relational and hierarchical inductive biases in deep learning for time series forecasting.
• To learn hierarchical cluster assignments directly from data using graph pooling, without relying on predefined hierarchies.
• To incorporate hierarchical constraints as both architectural bias and regularization via a differentiable reconciliation layer.
• To achieve state-of-the-art forecasting performance on multi-level hierarchical time series benchmarks.
• To demonstrate the self-supervised learning of meaningful, forecast-optimized clusters in real-world datasets.

Proposed method

• Modeling time series relationships as a weighted adjacency matrix to construct a static attributed graph structure.
• Representing hierarchical levels as pyramidal layers in a graph, with each layer corresponding to a level of aggregation.
• Applying trainable graph pooling operators to learn node clustering and hierarchical representations end-to-end.
• Integrating a differentiable reconciliation layer that enforces coherency of predictions across hierarchical levels.
• Using multi-level forecasting loss to jointly optimize cluster assignments and prediction accuracy.
• Formulating the learning objective as minimizing forecasting error at each hierarchical level, regularized by graph topology.

Experimental results

Research questions

• RQ1 Can hierarchical constraints be learned end-to-end from data using graph neural networks, rather than being predefined?
• RQ2 How does combining relational inductive biases with hierarchical inductive biases improve forecasting performance?
• RQ3 Can graph pooling operators effectively learn meaningful cluster assignments that align with forecasting objectives?
• RQ4 To what extent does differentiable reconciliation improve prediction accuracy in hierarchical forecasting?
• RQ5 Can the proposed framework serve as a self-supervised method for learning interpretable time series clusters?

Key findings

• HiGP achieves state-of-the-art forecasting accuracy on METR-LA and PEMS-BAY datasets, outperforming specialized models like DCRNN and GWNet.
• On METR-LA, HiGP (T) achieves MAE@15 of 2.68 ± 0.01, outperforming the second-best model SGP (2.69 ± 0.00).
• On PEMS-BAY, HiGP (T) achieves MAE@15 of 1.31 ± 0.00, matching the best-performing baseline SGP.
• The model learns meaningful hierarchical cluster assignments on CER-E and AQI datasets, with clusters exhibiting distinct temporal dynamics at each level.
• Cluster assignments are optimized via end-to-end training that minimizes forecasting error, resulting in aggregates that are easier to predict.
• The differentiable reconciliation layer ensures coherency of predictions across hierarchical levels, acting as a regularizer.