[Paper Review] Harmonic analysis on directed graphs and applications: from Fourier analysis to wavelets

Harry Sevi, Gabriel Rilling+1 more|arXiv (Cornell University)|Nov 28, 2018

Neural Networks and Applications4 citations

TL;DR

This paper introduces a novel harmonic analysis framework for directed graphs using the random walk operator's eigenvectors as a non-orthogonal Fourier basis, with frequency interpretation via Dirichlet energy and eigenvalue real parts. It extends diffusion wavelets to directed graphs, enabling redundant and decimated wavelet transforms, and demonstrates strong performance in semi-supervised learning and signal modeling on directed graphs like political blogs.

ABSTRACT

We introduce a novel harmonic analysis for functions defined on the vertices of a strongly connected directed graph of which the random walk operator is the cornerstone. As a first step, we consider the set of eigenvectors of the random walk operator as a non-orthogonal Fourier-type basis for functions over directed graphs. We found a frequency interpretation by linking the variation of the eigenvectors of the random walk operator obtained from their Dirichlet energy to the real part of their associated eigenvalues. From this Fourier basis, we can proceed further and build multi-scale analyses on directed graphs. We propose both a redundant wavelet transform and a decimated wavelet transform by extending the diffusion wavelets framework by Coifman and Maggioni for directed graphs. The development of our harmonic analysis on directed graphs thus leads us to consider both semi-supervised learning problems and signal modeling problems on graphs applied to directed graphs highlighting the efficiency of our framework.

Motivation & Objective

• Develop a Fourier-type analysis for functions on directed graphs, where traditional graph Laplacians fail due to non-symmetry.
• Address the lack of a consistent frequency interpretation in directed graphs by linking eigenvector variation (via Dirichlet energy) to the real part of eigenvalues of the random walk operator.
• Extend multi-scale analysis (wavelets) to directed graphs by generalizing the diffusion wavelets framework to non-symmetric operators.
• Demonstrate the utility of the proposed framework in practical applications such as semi-supervised learning and graph signal modeling on real directed networks.
• Establish a foundation for graph signal processing on directed graphs that preserves key properties of undirected graph analysis, including frequency interpretation and multi-resolution analysis.

Proposed method

• Use the random walk operator on a strongly connected directed graph as the core reference operator, replacing the symmetric graph Laplacian.
• Define a frequency interpretation by associating the Dirichlet energy of each eigenvector of the random walk operator with the real part of its eigenvalue.
• Construct a non-orthogonal Fourier basis from the eigenvectors of the random walk operator, enabling spectral analysis on directed graphs.
• Develop redundant and critically sampled wavelet transforms by extending the diffusion wavelets framework to directed graphs using heat kernel-based filters.
• Implement synthesis filter banks using heat kernels: $\mathbf{H}_j = e^{-\mathcal{L}t_j}$ with $t_j = 2^j$, and bandpass filters $\mathbf{G}_j = \mathbf{H}_{j-1} - \mathbf{H}_j$.
• Apply the framework to semi-supervised learning via $\ell_1$-regularized wavelet coefficient optimization: $\boldsymbol{w}^* = \arg\min \|\tilde{\mathbf{y}} - \mathbf{X}\mathbf{w}\|_2^2 + \lambda\|\mathbf{w}\|_1$, with $\mathbf{X} = \mathbf{M}\mathbf{K}$.

Figure 1: Eigenvalue distribution of the random walk matrix of a directed toroidal graph $\mathcal{T}_{6,4}$ . Eigenvalues on the dotted green line: $\mathfrak{Re}(\lambda^{g})=-0.25,\forall\lambda^{g}\in\boldsymbol{\Lambda}_{g}$ . Eigenvalues on the dotted orange line: $\mathfrak{Re}(\lambda^{o})=0

Experimental results

Research questions

• RQ1 Can a consistent frequency interpretation be established for functions on directed graphs, given the absence of a symmetric Laplacian?
• RQ2 How can the eigenvectors of the random walk operator on directed graphs be used to define a meaningful Fourier-type basis?
• RQ3 To what extent can the diffusion wavelets framework be generalized to non-symmetric operators on directed graphs?
• RQ4 Does the proposed harmonic analysis framework outperform existing methods in semi-supervised learning tasks on directed graphs?
• RQ5 How does the sparsity of the wavelet representation affect performance in signal recovery and labeling tasks on directed graphs?

Key findings

• The Dirichlet energy of an eigenvector of the random walk operator on a directed graph correlates with the real part of its associated eigenvalue, enabling a frequency interpretation.
• The eigenvectors of the random walk operator form a non-orthogonal Fourier-type basis for functions on directed graphs, generalizing classical Fourier analysis to directed structures.
• Redundant and critically sampled wavelet transforms were successfully constructed on directed graphs by extending the diffusion wavelets framework using heat kernel filters.
• The $\ell_1$-regularized semi-supervised learning formulation using redundant wavelet coefficients achieved competitive performance compared to $\ell_2$-regularized methods on the US political blogs network.
• Despite limited spatial localization of the wavelets, the $\ell_1$-based approach showed strong performance, suggesting potential for further improvement with better localized transforms.
• The framework was validated on real directed networks such as the US political blog graph, demonstrating practical relevance and robustness across varying label proportions.

Figure 3: Semi-supervised learning on $\mathcal{G}^{\prime}$ of the political blogs of the 2004 US presidential campaign. The notations $\operatorname{L_{norm}}$ and $\operatorname{L_{norm,sym}}$ correspond respectively to $\mathcal{L}$ and $\mathcal{L}_{\operatorname{sym}}$ .