[Paper Review] LCS Graph Kernel Based on Wasserstein Distance in Longest Common Subsequence Metric Space

Jianming Huang, Zhongxi Fang+1 more|arXiv (Cornell University)|Dec 7, 2020

Advanced Graph Neural Networks63 references22 citations

TL;DR

This paper proposes a novel graph kernel, LCS-GK, that leverages the Longest Common Subsequence (LCS) similarity to compute a Wasserstein distance in a metric space of path sequences, reducing over-smoothing and computational cost. By replacing the traditional R-convolution framework with optimal transport based on LCS-derived ground distances, the method improves graph similarity measurement, especially for paths of varying lengths, and introduces adjacent point merging to sparsify representations, achieving state-of-the-art performance on benchmark datasets.

ABSTRACT

For graph learning tasks, many existing methods utilize a message-passing mechanism where vertex features are updated iteratively by aggregation of neighbor information. This strategy provides an efficient means for graph features extraction, but obtained features after many iterations might contain too much information from other vertices, and tend to be similar to each other. This makes their representations less expressive. Learning graphs using paths, on the other hand, can be less adversely affected by this problem because it does not involve all vertex neighbors. However, most of them can only compare paths with the same length, which might engender information loss. To resolve this difficulty, we propose a new Graph Kernel based on a Longest Common Subsequence (LCS) similarity. Moreover, we found that the widely-used R-convolution framework is unsuitable for path-based Graph Kernel because a huge number of comparisons between dissimilar paths might deteriorate graph distances calculation. Therefore, we propose a novel metric space by exploiting the proposed LCS-based similarity, and compute a new Wasserstein-based graph distance in this metric space, which emphasizes more the comparison between similar paths. Furthermore, to reduce the computational cost, we propose an adjacent point merging operation to sparsify point clouds in the metric space.

Motivation & Objective

• To address over-smoothing in graph neural networks caused by message-passing mechanisms that aggregate excessive neighbor information.
• To overcome the limitation of existing path-based graph kernels that require equal-length paths for comparison, leading to information loss.
• To improve graph similarity computation by replacing the R-convolution framework with optimal transport using a novel LCS-based ground metric.
• To reduce computational cost through adjacent point merging in the LCS metric space, enabling efficient representation of path sequences.

Proposed method

• Proposes a new metric space using LCS similarity, defining the ground distance as d(x1,x2) = 1 − Flcs(x1,x2)/max(|x1|,|x2|), which satisfies metric axioms.
• Computes graph similarity using the Wasserstein distance between normalized mass vectors of path sequences, emphasizing comparisons between similar paths.
• Replaces the standard R-convolution framework with optimal transport to mitigate the impact of numerous dissimilar path pairs that degrade similarity estimation.
• Introduces an adjacent point merging operation that combines similar paths in the LCS metric space, reducing redundancy and sparsifying the path cloud representation.
• Employs the Sinkhorn algorithm to efficiently solve the optimal transport problem with the proposed ground distance matrix.
• Uses a Laplacian kernel with parameter λ to compute the final kernel value as kLCS(G1,G2) = exp(−λdG), where dG is the Wasserstein distance.

Experimental results

Research questions

• RQ1 Can LCS-based similarity enable effective comparison of graph paths with varying lengths, avoiding information loss from length constraints?
• RQ2 Does replacing the R-convolution framework with optimal transport in path-based graph kernels improve similarity estimation by downweighting dissimilar path pairs?
• RQ3 How can computational cost be reduced in path-based graph kernels without sacrificing representational expressiveness?
• RQ4 Can adjacent point merging in the LCS metric space effectively reduce redundancy while preserving structural information in path representations?

Key findings

• The proposed LCS metric satisfies all metric axioms, including positive definiteness, symmetry, and the triangle inequality, ensuring mathematical validity.
• The Wasserstein distance computed in the LCS metric space assigns less weight to dissimilar path pairs, leading to more robust and accurate graph similarity estimation.
• The adjacent point merging operation significantly reduces the number of path sequences, improving computational efficiency with minimal loss of discriminative power.
• The LCS-GK kernel achieves state-of-the-art performance on multiple benchmark graph classification datasets, outperforming existing graph kernels including WWL and Weisfeiler-Lehman.
• Empirical results show that the method is particularly effective in handling graphs with diverse path lengths and high redundancy, demonstrating superior generalization.