[Paper Review] Submodular Hypergraphs: p-Laplacians, Cheeger Inequalities and Spectral Clustering

Pan Li, Olgica Milenković|arXiv (Cornell University)|Mar 10, 2018

Advanced Graph Neural NetworksComputer Science39 references43 citations

TL;DR

The paper introduces submodular hypergraphs, defines p-Laplacians on them, proves nodal domain theorems and higher-order Cheeger inequalities, and proposes SDP and inverse power method-based spectral clustering algorithms for 1- and 2-Laplacians.

ABSTRACT

We introduce submodular hypergraphs, a family of hypergraphs that have different submodular weights associated with different cuts of hyperedges. Submodular hypergraphs arise in clustering applications in which higher-order structures carry relevant information. For such hypergraphs, we define the notion of p-Laplacians and derive corresponding nodal domain theorems and k-way Cheeger inequalities. We conclude with the description of algorithms for computing the spectra of 1- and 2-Laplacians that constitute the basis of new spectral hypergraph clustering methods.

Motivation & Objective

• Motivate clustering with higher-order vertex dependencies captured by submodular hypergraphs.
• Define p-Laplacians on submodular hypergraphs and establish nodal domain theorems.
• Derive k-way Cheeger inequalities for these p-Laplacians.
• Propose and analyze algorithms to compute 1- and 2-Laplacian spectra for clustering.
• Demonstrate empirical performance and provide implementation resources.

Proposed method

• Define submodular hypergraphs with edge-specific submodular weight functions and normalized max weights.
• Develop p-Laplacian operators for submodular hypergraphs and characterize their eigenpairs via set-theoretic and Lovász extension tools.
• Prove discrete nodal domain theorems and higher-order Cheeger inequalities linking eigenvalues to Cheeger constants.
• Propose an SDP-based algorithm to approximate the second eigenvalue for p=2 with provable guarantees (O(ζ(E)) approximation).
• Propose an inverse power method (IPM) based algorithm to approximate the second eigenvalue for p=1 with convergence guarantees.
• Discuss efficient inner-loop procedures via decomposable submodular minimization.

Experimental results

Research questions

• RQ1 How can p-Laplacians be defined for submodular hypergraphs and what are their spectral properties?
• RQ2 What are the nodal domain structures of eigenvectors of these p-Laplacians on submodular hypergraphs?
• RQ3 How do higher-order Cheeger inequalities relate the eigenvalues to multi-way conductance in submodular hypergraphs?
• RQ4 Can we devise efficient algorithms to compute the second eigenvalues for p=1 and p=2 to enable spectral clustering?
• RQ5 What are the performance guarantees of SDP- and IPM-based clustering methods on submodular hypergraphs?

Key findings

• Introduces submodular hypergraphs and assigns per-edge submodular weight functions with normalization and symmetry.
• Defines p-Laplacians for submodular hypergraphs and establishes discrete nodal domain theorems and k-way Cheeger inequalities.
• Proves that second eigenvalues λ2(p) provide tight approximations to m-k Cheeger constants via new bounds.
• Provides two clustering algorithms: an SDP-based method for λ2(2) with quadratic approximation guarantees and an IPM-based method for λ2(1) with convergence guarantees.
• Demonstrates empirical effectiveness of the IPM-based method on UC Irvine ML datasets and shares implementation code.
• Connects spectral theory with potential extensions to graph embedding, wavelets, and graph convolutional networks.