[Paper Review] Out-of-sample Extension for Latent Position Graphs

Minh Tang, Youngser Park+1 more|arXiv (Cornell University)|May 21, 2013

Data Management and Algorithms36 references3 citations

TL;DR

This paper proposes an out-of-sample extension for latent position graph embeddings, enabling efficient and consistent classification of new vertices without re-embedding the entire graph. It proves that the embedding of out-of-sample vertices converges to their true latent positions under the latent position model, ensuring universally consistent inference with high probability as the graph size grows.

ABSTRACT

We consider the problem of vertex classification for graphs constructed from the latent position model. It was shown previously that the approach of embedding the graphs into some Euclidean space followed by classification in that space can yields a universally consistent vertex classifier. However, a major technical difficulty of the approach arises when classifying unlabeled out-of-sample vertices without including them in the embedding stage. In this paper, we studied the out-of-sample extension for the graph embedding step and its impact on the subsequent inference tasks. We show that, under the latent position graph model and for sufficiently large $n$, the mapping of the out-of-sample vertices is close to its true latent position. We then demonstrate that successful inference for the out-of-sample vertices is possible.

Motivation & Objective

• To address the computational inefficiency of re-embedding graphs when classifying new, unlabeled vertices (out-of-sample vertices) in latent position graph models.
• To establish theoretical guarantees for the accuracy of out-of-sample vertex embeddings in the context of vertex classification.
• To demonstrate that the out-of-sample extension preserves the consistency of downstream inference tasks such as classification.
• To provide a computationally efficient alternative to full spectral re-computation for each new vertex.

Proposed method

• Proposes an out-of-sample extension mapping based on the adjacency spectral embedding of the graph, using the eigendecomposition of the adjacency matrix A.
• Defines the embedding of out-of-sample vertices via a linear transformation derived from the empirical kernel and projection operators in a reproducing kernel Hilbert space.
• Uses the isometric isomorphism between the empirical and true projection subspaces to bound the distance between the estimated and true latent positions.
• Applies concentration inequalities and spectral convergence results to control the error in the embedding of out-of-sample vertices.
• Establishes that the out-of-sample embedding converges to the true latent position at a rate of $ O\left(\delta_d^{-3} \sqrt{\frac{\log(n/\eta)}{n\rho_n}} \right) $, where $ \delta_d $ is the spectral gap and $ \rho_n $ controls edge density.
• Leverages results from random matrix theory and kernel methods to show that the mapping preserves geometric structure necessary for consistent classification.

Experimental results

Research questions

• RQ1 Can out-of-sample vertices be embedded into the latent space of a graph without re-computing the full spectral decomposition?
• RQ2 How close is the out-of-sample embedding to the true latent position of the vertex under the latent position model?
• RQ3 Does the proposed out-of-sample extension preserve the universal consistency of vertex classification?
• RQ4 What is the rate of convergence of the out-of-sample embedding to the true latent position?
• RQ5 Can the method be applied efficiently to large-scale graphs with streaming or incremental vertex addition?

Key findings

• The out-of-sample embedding of a vertex converges in probability to its true latent position at a rate of $ O\left(\delta_d^{-3} \sqrt{\frac{\log(n/\eta)}{n\rho_n}} \right) $, where $ \delta_d $ is the spectral gap and $ \rho_n $ controls edge density.
• The proposed out-of-sample extension ensures that the classification of new vertices remains universally consistent, even when they are not included in the initial embedding.
• The mapping from the empirical to the true latent space is shown to be an isometric isomorphism, preserving geometric structure necessary for inference.
• The error bound is derived using concentration inequalities and spectral convergence results, with high probability $ 1 - \eta $.
• The method avoids the $ O(n^3) $ cost of full spectral re-computation by using a linear mapping based on the existing embedding.
• Theoretical guarantees hold under the latent position model for sufficiently large $ n $, ensuring reliable inference on new vertices.