[Paper Review] Compressive Embedding and Visualization using Graphs

Johann Paratte, Nathanaël Perraudin+1 more|arXiv (Cornell University)|Feb 19, 2017

Complex Network Analysis Techniques5 references3 citations

TL;DR

This paper introduces Compressive Embedding (CE), a scalable graph-based framework that accelerates visualization and embedding of large datasets by applying any embedding algorithm only on a small subset of data points and then diffusing the results across the full dataset using a similarity graph. The method guarantees accurate low-dimensional representations using only O(log N) samples, significantly reducing computational cost while maintaining high fidelity.

ABSTRACT

Visualizing high-dimensional data has been a focus in data analysis communities for decades, which has led to the design of many algorithms, some of which are now considered references (such as t-SNE for example). In our era of overwhelming data volumes, the scalability of such methods have become more and more important. In this work, we present a method which allows to apply any visualization or embedding algorithm on very large datasets by considering only a fraction of the data as input and then extending the information to all data points using a graph encoding its global similarity. We show that in most cases, using only $\mathcal{O}(\log(N))$ samples is sufficient to diffuse the information to all $N$ data points. In addition, we propose quantitative methods to measure the quality of embeddings and demonstrate the validity of our technique on both synthetic and real-world datasets.

Motivation & Objective

• To address the scalability bottleneck of traditional dimensionality reduction and visualization algorithms on large datasets.
• To enable accurate low-dimensional embeddings using only a small fraction of data points through graph-based information diffusion.
• To provide theoretical guarantees on sampling requirements for preserving global structure in embeddings.
• To develop new quantitative metrics for evaluating embedding quality using graph signal processing tools.
• To demonstrate the method's effectiveness on both synthetic and real-world datasets with superior runtime performance.

Proposed method

• The method uses a similarity graph to encode global data relationships, where nodes represent data points and edge weights reflect pairwise distances.
• A graph sampling scheme selects O(log N) representative data points based on energy concentration of localized graph kernels.
• Information from the sampled points is diffused across the entire graph using localized low-pass graph filters to extend embeddings to all nodes.
• The approach leverages graph signal processing tools to ensure that the diffusion preserves local and global structure.
• Transductive learning is employed to extend embeddings from sampled nodes to all data points using spectral graph filtering.
• A novel quality metric based on graph cuts and localized filters is introduced to quantitatively assess embedding fidelity.

Experimental results

Research questions

• RQ1 Can a small subset of data points be used to generate accurate low-dimensional embeddings for large datasets?
• RQ2 What is the minimum number of samples required to preserve global structure in embeddings using graph-based diffusion?
• RQ3 How can the quality of visualizations be quantitatively measured using graph-theoretic principles?
• RQ4 Can the proposed method achieve comparable or better performance than state-of-the-art algorithms like t-SNE while scaling sub-quadratically?
• RQ5 To what extent does the method remain robust to noise and complex data structures?

Key findings

• The method achieves accurate embeddings using only O(log N) samples, significantly reducing computational cost compared to full-data methods.
• Theoretical analysis proves that sampling based on localized kernel energy ensures uniform coverage of the data manifold.
• Graph-based diffusion via low-pass filters successfully preserves local and global structure in the embedded space.
• The proposed quality metrics based on graph cuts and localized filters correlate well with visual inspection and provide objective evaluation.
• Experiments on synthetic and real-world datasets show that the method scales sub-quadratically and outperforms t-SNE and LargeVis in runtime while maintaining competitive embedding quality.
• The framework is general and can be applied to any existing embedding algorithm, making it broadly applicable across data analysis tasks.