[Paper Review] Embedding Projector: Interactive Visualization and Interpretation of Embeddings

Daniel Smilkov, Nikhil Thorat+4 more|arXiv (Cornell University)|Nov 16, 2016

Data Visualization and AnalyticsComputer Science7 references154 citations

TL;DR

A web-based interactive tool for visualizing and interpreting high-dimensional embeddings, supporting local neighborhood exploration, global geometry inspection, and discovery of meaningful directions, with PCA, t-SNE, and custom projections, integrated with TensorFlow.

ABSTRACT

Embeddings are ubiquitous in machine learning, appearing in recommender systems, NLP, and many other applications. Researchers and developers often need to explore the properties of a specific embedding, and one way to analyze embeddings is to visualize them. We present the Embedding Projector, a tool for interactive visualization and interpretation of embeddings.

Motivation & Objective

• Provide an interactive visualization platform for high-dimensional embeddings.
• Enable exploration of local neighborhoods and global geometry to build trust in models.
• Support discovery of semantically meaningful directions within embedding spaces.
• Integrate with TensorFlow for easy visualization of tensors and model checkpoints.
• Facilitate collaboration through shareable visualization states.

Proposed method

• Web-based Embedding Projector with a 2D/3D view and WebGL for smooth interaction.
• Three dimensionality reduction options: PCA (top 10 components) and t-SNE (2D/3D).
• Custom linear projections by user-defined directions via centroid differences from text searches or regex matches.
• Linked views: data panel, projection panel, and inspector with nearest neighbors and search.
• Selection tools to isolate subsets and re-project them for focused analysis.
• Collaboration via sharable state files containing embedding coordinates and bookmarks.

Experimental results

Research questions

• RQ1 How can users effectively explore local neighborhoods around a point in high-dimensional embeddings?
• RQ2 What global geometric structures and clusters are observable in embeddings using dimensionality reduction?
• RQ3 Can users identify meaningful directional vectors in embedding space through simple, heuristic projections?
• RQ4 How does an interactive visualization tool aid trust and interpretation of machine learning embeddings?
• RQ5 How can this tool integrate with TensorFlow to visualize model checkpoints and tensors?

Key findings

• The Embedding Projector enables interactive 2D/3D visualization with smooth WebGL performance.
• It supports local neighborhood exploration via nearest-neighbors lists and interactive selection tools.
• It provides multiple views (PCA, t-SNE) to examine global geometry and clusters.
• Users can create custom projection axes by specifying search terms, revealing meaningful directions.
• The tool integrates with TensorFlow, allowing visualization of embeddings derived from model checkpoints.
• Collaborative sharing of visualization states is supported through bookmarkable view configurations.