[Paper Review] Interacting with Massive Behavioral Data

Shih-Chieh Su|arXiv (Cornell University)|Aug 26, 2016

Data Visualization and Analytics5 references3 citations

TL;DR

This paper proposes the split-diffuse (SD) algorithm, which uniformly redistributes data points from any dimension reduction technique (e.g., t-SNE, MDS) into a grid-based layout while preserving local topology. The resulting topic grids enable efficient human interaction and visualization of massive behavioral data, particularly in cyber security, by allowing dynamic, time-resolved comparison of user and peer behavior across topics.

ABSTRACT

In this short paper, we propose the split-diffuse (SD) algorithm that takes the output of an existing word embedding algorithm, and distributes the data points uniformly across the visualization space. The result improves the perceivability and the interactability by the human. We apply the SD algorithm to analyze the user behavior through access logs within the cyber security domain. The result, named the topic grids, is a set of grids on various topics generated from the logs. On the same set of grids, different behavioral metrics can be shown on different targets over different periods of time, to provide visualization and interaction to the human experts. Analysis, investigation, and other types of interaction can be performed on the topic grids more efficiently than on the output of existing dimension reduction methods. In addition to the cyber security domain, the topic grids can be further applied to other domains like e-commerce, credit card transaction, customer service to analyze the behavior in a large scale.

Motivation & Objective

• To address the limitations of non-uniform, overlapping data point distributions in 2D/3D visualizations from dimension reduction techniques.
• To improve human perceivability and interactability with large-scale behavioral data by enabling uniform spatial distribution while preserving topological relationships.
• To develop a scalable, interactive visualization framework—topic grids—for analyzing user and peer behavior over time in cyber security and other domains.
• To support dynamic, time-resolved analysis of behavioral metrics by integrating a temporal dimension into the grid layout.
• To enable efficient human-driven anomaly detection by allowing intuitive interaction (e.g., hover, click) with topic-level summaries and activity overlays.

Proposed method

• The SD algorithm recursively splits data points along alternating x- and y-axes at each depth level using the median to ensure balanced partitioning.
• It assigns each data point a unique index in a 2^h × 2^h grid based on its path through recursive splits, ensuring uniform spatial distribution.
• The algorithm preserves point-wise topology by maintaining relative positional relationships (e.g., which point is on which side of another) across the transformation.
• The method is applied post-dimension reduction, making it compatible with outputs from t-SNE, MDS, PCA, and other embedding techniques.
• A 3D extension uses a 1D SD layout along the time axis to stack 2D topic grids, enabling temporal monitoring of behavioral patterns.
• Topic grids are constructed by mapping topics (from log content in 19K+ dimensional space) to grid cells, with metrics like risk or activity levels visualized per cell.

Experimental results

Research questions

• RQ1 Can a post-processing transformation of dimension-reduced data improve human perception and interaction with large-scale behavioral data?
• RQ2 How can uniform spatial distribution of data points be achieved without disrupting local topological relationships in visualization space?
• RQ3 To what extent does the topic grid framework enhance the detection of behavioral anomalies in cyber security logs?
• RQ4 Can the SD algorithm be generalized across different dimension reduction techniques and applied to diverse domains like e-commerce and credit card transactions?
• RQ5 How can temporal dynamics of user and peer behavior be effectively visualized and compared using a grid-based interaction model?

Key findings

• The split-diffuse (SD) algorithm successfully redistributes data points from t-SNE and MDS outputs into a uniform 2D grid layout while preserving local topology.
• Topic grids generated via SD enable more efficient human interaction than raw dimension reduction outputs, as demonstrated by improved visual tracking and event-based interaction (e.g., hover, click).
• The 3D extension of topic grids allows stacking 2D grids along a time axis, revealing consistent behavioral patterns or anomalies over time.
• The system supports dynamic visualization of multiple behavioral metrics—such as self-risk and peer-risk—on the same grid, enabling side-by-side comparison.
• The approach reduces visual clutter and overlap, significantly improving the perceivability of dense behavioral data in cyber security, e-commerce, and transaction monitoring contexts.
• The method is compatible with various dimension reduction techniques, including t-SNE, MDS, PCA, and spectral embedding, ensuring broad applicability.