[Paper Review] Evaluating network partitions through visualization

Chihiro Noguchi, Tatsuro Kawamoto|arXiv (Cornell University)|Jun 3, 2019

Complex Network Analysis Techniques4 citations

TL;DR

This paper proposes a visualization method called palette diagrams to assess network partitions derived from statistical inference, enabling users to identify informative clusters when statistical criteria fail to yield unique results. By combining group filtering via α-divergence clustering and vertex ordering via Isomap, the method transforms high-dimensional group assignment probabilities into interpretable 1D or 2D visualizations that reveal modular structure and similarity among partitions.

ABSTRACT

Network clustering requires making many decisions manually, such as the number of groups and a statistical model to be used. Even after filtering using an information criterion or regularizing with a nonparametric framework, we are commonly left with multiple candidates with reasonable partitions. In the end, the user has to decide which inferred groups should be regarded as informative. Here we propose a visualization method that efficiently represents network partitioning based on statistical inference algorithms. Our non-statistical assessment procedure based on visualization helps users extract informative groups when they cannot uniquely determine significant groups on the basis of statistical assessments. The proposed visualization is also effective for use as a benchmark test of different clustering algorithms.

Motivation & Objective

• To address the challenge of selecting informative network partitions when multiple statistically plausible groupings exist.
• To develop a non-statistical visualization tool that supports human judgment in cluster selection, especially when information criteria fail to resolve ambiguity.
• To enable benchmarking and comparison of different clustering algorithms through visual inspection of partition similarity and structure.
• To reduce visual clutter in high-dimensional group assignment data by filtering redundant and similar groups.
• To optimize vertex ordering such that similar group assignments are spatially proximate, enhancing visual interpretability of modular structure.

Proposed method

• Construct a raw assignment matrix P₀ of size N×M₀, where N is the number of vertices and M₀ is the total number of groups across all partitions.
• Apply α-divergence to measure similarity between group assignment distributions, using α to tune the divergence metric (e.g., Kullback–Leibler or L2).
• Perform k-means clustering on the M₀×M₀ similarity matrix to group similar distributions into M representative clusters, with M ≤ M₀.
• Select the most representative group from each cluster by choosing the one with the highest sum of assignment probabilities across all vertices.
• Use Isomap for nonlinear dimensionality reduction to reorder vertices such that those with similar group assignment patterns are adjacent, preserving modular structure in 1D.
• Generate one-dimensional palette diagrams by stacking group assignment probabilities vertically per vertex (with shifted origin), and two-dimensional diagrams by representing each group as a colored row with depth indicating probability.

Experimental results

Research questions

• RQ1 How can network clustering results with overlapping or soft assignments be effectively visualized to support human interpretation?
• RQ2 To what extent can visual inspection of group assignment patterns help identify informative clusters when statistical criteria are inconclusive?
• RQ3 How can redundant or highly similar groups across multiple partitions be automatically filtered out to reduce visual complexity?
• RQ4 What vertex ordering strategy best preserves modular structure in visualizations of network partitions?
• RQ5 Can palette diagrams serve as a reliable benchmarking tool for comparing different clustering algorithms?

Key findings

• Palette diagrams effectively reveal modular structure in network partitions by visualizing soft cluster assignments, even when partitions are overlapping or non-hierarchical.
• Group filtering based on α-divergence clustering successfully reduces redundancy by merging similar groups, improving visual clarity without loss of essential structure.
• Vertex ordering via Isomap significantly enhances visual interpretability by aligning vertices with similar assignment patterns, facilitating detection of coherent modules.
• The one-dimensional palette diagram allows clearer visualization of group boundaries and overlaps, while the two-dimensional version better reveals similarities in assignment patterns across groups.
• The method enables non-statistical assessment of network partitions, supporting user judgment when statistical criteria fail to yield unique solutions.
• The approach is robust across diverse network types, as demonstrated on the jazz musician collaboration network and synthetic examples.