[Paper Review] Understanding the Power of Persistence Pairing via Permutation Test

Chen Cai, Yusu Wang|arXiv (Cornell University)|Jan 16, 2020

Topological and Geometric Data Analysis39 references4 citations

TL;DR

This paper introduces a permutation test to isolate and quantify the contribution of persistence pairing in persistence diagrams (PDs) for machine learning tasks. By randomly shuffling point pairings while preserving coordinates, the method decouples geometric structure from topological pairing; results show persistence pairing significantly boosts performance on shape data but offers only marginal gains on graph data, revealing its critical role in geometric feature capture.

ABSTRACT

Recently many efforts have been made to incorporate persistence diagrams, one of the major tools in topological data analysis (TDA), into machine learning pipelines. To better understand the power and limitation of persistence diagrams, we carry out a range of experiments on both graph data and shape data, aiming to decouple and inspect the effects of different factors involved. To this end, we also propose the so-called \emph{permutation test} for persistence diagrams to delineate critical values and pairings of critical values. For graph classification tasks, we note that while persistence pairing yields consistent improvement over various benchmark datasets, it appears that for various filtration functions tested, most discriminative power comes from critical values. For shape segmentation and classification, however, we note that persistence pairing shows significant power on most of the benchmark datasets, and improves over both summaries based on merely critical values, and those based on permutation tests. Our results help provide insights on when persistence diagram based summaries could be more suitable.

Motivation & Objective

• To quantify the discriminative power of persistence pairing in persistence diagrams beyond critical values.
• To address the challenge of comparing PD-based features with vectorized or kernelized summaries, which are confounded by choice of kernel or distance metric.
• To develop a method that decouples the effects of critical values and persistence pairing for empirical analysis.
• To provide empirical guidelines on when and why persistence diagrams are effective in machine learning pipelines.

Proposed method

• Propose a permutation test that shuffles persistence point pairings while preserving coordinates, creating 'fake' PDs with identical geometry but no topological structure.
• Use the same kernel (e.g., Sliced Wasserstein) on both real and permuted PDs to ensure fair comparison across models.
• Train classifiers (e.g., SVM with kernel methods) on real and fake PDs to measure performance differences and assess the value of pairing.
• Perform confusion matrix and visualization analysis (e.g., t-SNE) to compare separation ability of true vs. fake diagrams.
• Apply the method across graph datasets (IMDB, REDDIT, DD) and shape datasets (ModelNet) with multiple filtration functions (degree, Ricci, connected components).
• Use f1-score and accuracy metrics to evaluate classification performance with and without persistence pairing.

Experimental results

Research questions

• RQ1 How much additional discriminative power does persistence pairing contribute beyond the distribution of critical values in persistence diagrams?
• RQ2 In what settings—graph vs. shape data—is persistence pairing most effective?
• RQ3 Can a permutation test reliably isolate the contribution of pairing from geometric features in PDs?
• RQ4 Why do PD-based methods underperform on graph data despite theoretical stability?
• RQ5 How do the structural differences between true and fake PDs affect classifier generalization?

Key findings

• For graph classification, fake PDs (with permuted pairings) achieve performance close to real PDs—e.g., 55.2% vs. 53.6% on ModelNet-10—indicating that most discriminative power comes from critical values, not pairing.
• On shape datasets like ModelNet-10, real PDs outperform fake PDs significantly (57.2% vs. 44.3% f1-score on 'bathtub'), showing pairing enhances feature learning for geometric data.
• The permutation test successfully decouples pairing from geometry, enabling clear comparison: true PDs are consistently separable from fake ones with 85–100% accuracy across datasets.
• Confusion matrices show that fake PDs are never misclassified as true ones, and true/fake diagrams are well-separated in embedding space, indicating structural differences are detectable.
• On IMDB-B, the permutation test achieves 99.8% accuracy in distinguishing true from fake PDs using the Sliced Wasserstein kernel, confirming strong structural signals in real pairings.
• For shape classification, the improvement from pairing is most pronounced when using appropriate featurization (e.g., PerVec), suggesting pairing is most effective when combined with geometrically sensitive representations.