[Paper Review] Topological Data Analysis in Text Classification: Extracting Features with Additive Information

Shafie Gholizadeh, Ketki Savle+2 more|arXiv (Cornell University)|Mar 29, 2020

Topological and Geometric Data Analysis47 references4 citations

TL;DR

This paper proposes two novel methods to extract topological features from text using topological data analysis (TDA): one treating word embeddings as high-dimensional time series and another analyzing TF-IDF vectors of text blocks via persistent homology. Results show that adding these features to conventional models boosts classification performance by up to 5.1% in F1 score, demonstrating that TDA captures exclusive, non-redundant information in text data.

ABSTRACT

While the strength of Topological Data Analysis has been explored in many studies on high dimensional numeric data, it is still a challenging task to apply it to text. As the primary goal in topological data analysis is to define and quantify the shapes in numeric data, defining shapes in the text is much more challenging, even though the geometries of vector spaces and conceptual spaces are clearly relevant for information retrieval and semantics. In this paper, we examine two different methods of extraction of topological features from text, using as the underlying representations of words the two most popular methods, namely word embeddings and TF-IDF vectors. To extract topological features from the word embedding space, we interpret the embedding of a text document as high dimensional time series, and we analyze the topology of the underlying graph where the vertices correspond to different embedding dimensions. For topological data analysis with the TF-IDF representations, we analyze the topology of the graph whose vertices come from the TF-IDF vectors of different blocks in the textual document. In both cases, we apply homological persistence to reveal the geometric structures under different distance resolutions. Our results show that these topological features carry some exclusive information that is not captured by conventional text mining methods. In our experiments we observe adding topological features to the conventional features in ensemble models improves the classification results (up to 5\%). On the other hand, as expected, topological features by themselves may be not sufficient for effective classification. It is an open problem to see whether TDA features from word embeddings might be sufficient, as they seem to perform within a range of few points from top results obtained with a linear support vector classifier.

Motivation & Objective

• To investigate whether topological data analysis (TDA) can extract meaningful, non-redundant features from text representations like word embeddings and TF-IDF vectors.
• To develop a method that treats word embeddings as high-dimensional time series to preserve sequential and semantic structure for topological analysis.
• To evaluate whether topological features derived from TDA improve text classification performance when combined with conventional features.
• To explore the potential of TDA in capturing geometric invariants in text that are not captured by standard NLP methods.
• To identify limitations and open problems in applying TDA to text, especially regarding short documents and interpretability.

Proposed method

• The first method converts a document’s sequence of word embeddings into a high-dimensional time series, treating each embedding dimension as a time point, and applies persistent homology to analyze the resulting topological structure.
• The second method divides a document into fixed-size blocks, computes TF-IDF vectors for each block, and constructs a graph where nodes are blocks and edges are based on distance thresholds, then applies persistent homology to extract topological features.
• Topological features are extracted from persistence diagrams, including birth and death diameters in dimensions 0 and 1, their derivatives (e.g., number of holes, ratio of average to standard deviation of death diameter), and duration metrics.
• The topological features are combined with conventional features (e.g., TF-IDF, word embeddings) in ensemble models to evaluate performance gains in text classification.
• The analysis uses persistent homology to detect stable topological features such as connected components (β₀) and loops (β₁) across varying distance thresholds.
• The method is applied to document classification tasks using standard NLP datasets, with evaluation on precision, recall, and F1 score.

Experimental results

Research questions

• RQ1 Can topological features extracted from word embeddings using time-series interpretation improve text classification beyond standard features?
• RQ2 Can topological analysis of TF-IDF block representations reveal geometric structures that enhance classification performance?
• RQ3 Do topological features carry exclusive information not captured by conventional text mining methods?
• RQ4 To what extent do topological features improve ensemble models when combined with traditional features?
• RQ5 What are the limitations of TDA-based features in short or low-context documents?

Key findings

• Adding topological features extracted from word embeddings or TF-IDF representations to conventional models improved the F1 score by up to 5.1%.
• The precision increased by 0.5% and recall by 4.9% when topological features were added to ensemble models.
• Topological features alone were insufficient for strong classification performance, indicating they are complementary rather than standalone.
• The method of treating embeddings as time series preserved word order and worked effectively across documents of varying lengths.
• The topological structure in short texts was unstable, suggesting limitations in low-context or short-document scenarios.
• The persistence diagrams revealed stable topological signatures such as loops and connected components, indicating the presence of latent geometric structure in text representations.