[Paper Review] Topological Autoencoders

Michael Moor, Max Horn+2 more|arXiv (Cornell University)|Jun 3, 2019

Topological and Geometric Data AnalysisComputer Science30 references3 citations

TL;DR

This paper introduces Topological Autoencoders (TopoAE), a novel differentiable loss function that preserves topological structures—such as connected components and cycles—across data and latent spaces using persistent homology. By enabling backpropagation through topological signatures, TopoAE achieves state-of-the-art latent representations with improved topological fidelity while maintaining low reconstruction error on synthetic and real-world datasets like MNIST and CIFAR-10.

ABSTRACT

We propose a novel approach for preserving topological structures of the input space in latent representations of autoencoders. Using persistent homology, a technique from topological data analysis, we calculate topological signatures of both the input and latent space to derive a topological loss term. Under weak theoretical assumptions, we construct this loss in a differentiable manner, such that the encoding learns to retain multi-scale connectivity information. We show that our approach is theoretically well-founded and that it exhibits favourable latent representations on a synthetic manifold as well as on real-world image data sets, while preserving low reconstruction errors.

Motivation & Objective

• To address the challenge of integrating topological constraints into deep learning, where topological computations are typically non-differentiable and incompatible with backpropagation.
• To develop a differentiable topological loss that preserves multi-scale connectivity features (e.g., connected components, loops) in the latent space of autoencoders.
• To ensure theoretical stability of topological loss computation under mini-batch sampling, enabling reliable optimization.
• To empirically validate that topological preservation enhances interpretability and quality of learned representations without sacrificing reconstruction accuracy.

Proposed method

• Proposes a differentiable topological loss by approximating persistent homology using a differentiable relaxation of the Vietoris–Rips complex construction.
• Uses matrix reduction algorithms to compute homology groups and persistence diagrams in a differentiable manner via smooth approximations of discrete topological features.
• Derives a differentiable bottleneck distance between persistence diagrams of input and latent spaces to serve as the topological loss term.
• Integrates the topological loss with standard reconstruction loss (e.g., MSE) in a joint optimization objective for autoencoders.
• Applies mini-batch sampling with theoretical guarantees on stability of topological signatures, ensuring reliable gradient estimation.
• Employs a differentiable persistence diagram comparison via soft assignment of birth-death pairs to enable gradient flow through topological features.

Experimental results

Research questions

• RQ1 Can topological features such as connected components and cycles be preserved in learned latent representations of autoencoders through a differentiable loss?
• RQ2 Is it possible to backpropagate through persistent homology computations to train deep autoencoders with topological inductive biases?
• RQ3 How does the inclusion of a topological loss affect the quality and interpretability of latent representations compared to standard autoencoders and dimensionality reduction baselines?
• RQ4 Does the proposed method maintain low reconstruction error while improving topological fidelity across diverse datasets, including synthetic manifolds and real image data?

Key findings

• TopoAE achieves significantly improved topological fidelity, as measured by the ℓ-Trust and ℓ-Cont metrics, across all datasets, including SPHERES, F-MNIST, MNIST, and CIFAR-10.
• On the SPHERES dataset, TopoAE achieves a ℓ-Trust score of 0.65775 ± 0.01428, outperforming vanilla autoencoders (0.58843 ± 0.00475) and TopoPCA (0.62260 ± 0.00251).
• On MNIST, TopoAE maintains a low reconstruction error (RMSE: 19.57784 ± 0.01812) while achieving a high ℓ-Trust score of 0.92844 ± 0.00142, indicating strong preservation of topological structure.
• On CIFAR-10, TopoAE achieves a ℓ-Trust of 0.84514 ± 0.00359, outperforming vanilla autoencoders (0.86359 ± 0.00442) and TopoPCA (0.81551 ± 0.00139) in topological preservation despite higher reconstruction error.
• Visualizations (Figures A.7 and A.8) show that TopoAE produces more structured and interpretable latent spaces than standard autoencoders, t-SNE, UMAP, and PCA, particularly in preserving class-wise clustering and manifold structure.
• The method demonstrates theoretical stability of topological loss computation under mini-batch sampling, ensuring reliable and consistent optimization.