[Paper Review] Flow-based Alignment Approaches for Probability Measures in Different Spaces

Tam Le, Nhat Ho+1 more|arXiv (Cornell University)|Oct 10, 2019

Generative Adversarial Networks and Image Synthesis64 references4 citations

TL;DR

This paper proposes FlowAlign and DepthAlign, novel discrepancies for comparing probability measures in different metric spaces by leveraging tree-structured flows instead of pairwise distances. By aligning flows from a root to each support, the methods achieve fast, scalable computation similar to univariate optimal transport, with theoretical guarantees as pseudo-distances and empirical superiority over baselines in benchmark tasks.

ABSTRACT

Gromov-Wasserstein (GW) is a powerful tool to compare probability measures whose supports are in different metric spaces. GW suffers however from a computational drawback since it requires to solve a complex non-convex quadratic program. We consider in this work a specific family of cost metrics, namely extit{tree metrics} for a space of supports of each probability measure, and aim for developing efficient and scalable discrepancies between the probability measures. By leveraging a tree structure, we propose to align extit{flows} from a root to each support instead of pair-wise tree metrics of supports, i.e., flows from a support to another, in GW. Consequently, we propose a novel discrepancy, named Flow-based Alignment (\FlowAlign), by matching the flows of the probability measures. We show that \FlowAlign~shares a similar structure as a univariate optimal transport distance. Therefore, \FlowAlign~is fast for computation and scalable for large-scale applications. By further exploring tree structures, we propose a variant of \FlowAlign, named Depth-based Alignment (\DepthAlign), by aligning the flows hierarchically along each depth level of the tree structures. Theoretically, we prove that both \FlowAlign~and \DepthAlign~are pseudo-distances. Moreover, we also derive tree-sliced variants, computed by averaging the corresponding \FlowAlign~/ \DepthAlign~using random tree metrics, built adaptively in spaces of supports. Empirically, we test our proposed discrepancies against other baselines on some benchmark tasks.

Motivation & Objective

• Address the computational inefficiency of Gromov-Wasserstein (GW) distance in large-scale settings due to its non-convex quadratic programming nature.
• Develop scalable and efficient discrepancies for comparing probability measures when supports lie in different metric spaces.
• Leverage tree metrics on support spaces to enable fast computation while preserving structural information.
• Propose FlowAlign by aligning flows from a root to each support, avoiding costly pairwise comparisons.
• Extend to DepthAlign for hierarchical, depth-level flow alignment and derive tree-sliced variants for robustness.

Proposed method

• Define tree metrics on the supports of probability measures to enable efficient computation.
• Propose FlowAlign by modeling flows from a root node to each support, reducing pairwise comparisons in GW.
• Formulate FlowAlign as a univariate optimal transport-like problem, enabling fast Sinkhorn-style computation.
• Introduce DepthAlign by hierarchically aligning flows at each depth level of the tree structure.
• Derive tree-sliced variants by averaging FlowAlign and DepthAlign over multiple randomly sampled tree metrics.
• Use adaptive tree metric sampling (e.g., clustering-based) to construct stable and informative tree structures.

Experimental results

Research questions

• RQ1 Can flow-based alignment on tree-structured supports provide a scalable alternative to Gromov-Wasserstein for comparing probability measures in different spaces?
• RQ2 Does aligning flows from a root to each support preserve structural information while reducing computational complexity?
• RQ3 Can hierarchical depth-level alignment (DepthAlign) improve alignment accuracy compared to flow-based alignment?
• RQ4 How do tree-sliced variants of FlowAlign and DepthAlign perform in practice compared to existing baselines?
• RQ5 What is the empirical relationship between FlowAlign, DepthAlign, and other discrepancies like sliced GW and entropic GW?

Key findings

• FlowAlign and DepthAlign are proven to be pseudo-distances, satisfying symmetry and the triangle inequality.
• FlowAlign achieves computation speed comparable to univariate optimal transport due to its structural similarity.
• Tree-sliced variants of FlowAlign and DepthAlign show robust performance across diverse datasets, including high-dimensional ones.
• Empirical results demonstrate that FlowAlign and DepthAlign outperform or match baselines like sliced GW and entropic GW in benchmark tasks.
• On high-dimensional datasets (e.g., TWITTER, CLASSIC, AMAZON), FlowAlign and DepthAlign show better structural capture than one-dimensional projections used in sliced GW.
• Empirical relations show limited correlation between FlowAlign/DepthAlign and entropic GW variants, indicating distinct behavior and potential advantages in capturing complex structures.